issue with NTLM and some SSL Proxies

VERIFIED FIXED in Firefox 18

Status

()

--
major
VERIFIED FIXED
6 years ago
6 years ago

People

(Reporter: howardm, Assigned: mcmanus)

Tracking

({regression, verifyme})

18 Branch
mozilla21
x86
Windows XP
regression, verifyme
Points:
---

Firefox Tracking Flags

(firefox18+ verified, firefox19+ verified, firefox20+ verified, firefox21 verified, firefox-esr17 unaffected)

Details

Attachments

(5 attachments)

(Reporter)

Description

6 years ago
User Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1250.0 Iron/22.0.2150.0 Safari/537.4

Steps to reproduce:

Updated to Firefox 18.0


Actual results:

unable to access the internet through proxy. (keeps asking for login) mainly on HTTPS sites but also effects normal HTTP sites.  


Expected results:

should not ask for authentication.

Updated

6 years ago
Version: 21 Branch → 18 Branch
(Reporter)

Comment 1

6 years ago
This has been confirmed by multiple uses within the company I work at.

This issue is not apparent in Firefox 17.0.*

Comment 2

6 years ago
Same here:
Unable to access HTTPS sites through our company proxy. After the update to Firefox 18 the browser asking for login.
 
We use squid 2.7 with Basic and NTLM authentification.
Do you use a pac file, let firefox determine the system proxy configuration or have you configured a direct http(s) proxy in Firefox?

It asks for the authentication but doesn't accept them or it keep asking for multiple resources on the page but the authentication itself is accepted ? 

Could you please attach a http log where this issue happens ?
https://developer.mozilla.org/en-US/docs/HTTP_Logging
Component: Untriaged → Networking
Keywords: regression
Product: Firefox → Core

Comment 4

6 years ago
I have tried the pac file and the direct configuration of the proxy (proxy:3128)

I tried the login several times, but it looks like that the authentication not works. The squid log says also TCP_DENIED on HTTPS request.

Attached you find the http log from my firefox.

Comment 5

6 years ago
Created attachment 699926 [details]
http log proxy auth

direct proxy configuration (proxy:3128), request webpage "google.at"

Comment 6

6 years ago
Comment on attachment 699926 [details]
http log proxy auth

direct proxy configuration (proxy:3128), request webpage "google.at"
marking new based on 2 independent reports
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Reporter)

Comment 8

6 years ago
Created attachment 700225 [details]
http log proxy auth (howardm@cromwell.co.uk)

attempted to access https://google.co.uk
(Reporter)

Comment 9

6 years ago
Have been informed that this issues may be due to 'network.http.proxy.keep-alive' no longer being supported?
(Reporter)

Updated

6 years ago
Severity: normal → blocker
(Assignee)

Comment 10

6 years ago
(In reply to howardm from comment #9)
> Have been informed that this issues may be due to
> 'network.http.proxy.keep-alive' no longer being supported?

we still do keep-alives.. the pref that lets you stop that has been removed.
Severity: blocker → major
Duplicate of this bug: 828999

Comment 12

6 years ago
As per https://bugzilla.mozilla.org/show_bug.cgi?id=570283 

From what I can see for the CONNECT method no keep-alive header is sent after the proxy NTLM challenge: 

CONNECT google.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0
Host: google.com
Proxy-Authorization: NTLM <SNIPPED HASH>

In previous versions FF would send a Proxy-Connection header: 

CONNECT www.google.co.uk:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20100101 Firefox/17.0
Proxy-Connection: keep-alive
Host: www.google.co.uk
Proxy-Authorization: NTLM <SNIPPED HASH>
(Assignee)

Comment 13

6 years ago
(In reply to Chris Dewar from comment #12)
> As per https://bugzilla.mozilla.org/show_bug.cgi?id=570283 
> 
> From what I can see for the CONNECT method no keep-alive header is sent
> after the proxy NTLM challenge: 
> 
> CONNECT google.com:443 HTTP/1.1

Keep-Alive is implicit in a HTTP/1.1 request.. but maybe that's the issue here. I'd actually expect we'd see the normal connection header.

I'd like to see that squid is closing the connection after sending the 407.. do you have a packet trace for this (you can make one with wireshark)

Comment 14

6 years ago
I have the same issue for my company.

I don't know if this information will help you but I send :
#Here is the log on my squid proxy server for HTTPS :
1357832158.979      0 10.0.0.54 TCP_DENIED/407 1737 CONNECT www.google.fr:443 - NONE/- text/html
1357832159.012      0 10.0.0.54 TCP_DENIED/407 1711 CONNECT www.google.fr:443 - NONE/- text/html

#Here is the log on my squid proxy server for HTTP :
1357832185.361    117 10.0.0.54 TCP_MISS/200 21160 GET http://www.google.fr/ MY_DOMAIN\my_username DIRECT/173.194.67.94 text/html
1357832185.624      0 10.0.0.54 TCP_DENIED/407 1800 GET http://clients1.google.fr/generate_204 - NONE/- text/html
1357832185.662      0 10.0.0.54 TCP_DENIED/407 1922 GET http://clients1.google.fr/generate_204 - NONE/- text/html
1357832185.707      5 10.0.0.54 TCP_NEGATIVE_HIT/204 328 GET http://clients1.google.fr/generate_204 MY_DOMAIN\my_username NONE/- text/html
1357832185.812     53 10.0.0.54 TCP_MISS/204 395 GET http://www.google.fr/csi? MY_DOMAIN\my_username DIRECT/173.194.67.94 image/gif

(10.0.0.54 is client IP)

So Firefox doesn't send authentication to proxy for HTTPS seb site.

Comment 15

6 years ago
(In reply to Patrick McManus [:mcmanus] from comment #13)
> (In reply to Chris Dewar from comment #12)
> > As per https://bugzilla.mozilla.org/show_bug.cgi?id=570283 
> > 
> > From what I can see for the CONNECT method no keep-alive header is sent
> > after the proxy NTLM challenge: 
> > 
> > CONNECT google.com:443 HTTP/1.1
> 
> Keep-Alive is implicit in a HTTP/1.1 request.. but maybe that's the issue
> here. I'd actually expect we'd see the normal connection header.
> 
> I'd like to see that squid is closing the connection after sending the 407..
> do you have a packet trace for this (you can make one with wireshark)

Yes version of squid is HTTP/1.0 so would epxlain it - and yes squid closes the connection after the request: 

HTTP/1.0 407 Proxy Authentication Required
Server: squid/2.6.STABLE9
Date: Thu, 10 Jan 2013 13:28:06 GMT
Content-Type: text/html
Content-Length: 1265
Expires: Thu, 10 Jan 2013 13:28:06 GMT
X-Squid-Error: ERR_ACCESS_DENIED 0
Proxy-Authenticate: NTLM
Proxy-Authenticate: Basic realm="Internet via 100W"
X-Cache: MISS from apollo
Via: 1.0 apollo:8881 (squid/2.6.STABLE9)
Proxy-Connection: close

I can attach the entire trace in pcap format if it helps?
(Assignee)

Comment 16

6 years ago
(In reply to Chris Dewar from comment #15)

> Proxy-Connection: close
> 
> I can attach the entire trace in pcap format if it helps?


please do attach. I'll try and cobble something together for you to test.
Duplicate of this bug: 828937

Comment 18

6 years ago
Created attachment 700437 [details]
Shows NTLM auth failing due to non peristant connections

Firefox/18.0

Comment 19

6 years ago
For information, her we are using squid/2.7.STABLE7
(Reporter)

Comment 20

6 years ago
Here us what Bloxx (Our Proxy System Provider) has said the cause of the issue may be:

"It looks like Firefox latest update to 18.0 breaks NTLM authentication.

As they no longer send the ”Proxy-Connection: Keep-Alive” when negotiating NTLM auth squid (the core proxy service we use) responds with: “Proxy-Connection: close” at all stages of the NTLM challenge, breaking the negotiate, challenge, authenticate sequence. 

The challenge and authenticate message should always be done in the same connection but because the connection is closed firefox will now use a new connection for each stage which it should not."
(Assignee)

Updated

6 years ago
Blocks: 570283
(Assignee)

Comment 21

6 years ago
so 570283 can't be easily reverted - too much bitrot.

but the only piece that's really a problem is on the CONNNECT tunnel setup. Elsewhere it sends the normal "Connection: keep-alive" which is fine in practice - its just with the CONNECT it doesn't currently send either variant and HTTP/1.0 proxies will do the wrong thing.

So we can do that with a very targeted micro-revert of 570283 (adding in both variants as P-C is not standards defined).

I have a copy of squid 2.6 available and this seems to work, but I don't have NTLM configured with it so I'll need reporters on this bug to verify.

I'll have the CI make some builds. stay tuned.

diff --git a/netwerk/protocol/http/nsHttpConnection.cpp b/netwerk/protocol/http/nsHttpConnection.cpp
--- a/netwerk/protocol/http/nsHttpConnection.cpp
+++ b/netwerk/protocol/http/nsHttpConnection.cpp
@@ -1442,16 +1442,20 @@ nsHttpConnection::SetupProxyConnect()
 
     // CONNECT host:port HTTP/1.1
     nsHttpRequestHead request;
     request.SetMethod(nsHttp::Connect);
     request.SetVersion(gHttpHandler->HttpVersion());
     request.SetRequestURI(buf);
     request.SetHeader(nsHttp::User_Agent, gHttpHandler->UserAgent());
 
+    // a CONNECT is always persistent
+    request.SetHeader(nsHttp::Proxy_Connection, NS_LITERAL_CSTRING("keep-alive"));
+    request.SetHeader(nsHttp::Connection, NS_LITERAL_CSTRING("keep-alive"));
+
     val = mTransaction->RequestHead()->PeekHeader(nsHttp::Host);
     if (val) {
         // all HTTP/1.1 requests must include a Host header (even though it
         // may seem redundant in this case; see bug 82388).
         request.SetHeader(nsHttp::Host, nsDependentCString(val));
     }
 
     val = mTransaction->RequestHead()->PeekHeader(nsHttp::Proxy_Authorization);
(Assignee)

Updated

6 years ago
Summary: moz-proxy issue → issue with NTLM and some SSL Proxies
(Assignee)

Updated

6 years ago
Depends on: 829169

Updated

6 years ago
No longer depends on: 829169
(Assignee)

Comment 22

6 years ago
Created attachment 700613 [details] [diff] [review]
patch 0

570283 removed proxy-connection and that was the only connection oriented header we were sending on a CONNECT request. HTTP/1.0 squid didn't realize we supported K-A and therefore fell down on doing NTLM auth (which requires multiple transactions on the same connection).

This patch restores proxy-connection in that case and also adds in the standards compliant connection.
Attachment #700613 - Flags: review?(jduell.mcbugs)

Updated

6 years ago
tracking-firefox18: --- → ?
(Assignee)

Comment 23

6 years ago
(In reply to Patrick McManus [:mcmanus] from comment #21)

> 
> I'll have the CI make some builds. stay tuned.
> 

Builds will appear in http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-95a9f44c52fd

I just started them moments ago, so it could take anywhere from 30 minutes to several hours for them to appear (depending on load and platform being built).

Chris, Peter, Michael, Thomas, (anyone else with the problem), 

can you try a build from there and report on if it works for you?
Flags: needinfo?(howardm)

Updated

6 years ago
Attachment #700613 - Flags: review?(jduell.mcbugs) → review+
(Assignee)

Updated

6 years ago
status-firefox18: --- → affected
status-firefox19: --- → affected
status-firefox20: --- → affected
status-firefox21: --- → affected
status-firefox-esr17: --- → unaffected

Comment 24

6 years ago
(In reply to Patrick McManus [:mcmanus] from comment #23)
> (In reply to Patrick McManus [:mcmanus] from comment #21)
> 
> > 
> > I'll have the CI make some builds. stay tuned.
> > 
> 
> Builds will appear in
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.
> com-95a9f44c52fd
> 
> I just started them moments ago, so it could take anywhere from 30 minutes
> to several hours for them to appear (depending on load and platform being
> built).
> 
> Chris, Peter, Michael, Thomas, (anyone else with the problem), 
> 
> can you try a build from there and report on if it works for you?

Good Job! Works for me: 

After the first 407 the FF test build sends this when i request https://www.google.co.uk: 

CONNECT www.google.co.uk:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130110 Firefox/17.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: www.google.co.uk
Proxy-Authorization: NTLM <NTLM AUTH TOKEN>

To which squid keeps the connection alive until the client fully authenticates. 

Side note (just incase it helps with the reasoning behind https://bugzilla.mozilla.org/show_bug.cgi?id=570283): I'm not 100% sure you need both 
request.SetHeader(nsHttp::Proxy_Connection, NS_LITERAL_CSTRING("keep-alive"));	
AND
request.SetHeader(nsHttp::Connection, NS_LITERAL_CSTRING("keep-alive"));

Squid(squid/2.6.STABLE9) at least, will keep the connection alive even with the Connection: keep-alive header alone (it does with a standard http GET anyway)

Updated

6 years ago
tracking-firefox18: ? → +
tracking-firefox19: --- → +
tracking-firefox20: --- → +

Comment 25

6 years ago
For me also works!!!
thanks

Comment 26

6 years ago
(In reply to Patrick McManus [:mcmanus] from comment #23)
> (In reply to Patrick McManus [:mcmanus] from comment #21)
> 
> > 
> > I'll have the CI make some builds. stay tuned.
> > 
> 
> Builds will appear in
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.
> com-95a9f44c52fd
> 
> I just started them moments ago, so it could take anywhere from 30 minutes
> to several hours for them to appear (depending on load and platform being
> built).
> 
> Chris, Peter, Michael, Thomas, (anyone else with the problem), 
> 
> can you try a build from there and report on if it works for you?

Hi,

I try

seems to be working now :-)

here is my proxy log :
1357890740.485      0 10.10.10.101 TCP_DENIED/407 1743 CONNECT mail.google.com:443 - NONE/- text/html
1357890740.515      0 10.10.10.101 TCP_DENIED/407 1865 CONNECT mail.google.com:443 - NONE/- text/html
1357890740.638      0 10.10.10.101 TCP_DENIED/407 1755 POST http://ocsp.thawte.com/ - NONE/- text/html
1357890740.678      0 10.10.10.101 TCP_DENIED/407 1877 POST http://ocsp.thawte.com/ - NONE/- text/html
1357890740.808     91 10.10.10.101 TCP_MISS/200 1593 POST http://ocsp.thawte.com/ MY_DOMAIN\my_username DIRECT/199.7.55.72 application/ocsp-response
1357890740.902      0 10.10.10.101 TCP_DENIED/407 1755 CONNECT accounts.google.com:443 - NONE/- text/html
1357890740.933      0 10.10.10.101 TCP_DENIED/407 1877 CONNECT accounts.google.com:443 - NONE/- text/html
1357890741.156    113 10.10.10.101 TCP_MISS/200 1593 POST http://ocsp.thawte.com/ MY_DOMAIN\my_username DIRECT/199.7.55.72 application/ocsp-response


there is "TCP_DENIED/407" twice, then firefox send authentication "MY_DOMAIN\my_username" .

You will add this fix on the standard firefox setup ? (I am newbie here)

Kind regards

Comment 27

6 years ago
I have tried the nightly build and it works!

Thanks
(Reporter)

Comment 28

6 years ago
I can confirm that this has been resolved in the nightly build and no longer get this issue.
Flags: needinfo?(howardm)
(Reporter)

Comment 29

6 years ago
One user has reported it promptly crashed their graphics driver.

No other issues have been found and I personally have had no issues and everything seems to be working.
(Assignee)

Comment 30

6 years ago
(In reply to Michael Howard from comment #29)
> One user has reported it promptly crashed their graphics driver.

definitely unrelated to the change in here.

> 
> No other issues have been found and I personally have had no issues and
> everything seems to be working.

thanks!
(Assignee)

Comment 32

6 years ago
Comment on attachment 700613 [details] [diff] [review]
patch 0

[Approval Request Comment]
Regression caused by (bug #): 570283
User impact if declined: Users behind some proxies that are configured to do windows integrated authentication (NTLM) won't be able to use the proxy; the proxy is often a requirement for network access.
Testing completed (on m-c, etc.): on m-i now. Passes CI tests of try. Independent reporter confirmations of a test build based on FF18 in Comments 24, 25, 26, 27, and 28.
Risk to taking this patch (and alternatives if risky): very low - it reverts a portion of 570283 and simply adds 2 headers to a CONNECT tunnel transaction (one of which we've had in <= FF17 and the other which is the open standards equivalent)

String or UUID changes made by this patch: none
Attachment #700613 - Flags: approval-mozilla-release?
Attachment #700613 - Flags: approval-mozilla-beta?
Attachment #700613 - Flags: approval-mozilla-aurora?

Updated

6 years ago
Attachment #700613 - Flags: approval-mozilla-beta?
Attachment #700613 - Flags: approval-mozilla-beta+
Attachment #700613 - Flags: approval-mozilla-aurora?
Attachment #700613 - Flags: approval-mozilla-aurora+
https://hg.mozilla.org/mozilla-central/rev/fbc20626801f
Assignee: nobody → mcmanus
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla21
status-firefox19: affected → fixed
status-firefox20: affected → fixed
status-firefox21: affected → fixed
Duplicate of this bug: 830493

Updated

6 years ago
Attachment #700613 - Flags: approval-mozilla-release? → approval-mozilla-release+

Updated

6 years ago
Keywords: verifyme
Michael, could you help us verify this on the latest beta and release candidates?

ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/19.0b2-candidates/build1/ (beta)

ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/18.0.1-candidates/build1/ (release)

We were able to verify a couple of proxy related issues, but I don't know how to go about this one. Per comment #28 I'll mark this as verified in fx21 (nightly).
status-firefox21: fixed → verified
(Reporter)

Comment 38

6 years ago
I can confirm that this issue has been resolved in the following versons.

18.0.1
19.0b2
21.0a1

this issue is still showing in 20.0a1
(Reporter)

Updated

6 years ago
status-firefox20: fixed → ---
(Assignee)

Comment 39

6 years ago
(In reply to Michael Howard from comment #38)

> this issue is still showing in 20.0a1

hmm, this doesn't make any sense to me unless maybe you tested an old build? Can you retest and supply the about:buildconfig revision?

The same code is on all those branches (I just confirmed - indeed 21 and 20 actually still have byte for byte equivalency between nsHttpConnection) and I also confirmed that it does the expected thing on aurora-20 (i..e sends the connection header when making a CONNECT).. I did not confirm that NTLM actually works because I don't have that setup in my proxy, but I did confirm that the connect header is sent which is what was diagnosed as the root of this bug.

If it persists for you can we see a wireshark packet capture (in pcap format)?

Updated

6 years ago
status-firefox18: fixed → verified
status-firefox19: fixed → verified
(Reporter)

Comment 40

6 years ago
Created attachment 703307 [details]
About:BuildConfign from Firefix 20.0a1
(Assignee)

Comment 41

6 years ago
(In reply to Michael Howard from comment #40)
> Created attachment 703307 [details]
> About:BuildConfign from Firefix 20.0a1

That's not the current firefox 20 code, which is found on the aurora channel and declares itself as 20.0a2

I believe you have code from when 20 was on nightly because your about:buildconfig shows a mozilla-central branch with the last changeset dated 1/7.. the fix for this bug was not made until 1/9 so I would not expect it to be there.

Can you try out aurora?
(Reporter)

Comment 42

6 years ago
Ok i can confirm this works with 20.0a2
Status: RESOLVED → VERIFIED
(Assignee)

Comment 43

6 years ago
(In reply to Michael Howard from comment #42)
> Ok i can confirm this works with 20.0a2

thanks!
status-firefox20: --- → verified
You need to log in before you can comment on or make changes to this bug.