Closed Bug 602814 Opened 14 years ago Closed 11 years ago

Sites requiring an NTLM authentication don't work through a proxy

Categories

(Core :: Networking: HTTP, defect)

Product:
Core
Component:
Networking: HTTP
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla23
Tracking Flags:
Tracking Status
blocking2.0 --- -
blocking1.9.2 --- -

People

(Reporter: granaivo, Assigned: s-mozilla)

References

Details

(Keywords: regression)

Attachments

(1 file, 1 obsolete file)

patch v2
3.57 KB, patch
mcmanus
: review+
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10

The bug occurs with Firefox 3.6.10 an Firefox 4.0b6 on both Windows XP and Linux, it works with Firefox 3.5 though.

When browsing through a proxy and attempting to access a site requiring an NTLMv2 authentication, the authentication fail (note that the authentication is on the site, NOT on the proxy). It seems to be a keepalive problem:

GET http://localhost:8080/test_page HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr-fr,en;q=0.8,en-us;q=0.5,zu;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 115
Proxy-Connection: keep-alive
Referer: http://localhost:8080/
Cookie: session_id=069478d56a2282ecc4a005344fd83ce65b9e8c23
Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=

HTTP/1.0 401 Unauthorized
Date: Fri, 08 Oct 2010 08:18:29 GMT
Content-Length: 1256
Content-Type: text/html;charset=utf-8
WWW-Authenticate: NTLM TlRMTVNTUAACAAAADAAMAHAAAAAFgooAebV1UHkjepEAAAAAAAAAAEAAQAAwAAAAAQAMAG0AaQBrAGEAZABvAAIAKABtAGkAawBhAGQAbwAuAG8AbABmAGUAbwAtAGwAYQBiAC4AbgBlAHQAAAAAAG0AaQBrAGEAZABvAA==
Server: CherryPy/3.1.2
Set-Cookie: session_id=069478d56a2282ecc4a005344fd83ce65b9e8c23; expires=Fri, 08 Oct 2010 10:18:29 GMT; Path=/
Proxy-support: Session-Based-Authentication
Connection: Proxy-support
X-Cache: MISS from localhost
X-Cache-Lookup: MISS from localhost:3128
Via: 1.0 localhost (squid/3.1.6)
Proxy-Connection: keep-alive
<body>

Firefox cuts the connection here, and then reconnects to do the second part of the authentication. From my understanding, the NTLM protocol requires the HTTP connection to be kept alive.

The authentication works without a proxy.

Tested with Squid 3.1 and 2.6, our own web server and this one: http://code.google.com/p/python-ntlm/source/browse/#svn/branches/clientserver

Reproducible: Always

Steps to Reproduce:
1. Surf through a proxy
2. Go to a site requiring NTLM authentication

Actual Results:  
Authentication fail

Expected Results:  
Authentication success
OS: Linux → All
Version: unspecified → 3.6 Branch
maybe bug 592197 is related ?
Component: General → Networking: HTTP
Product: Firefox → Core
QA Contact: general → networking.http
Version: 3.6 Branch → unspecified
Doesn't seem likely, no; that bug is SSL-specific.

Reporter, is this a regression from an earlier build?
The bug is not present in FF 3.5.13, that's all I know.
OK, thanks.  Would you be able to check 3.6.0 as well?
Thseems is present in this version:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)
*The bug
Thanks!
Status: UNCONFIRMED → NEW
blocking1.9.2: --- → ?
blocking2.0: --- → ?
Ever confirmed: true
Keywords: regression
But bug 592197 says NTLM auth through a proxy works. It was broken for SSL temporarily, but that works too.

So what's different between this reporter's config/network and the people who reported, worked on, and tested bug 592197?

Not "blocking" 1.9.2 security updates if it's been a problem since the initial 3.6 release, obviously the scenario in which this bug shows up is somewhat rare. Needs investigation, and if there's a patch we will of course consider it for the branch releases.

Can the reporter please try a Firefox 4 beta? it would be good to know if this problem is specific to the 1.9.2 branch (Firefox 3.6.x) that got fixed, or if it's still ongoing.
blocking1.9.2: ? → -
(In reply to comment #8)
Daniel, as I said the problem happens when it is the remote site that requires an NTLM authentication, not the proxy, unlike bug 592197 (unless I misread something?)

FF 4.0b6 and 3.6.x don't work, FF 3.5.x does.
Would you be willing to try narrowing down when the problem appeared using nigtly builds from between 3.5 and 3.6?
After some additional tests, nightly build firefox-2009-07-20-03-mozilla-central works, while firefox-2009-07-21-03-mozilla-central doesn't.

Looking at the changelog from that day, I suspect this changeset may be the problem:
http://hg.mozilla.org/mozilla-central/rev/0a13ddc99c95
I'm really not sure though, since I can't build this revision myself due to (unrelated) compiling errors.
Hmm.  That's very possible, yes.  Honza, can you take a look?
Blocks: 475053
-> me

Gaël, do you have any testing site that requires NTLMv2 auth I could test with?
Assignee: nobody → honzab.moz
Status: NEW → ASSIGNED
blocking2.0: ? → -
(In reply to comment #13)
> -> me
> 
> Gaël, do you have any testing site that requires NTLMv2 auth I could test with?

Sorry, I don't know of any site that is accessible on the Internet.

I tested the bug at work on our internal NTLM server. The bug is also reproducible using this simple python NTLM server:
http://code.google.com/p/python-ntlm/source/browse/#svn/branches/clientserver
Bug 542951 is a duplicate of this bug.

I can alos confirm the Problem.
Firefox 3.6+ can not access NTLM protected webpages, when a proxy is used.

Proxy Setup:

 * squid 2.6 or 3.1 (both have "connection pinning" feature for NTLM
passthrough)
 * proxy_auth is not used

Results:
* IE works fine with both squid versions.
* Firefox 2.0, 3.0 and 3.5 work fine with both squid versions.
* Firefox 3.6.x and 4.0_beta keep prompting for valid credentials when either of
these 2 squid versions is used.
* Firefox 3.6.x and 4.0_beta work fine when no proxy is involved.
* Changing "network.automatic-ntlm-auth.trusted-uris" for automatic NTLM Logon
doesn't change the buggy behaviour.
I did some additional testing with a NetCache NetApp/6.0.6P2 Proxy instead of squid. Firefox 3.6.13 did not show the buggy behaviour with this proxy.
Here are the differet reply headers for the same page via different proxies (Note the different HTTP Versions used). The request headers were identical.

Request Header:
GET http://sharepointdevel.mydomain/Pages/default.aspx HTTP/1.1
Host: sharepointdevel.mydomain
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.21022)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive

NetApp Reply:
HTTP/1.1 401 Unauthorized
Date: Tue, 28 Dec 2010 12:10:26 GMT
Content-Length: 1656
Content-Type: text/html
Server: Microsoft-IIS/6.0
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="localdomain"
MicrosoftSharePointTeamServices: 12.0.0.6335
X-Powered-By: ASP.NET
Via: 1.1 proxy1 (NetCache NetApp/6.0.6P2)
Proxy-Support: Session-Based-Authentication

Squid Reply:
HTTP/1.0 401 Unauthorized
Content-Length: 1656
Content-Type: text/html
Server: Microsoft-IIS/6.0
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="localdomain"
MicrosoftSharePointTeamServices: 12.0.0.6335
X-Powered-By: ASP.NET
Date: Tue, 28 Dec 2010 12:25:57 GMT
Proxy-support: Session-Based-Authentication
Connection: Proxy-support
X-Cache: MISS from proxy2
X-Cache-Lookup: MISS from proxy2:8080
Via: 1.0 proxy2 (squid/3.1.8)
Connection: keep-alive

The TCP connection via squid is closed (Wireshark shows TCP-FIN initiatd by Firefox IP) after the "HTTP/1.0 401" reply from squid.
"Connection: keep-alive" doesn't seem to be honored. NTLM auth continues in a new TCP Session, wich seems to make NTLM fail.
Also Note that the reply header in Gaëls initial report contained a reply header "Proxy-Connection: keep-alive", while mine has a
"Connection: keep-alive" header.

The TCP connection via NetApp however, continues to stay open. NTLM works and the page is delivered.

I'm very interested in resolving this bug, as it gave me a lot of headaches in the last months. I'm not a programmer but a net & sysadmin and I can do some testing if you need me to.
> Squid Reply:
> Connection: Proxy-support
> X-Cache: MISS from proxy2
> X-Cache-Lookup: MISS from proxy2:8080
> Connection: keep-alive

This suggests there might be a problem with parsing Connection options other than "close" and "keep-alive".
Attached patch bare minimum fix (obsolete) — — Splinter Review 
Brian: that seems to be the case. For keep-alive over an HTTP/1.0 connection, nsHttpConnection.cpp currently requires the Connection: header to match "keep-alive" exactly, but this will never happen when a second Connection: header is present since the two headers are combined elsewhere to form "proxy-support, keep-alive".

Attached a patch that uses HasHeaderValue to find the keep-alive token instead, seems to work for me.
This bug exists in FF 4b12. There was no problem in FF 4b11.
Olibu, in that case what you're seeing is NOT this bug.  This bug is very much present in 4b11.

Please file a separate bug on your issue with more information.
(In reply to Simon J from comment #18)
> Created attachment 508151 [details] [diff] [review]
> bare minimum fix
> 
> Brian: that seems to be the case. For keep-alive over an HTTP/1.0
> connection, nsHttpConnection.cpp currently requires the Connection: header
> to match "keep-alive" exactly, but this will never happen when a second
> Connection: header is present since the two headers are combined elsewhere
> to form "proxy-support, keep-alive".
> 
> Attached a patch that uses HasHeaderValue to find the keep-alive token
> instead, seems to work for me.

So, is this patch fixing this issue?  If so, it may get reviewed and landed.  

The only problem here is how to check and verify this fix.  I might have a testing env, but not time to do any testing right now.

Releasing the bug for now, but I may return to this later.
Assignee: honzab.moz → nobody
Status: ASSIGNED → NEW
Could anyone tell me how to apply the patch or maybe provide me a patched (portable) Firefox Version?
I'd be happy to test the patch above.

I also did some more investigation and found that squid changed the reply-header from "Proxy-Connection: keep-alive" into "Connection: keep-alive" in Version 3.1.7 to be more RfC compliant.
So I downgraded squid to Version 3.1.4. to see if avoiding the connection header is a solution. Result:
The "Connection: keepalive" header is gone, but Firefox' NTLM-Problem is still there. Here is a sample reply header of squid 3.1.4 when accessing an NTLM-protected website:

HTTP/1.0 401 Unauthorized
Date: Mon, 21 May 2012 16:01:07 GMT
Server: Apache/2.2.17 (Win32) mod_auth_sspi/1.0.1 PHP/5.2.17
WWW-Authenticate: NTLM TlR.........
Content-Length: 401
Content-Type: text/html; charset=iso-8859-1
Proxy-support: Session-Based-Authentication
Connection: Proxy-support
X-Cache: MISS from PROXY1
X-Cache-Lookup: MISS from PROXY1:8080
Via: 1.0 PROXY1 (squid)
Proxy-Connection: keep-alive

I'm not sure how much of the NTLM handling comes from the underlaying OS, but 
even IE8 on Win7 does not work with squid-3.1.10 headers on NTLM web pages. 
- IE8 on Win7 with squid-3.1.4 works fine.
- IE8 on WinXP works fine with squid 3.1.10 and 3.1.4 and NTLM web pages.
i applied patch to release sources of firefox 15
it helps :)
i use squid 2.7
Just wanted to ask if there is a chance to get this bug corrected. Running currently on 17.0.5 ESR and still facing this misbehavior.
tr3027@seznam.cz: Please report a new bug, this one is closed
(In reply to Matthias Versen [:Matti] from comment #27)
> tr3027@seznam.cz: Please report a new bug, this one is closed

no, it`s status is NEW
and we stil waiting for apply this patch in release
Simon, Alexxis, tr3027 - this patch looks right to me. Ironically firefox has been bitten by servers who parse the connection header incorrectly too.

I've submitted it to the "try" builder. If it passes the automated tests and someone can verify the resulting build fixes the problem I'll merge the patch. It doesn't met ESR criteria so probably won't be backported.

The builds will take a couple hours to make. When they are ready they will be here: https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-1d604544b05a/

Can {Simon, Alexxis, tr3027} do that verification?

Thanks,
-Patrick
i`ll try it on our ntlm server
For verification you can use any Sharepoint Demo that allows you to login, like http://www.infowisesolutions.com/sandbox.aspx 

Sharepoint uses NTLM, when your client is not in the same Windows-Domain as the webserver. That makes it a very safe bet.
You can verify that you are using NTLM with FF-Plugin "HTTP-Live-Headers". The requests should contain a line "Auhtorization: NTLM..."

I'd vote for a backport to ESR, since NTLM is mostly used in corporate Active-Directory environments. Though I'm not sure if this hits any of the criteria for ESR.
I'll test on our intern servers if you build Win version(s) as well. It'd be great if this patch could hit ESR as well as sela is right with his last paragraph ...
(In reply to tr3027 from comment #32)
> I'll test on our intern servers if you build Win version(s) as well.

Great. It is already building a windows version - that just takes longer. It will show up when its ready.

> It'd be
> great if this patch could hit ESR as well as sela is right with his last
> paragraph ...

You're welcome to do the nomination yourself if you like (release management makes those decisions).. but these are the criteria (below) and this doesn't meet the bar imo:

"Maintenance of each ESR, through point releases, is limited to high-risk/high-impact security vulnerabilities and in rare cases may also include off-schedule releases that address live security vulnerabilities. Backports of any functional enhancements and/or stability fixes are not in scope."
Unfortunately in my case the problem is NOT fixed with Patricks Win32 Build.
I tested on WinXP and Windows Server2008 using squid as a proxy.

Internet Explorer has no NTLM Problem in the same setup.
it works for my server
win version + squid2.7
I experience the same as sela - Win32 build does still not work for me and our IIS 7.5 WinBasedAuth site via squid 2.6. In Wireshark I can see the following:

* TCP Handshake to Squid
* HTTP request
  GET http://server.domain.int:803/ HTTP/1.1
  Host: server.domain.int:803
  User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20130416 Firefox/23.0
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  Accept-Language: en-US,en;q=0.5
  Accept-Encoding: gzip, deflate
  Connection: keep-alive
* HTTP response
  HTTP/1.0 401 Unauthorized
  Content-Type: text/html
  Server: Microsoft-IIS/7.5
  WWW-Authenticate: Negotiate
  WWW-Authenticate: NTLM
  X-Powered-By: ASP.NET
  Date: Wed, 17 Apr 2013 05:05:46 GMT
  Content-Length: 1293
  Proxy-support: Session-Based-Authentication
  Connection: Proxy-support
  X-Cache: MISS from proxy.domain.int
  Via: 1.0 proxy.domain.int:8080 (squid/2.6.STABLE18)
  Proxy-Connection: keep-alive
* TCP FIN initiated from the client machine
* another TCP Handshake to proxy
* HTTP request with the same headers as previously plus
  Authorization: Negotiate YIIE1wYGKwYBBQUC.......
* HTTP response
  HTTP/1.0 401 Unauthorized
  Content-Type: text/html
  Server: Microsoft-IIS/7.5
  WWW-Authenticate: Negotiate oRUwE6ADCgEDoQwGCisGAQQBgjcCAgo=
  WWW-Authenticate: NTLM
  X-Powered-By: ASP.NET
  Date: Wed, 17 Apr 2013 05:05:46 GMT
  Content-Length: 1293
  Proxy-support: Session-Based-Authentication
  Connection: Proxy-support
  X-Cache: MISS from proxy.domain.int
  Via: 1.0 proxy.domain.int:8080 (squid/2.6.STABLE18)
  Proxy-Connection: keep-alive
* TCP FIN initiated by Firefox

Right after that I get a login popup from FF. Entering a username/PW goes through another TCP Handshake/HTTP request and response and TCP FIN from FF and a new login popup ...
I think I found the reason why it doesn't work for everyone. It depends on the proxy you're using. 
If the proxy uses "connection: keepalive" it works with the nightly. 
If the proxy uses "proxy-connection: keepalive" it doesn't work.
As mentioned before, squid uses the "connection" Header from 3.1.7 onwards. I think this change is backported to recent 2.7 versions of squid.

I use RedHat's Squid, because they introduced a switch to change this behaviour.
See the option "http10" on http://rhn.redhat.com/errata/RHSA-2013-0505.html
They introduced it because IE had the same problems - just the other way around.

squid with "connection" header:  FF-nightly works - IE fails
squid with "proxy-connection" header: FF-nightly fails - IE works
It's kind of funny, hm? but not really suitable for networks where IE and FF coexist.

So the patch does help, since Firefox used to Fail on both Headers. Now at least one works. And it works equally on WinXP and Win7. I can't say that about IE.
In fact I think the reason why it is not working (at least in my case) is that my squid sends the following response headers:
  Connection: Proxy-support
  Proxy-Connection: keep-alive

The above patch added the following lines:
+    const nsHttpAtom connectionHeader
+        = responseHead->PeekHeader(nsHttp::Connection)
+          ? nsHttp::Connection : nsHttp::Proxy_Connection;

In my case there are both headers - Connection AND Proxy-Connection so it picks the first one. And as its value is "Proxy-Support" instead of "keep-alive" it closes the TCP session right after getting the HTTP response.

It looks like it'd be better to turn this code to:
+    const nsHttpAtom connectionHeader
+        = responseHead->PeekHeader(nsHttp::Proxy_Connection)
+          ? nsHttp::Proxy_Connection : nsHttp::Connection;
... at least in my case it maight work. But of course I have no idea if that would break something else :-(
(In reply to tr3027 from comment #38)
> In fact I think the reason why it is not working (at least in my case) is
> that my squid sends the following response headers:
>   Connection: Proxy-support
>   Proxy-Connection: keep-alive
> 
> The above patch added the following lines:
> +    const nsHttpAtom connectionHeader
> +        = responseHead->PeekHeader(nsHttp::Connection)
> +          ? nsHttp::Connection : nsHttp::Proxy_Connection;
> 
> In my case there are both headers - Connection AND Proxy-Connection so it
> picks the first one. And as its value is "Proxy-Support" instead of
> "keep-alive" it closes the TCP session right after getting the HTTP response.

I think you have diagnosed it correctly.

I've tweaked the patch to determine if you have an explicit close (and if that fails, an explicit keep-alive) and then to do the normal logic based on that.

The definition of explicit-value is "is value present in connection header, or is it present in the proxy-connection header". This will ignore other, legitimate but irrelevant, values in those headers.

This is correct behavior. (the last version was in turn also more correct than what is checked into nightly as it ignored extraneous values if they were in the right header, but could ignore the right header completely).
Attached patch patch v2 — — Splinter Review 

        Attachment #508151 -
        Attachment is obsolete: true

        Attachment #738467 -
        Flags: review+

    
    

    
  
As before, I have sent this to the try builder and need verification of it.

tr3027, sela, alexxis - thanks in advance!

give the builds a couple hours to complete, and they should be here:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-a4fc681cb777

    
    

    
  
Great Patrick - Win32 works for me with and without a proxy now - thank you! Looking forward to see that patch in the ESR :-)

    
    

    
  
mine is working too
waiting for release :-)

    
    

    
  
Same here. It works like a charm. Thanks a lot guys.

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/4078a8fd22ce
Assignee: nobody → simon-mozilla
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla23

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: