331210 - XMLHttpRequest loses NTLM authentication if it uses Connection: Close

Status: RESOLVED INVALID

(Core :: Networking: HTTP, defect)

Description

[Charles Leeds]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1

I am developing a Rails app on Apache that uses mod_auth_sspi for user authentication. I have configured Firefox using the network.automatic-ntlm-auth.trusted-uris setting to automatically authenticate me, which works fine when browsing pages normally. However, any fancy AJAX stuff using XMLHttpRequest does not get NTLM authenticated - it pops up a Basic Auth dialog. 

Reproducible: Always

Steps to Reproduce:
1. Create NTLM authenticated site that uses XMLHttpRequest (I use Apache with mod_sspi module)
2. Add site to network.automatic-ntlm-auth.trusted-uris 
3. Load a regular page protected by NTLM authentication
4. Now use a part of the site that generates an XMLHttpRequest call

Actual Results:  
You will get Basic Auth dialog boxes.

Expected Results:  
It would pass through your NTLM credentials and not prompt you.

This is important to make Firefox a viable browser in Windows centric corporations that require NTLM be used for intranet apps.  These intranet web apps increasingly use XMLHttpRequest.

Comment 1

[Christian :Biesinger (don't email me, ping me on IRC)]

fixing summary (the dialog is used for other auth methods too, not just Basic)

Comment 2

[Will Rogers]

It looks like this is my report from the MozillaZine forum.  Thanks, Charles, for filing a bug report, since I only got around to doing it today.  I can confirm that I am still experiencing this with Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20060405 Firefox/2.0a1 ID:2006040504.

A fix would be most appreciated.

Comment 3

[Matt M]

Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3

I'm also experiencing this issue as I'm developing a PHP/Ajax application with   Firefox/1.5.0.3 on IIS 6.0 on a Windows 2003 Server box.  What's odd is that the NTLM will work for a while with the XMLHTTPRequests, but then it appears as if Firefox "forgets" to send the authentication info on certain requests, as seen by the missing username and 401 response from the IIS logs:

2006-05-09 00:06:21 W3SVC1114527625 BODHI GET /images/indicator.gif - - 127.0.0.1 localhost 401 2 1849 520
2006-05-09 00:06:21 W3SVC1114527625 BODHI GET /images/indicator.gif - - 127.0.0.1 localhost 401 1 1844 586
2006-05-09 00:06:21 W3SVC1114527625 BODHI GET /images/indicator.gif - BODHI\Matt 127.0.0.1 localhost 200 0 1781 718
2006-05-09 00:06:21 W3SVC1114527625 BODHI POST /dhl.php/rsb/validate - BODHI\Matt 127.0.0.1 localhost 200 0 1954 1129
2006-05-09 00:06:21 W3SVC1114527625 BODHI GET /sf/images/sf_admin/cancel.png - - 127.0.0.1 localhost 200 0 1080 529
2006-05-09 00:06:21 W3SVC1114527625 BODHI GET /sf/images/sf_admin/error.png - - 127.0.0.1 localhost 200 0 964 540

2006-05-09 00:06:49 W3SVC1114527625 BODHI POST /dhl.php/rsb/validate - BODHI\Matt 127.0.0.1 localhost 200 0 1954 1133
2006-05-09 00:06:51 W3SVC1114527625 BODHI POST /dhl.php/rsb/validate - - 127.0.0.1 localhost 401 2 1868 1126
2006-05-09 00:06:51 W3SVC1114527625 BODHI POST /dhl.php/rsb/validate - - 127.0.0.1 localhost 401 1 1863 1212
2006-05-09 00:06:56 W3SVC1114527625 BODHI POST /dhl.php/rsb/validate - - 127.0.0.1 localhost 401 1 1751 1364

Some server vars:
    [AUTH_TYPE] => NTLM
    [AUTH_USER] => BODHI\Matt
    [CONTENT_LENGTH] => 528
    [CONTENT_TYPE] => application/x-www-form-urlencoded
    [PATH_TRANSLATED] => C:\codebase\branches\current\framework\www\rsb\validate
    [QUERY_STRING] =>
    [REMOTE_ADDR] => 127.0.0.1
    [REMOTE_HOST] => 127.0.0.1
    [REMOTE_USER] => BODHI\Matt
    [REQUEST_METHOD] => POST
    [SERVER_NAME] => localhost
    [SERVER_PORT] => 80
    [SERVER_PROTOCOL] => HTTP/1.1
    [SERVER_SOFTWARE] => Microsoft-IIS/6.0

Comment 4

[Mike Schroepfer]

Darin do you have any cycles or thoughts on this one (or recommendations on who could help)?

Comment 5

[Will Rogers]

Are there any new thoughts on this bug?  Is it possible that it might get fixed for Firefox 2.0?  I noticed that bug 170477 was just fixed, perhaps whoever worked on that would be a good resource?

If I can be of any more help with additional details about my environment or logs or anything, just let me know.

Comment 6

[Darin Fisher]

Will, can you please generate a HTTP log for us and attach it to this bug?  Instructions here:
http://www.mozilla.org/projects/netlib/http/http-debugging.html

Thanks!

Comment 7

[Will Rogers]

Attachment: firefox HTTP log demonstrating failed auth

Here's the HTTP log.  I apologize for all the noise with fetching images, css, js, etc (makes me realize I should turn off authentication for images), but I was afraid of accidentally snipping something important if I tried to prune it.  The first successful NTLM authenticated request starts around line 723, and the failing XMLHttpRequest begins circa line 13169.

The log reveals that Firefox does actually perform the NTLM handshake, but it doesn't work for some reason.  Apache log contains this error:

[Fri Jul 14 11:34:23 2006] [error] [client 138.39.10.124] (OS 87)The parameter is incorrect.  : authentication failure for "/comment/save": user unknown, reason: cannot generate context, referer: http://madbugs.pepcoholdings.biz/bug/show/457

Comment 8

[Darin Fisher]

Hmm... I wonder if this is related to bug 318253.

Comment 9

[Will Rogers]

It's hard to say.  As an informal, unscientific test, I just browsed around my application in Firefox for a few minutes, clicking on various links, and doing POSTs.  I could not reproduce the eventual failure described in bug 318253; it continued to work correctly over ~10 minutes and >100 page loads.  XMLHttpRequest always fails authentication, every time.

That certainly doesn't prove they're *not* related, but the symptoms do seem to be different.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1b1) Gecko/20060714 BonEcho/2.0b1 ID:2006071403

Comment 10

[Darin Fisher]

OK, thanks for taking the time to evaluate that.

Comment 11

[Oliver Paulus]

I have the same problem. I use the prototype library to create a new AJAX request. On Windows with FF I get authentication popups if I start the AJAX request. I send the AJAX request with POST method - not GET. I use NTLM authentication on server side. FF version is 1.5.0.4.

Comment 12

[Oliver Paulus]

(In reply to comment #11)
> I have the same problem. I use the prototype library to create a new AJAX
> request. On Windows with FF I get authentication popups if I start the AJAX
> request. I send the AJAX request with POST method - not GET. I use NTLM
> authentication on server side. FF version is 1.5.0.4.
> 

I have tested this issue with FF 2.0 beta 1 too. FF beta 1 does have the same problem with NTLM authentication and AJAX requests with POST action.

Comment 13

[Oliver Paulus]

I have tested the configuration with "network.ntlm.send-lm-response" and "network.automatic-ntlm-auth.trusted-uris" - but nothing helps.

Comment 14

[Oliver Paulus]

Attachment: test script - shows AJAX POST NTLM request problem

This test pages/script shows the problem with AJAX POST requests in FF with NTLM authentication.

Comment 15

[Darin Fisher]

This bug is caused by the addition of 'Connection: close' in the request headers for the XMLHttpRequest POST.  NTLM authentication only works if you use persistent 'keep-alive' connections since the objective of NTLM is to authenticate the connection and not the request sent over the connection.

The prototype.js file in the testcase appears to be responsible for setting the 'Connection: close' request header.

I'm not sure how to best deal with this in Necko.  Should I ignore the 'Connection: close' header since it is impossible to do NTLM authentication when that is specified, or should I continue to fail, but report some information to the Error Console?

Comment 16

[Darin Fisher]

prototype.js seems to think that it needs to set 'Connection: close' to workaround Mozilla bug 246651, but that bug was fixed in Firefox 1.5.

I believe this bug is somewhat invalid, but perhaps we should keep it open to do something better in Necko when this situation occurs.

Comment 17

[Will Rogers]

I just removed the relevant lines from prototype.js in my application, and it worked.  It is indeed a prototype problem, not a Mozilla one.  I recommend that you at least make it log something to the error console so other poor lusers can discover what's happening.

Thanks so much for your work on this, Darin.  Did you already report this to the prototype maintainer?  If not, I will.

Comment 18

[Darin Fisher]

Nope, I haven't reported the problem to the prototype maintainer.  I definitely would appreciate you doing so.  Thanks!  I'll look into finding a way to report this problem in the Firefox error console.

Comment 19

[Oliver Paulus]

Thank you for your support. I hope the protoype maintainer will find a way to support FF < 1.5 (because of bug 246651) and >= 1.5.

Comment 20

[Will Rogers]

Just FYI, I am still trying to report the Prototype bug, but dev.rubyonrails.org has been unresponsive and/or broken since this was figured out.  I'll post here with the ticket number once it actually does work.

Comment 21

[Will Rogers]

I filed http://dev.rubyonrails.org/ticket/5809

Comment 22

[Bob]

I'm developing on Firefox v2.0.0.7 using my own javascript to generate XmlHttpRequests and am running into this problem.  The same code works with GETs, but not with POSTs.  The server is Windows 2003 R2/IIS v6 with Kerberos only authentication allowed.  When the authentication dialog appears, even typing in proper credentials will cause the dialog to reappear endlessly.

Comment 23

[Bob]

My mistake.  It works fine without the connection close header.  Thanks Charles.

Comment 24

[timeless]

I think sending a warning to the error console would be a good thing to do.

re comment 21, http://dev.rubyonrails.org/ticket/5606 is fixed

http://www.mail-archive.com/[e-mail]/msg06168.html

the ie sniff doesn't seem to include Connection: Close, in fact, it includes Connection: Keep-Alive

http://www.innovation.ch/personal/ronald/ntlm.html seems to argue that you should indeed ignore Connection: Close.

fwiw, I think Gmail uses connection: close, because I was chasing an xmlhttprequest thing Wednesday and kept seeing it. You might want to find out if they have good reasons for doing so, or if they're doing it for the same reason prototype was.

Build identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3pre) Gecko/2008010205 Minefield/3.0b3pre

Comment 25

[timeless]

I think sending a warning to the error console would be a good thing to do.

re comment 21, http://dev.rubyonrails.org/ticket/5606 is fixed

http://www.mail-archive.com/[e-mail]/msg06168.html

the ie sniff doesn't seem to include Connection: Close, in fact, it includes Connection: Keep-Alive

http://www.innovation.ch/personal/ronald/ntlm.html seems to argue that you should indeed ignore Connection: Close.

fwiw, I think Gmail uses connection: close, because I was chasing an xmlhttprequest thing Wednesday and kept seeing it. You might want to find out if they have good reasons for doing so, or if they're doing it for the same reason prototype was.

Build identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3pre) Gecko/2008010205 Minefield/3.0b3pre

Comment 26

[Patrick McManus [:mcmanus]]

comment 17