1293765 - POST request sends no parameters if NTLM is involved

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[bugzilla]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 /
Build ID: 20160809004002

Steps to reproduce:

I developed PHP pages that use NTLM to get the user name of the windows client. The web interface is making use of AJAX with jQuery.
In version 50.0a2 of the developer edition (maybe earlier) XHR POST requests don't work anymore. In the common release v48 it is still working.
My NTLM routine in PHP is quite old so I have to set network.auth.force-generic-ntlm-v1 to true.


Actual results:

In PHP I get a POST request with no data at all. It also looks different in the Network module of the developer tools: in version 50.0a2 I only see two requests: the first one gets the 401 response of the server and still has the parameters set by the JavaScript. The second request has the 200 response but no parameters and accordingly the response of a "disappointed" PHP script. :-)


Expected results:

Like in version 48 of the "normal" version the POST request should keep its parameters throughout the NTLM ping-pong. In the network module I see three requests: two with 401 response and the last one with 200 response. All three show the parameters set by the JavaScript.

If it helps I can post the source code of the PHP implementation of the NTLM handshake but I don't think that the problem is there because it worked fine so far and still does in v48 and also in a recent Chrome.
I tried it in v48 and v50.0a2 on Linux (by manually entering user name and password) with newly created profiles, so no extensions or Windows madness involved. I just turned network.auth.force-generic-ntlm-v1 to true to work with my server side scripts.

Comment 1

[bugzilla]

Sorry for not daring to think that this would also affect a POST request without XHR. You can test it on https://latzinator.com/post_test.php. You can enter whatever you like when prompted for credentials.

Comment 2

[:Gijs (he/him)]

Dragana, can you look into this?

Comment 3

[u408661]

Honza, IIRC you know a bit about NTLM. Can you take a look at this and re-assign or re-triage as appropriate?

Comment 4

[Honza Bambas (:mayhemer)]

(In reply to bugzilla from comment #1)
> Sorry for not daring to think that this would also affect a POST request
> without XHR. You can test it on https://latzinator.com/post_test.php. You
> can enter whatever you like when prompted for credentials.

I need some more info.  What is the expected script behavior?  On Windows, in chrome dev and edge the page asks for creds and then fails to load (unreachable).  In fx it returns "message 01" as a response content with force-generic-ntlm-v1 = true.

Honestly, I would more appreciate your PHP script to run it locally.

Comment 5

[bugzilla]

Attachment: simple php script that POSTs to itself and uses NTLM auth

Comment 6

[Honza Bambas (:mayhemer)]

Thanks, going to try it out!

Comment 7

[Honza Bambas (:mayhemer)]

Sorry, but I still find the test case pretty insufficient.  I absolutely don't understand how it works, what I'm suppose to do.  It behaves the same way as on your web when I host it locally.  I can't even get to the point to see the form and to try a POST.

I appreciate the effort and believe there is a bug, but to reproduce exactly what you do I simply need something more 'complete' yet still minimalistic.

Comment 8

[bugzilla]

hm, that's strange. If I load the page I get presented with an auth dialog and if I enter something I get to the form. As I mentioned you have to set network.auth.force-generic-ntlm-v1 to true in about:config to make it work. I have not yet searched for a more modern implementation because it worked quite nicely so far.
If you then enter something in the form and submit it, Firefox will reuse the credentials and do the POST. Then the script shows what it got.
This also works for me in Chrome 52 on Linux. I guess on a Windows client the browser would automatically forward the user information with a hashed password instead. At least if the script runs on a trusted domain. Otherwise there should also be a dialog for username and password.
I don't know how to reduce this even further because I just "reused" the routine for NTLM. :-)

Comment 9

[Honza Bambas (:mayhemer)]

(In reply to bugzilla from comment #8)
> hm, that's strange. If I load the page I get presented with an auth dialog
> and if I enter something I get to the form. As I mentioned you have to set
> network.auth.force-generic-ntlm-v1 to true in about:config to make it work.
> I have not yet searched for a more modern implementation because it worked
> quite nicely so far.

I did that.

> If you then enter something in the form and submit it, Firefox will reuse
> the credentials and do the POST. Then the script shows what it got.
> This also works for me in Chrome 52 on Linux. I guess on a Windows client
> the browser would automatically forward the user information with a hashed
> password instead. At least if the script runs on a trusted domain. Otherwise
> there should also be a dialog for username and password.
> I don't know how to reduce this even further because I just "reused" the
> routine for NTLM. :-)

OK, i do some more experiments of my own, will report later.

Comment 10

[Honza Bambas (:mayhemer)]

Confirmed on 50.

Definitely something fishy going on here.  It's intermittent, first idea is 
something with connection persistence and timeout, but need to look into logs to 
find for sure.

Comment 11

[Honza Bambas (:mayhemer)]

Regression from bug 1264566.  Apparently when we are doing authentication loops in the http channel, we can't just drop mUploadStream (carrying the form data) in OnStopRequest that easily.

Comment 12

[Honza Bambas (:mayhemer)]

Attachment: v1 (don't release upload stream when auth is pending)

selfexplanatory.  no need for a try push, we (apparently) don't have tests for this at all...

manually tested with own test case, ran net xpc tests.

Comment 14

[Valentin Gosu [:valentin] (he/him)]

Comment on attachment 8781996 [details] [diff] [review]
v1 (don't release upload stream when auth is pending)

Review of attachment 8781996 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8cc4907776c8
Don't release upload stream in HTTP channel until authentication loop is done. r=valentin

Comment 16

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/8cc4907776c8

Comment 17

[Honza Bambas (:mayhemer)]

Comment on attachment 8781996 [details] [diff] [review]
v1 (don't release upload stream when auth is pending)

Approval Request Comment
[Feature/regressing bug #]: 1264566 (on 50, m-b)
[User impact if declined]: broken POST when Negotiate/NTLM authentication is involved
[Describe test coverage new/current, TreeHerder]: no automated, not NTLM tests.. but it's for a long time on m-c (just merged to m-a)
[Risks and why]: very low, if none
[String/UUID change made/needed]: none

Sorry for requesting approval that late, but I was confused what all versions were affected with this bug.

Comment 20

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hello there, could you please verify this issue is fixed as expected on a latest Nightly build? Thanks!

Comment 21

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8781996 [details] [diff] [review]
v1 (don't release upload stream when auth is pending)

Recent regression in Fx50, has been in Nightly for a month, Beta50+

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/88fa08847a2f

Comment 23

[bugzilla]

(In reply to Ritu Kothari (:ritu) from comment #20)
> Hello there, could you please verify this issue is fixed as expected on a
> latest Nightly build? Thanks!

Sorry, totally overlooked that one. Yes, I can confirm that it worked as soon as it got released on Aurora channel.

Comment 24

[Abhijit Kumar]

I also try in Firefox Nightly 53.0a1 (2017-01-08) (64-bit)