The default bug view has changed. See this FAQ.

ISA proxy negotiate auth broken (regression)

RESOLVED FIXED in Firefox 19

Status

()

Core
Networking
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: will69, Assigned: mcmanus)

Tracking

(Blocks: 1 bug, {regression})

Trunk
mozilla19
x86
Windows XP
regression
Points:
---
Bug Flags:
in-testsuite ?

Firefox Tracking Flags

(firefox16 unaffected, firefox17+ unaffected, firefox18+ unaffected, firefox19 fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
Requesting any web page yields HTTP error 407 (proxy authentication required) and status 12209 (The ISA Server requires authorization to fulfill the request. Access to the Web Proxy service is denied).

STR: Use a MS ISA server as your proxy with a default configuration of Firefox 17.

Workaround: Set "network.negotiate-auth.allow-proxies" from "true" to "false" in "about:config" (just a double click)

Regression range:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=20db7c6d82cc&tochange=8b96a33ecbd2

Download:

GOOD: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/07/2012-07-26-05-28-03-mozilla-central/firefox-17.0a1.en-US.win32.installer.exe

FAIL: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/07/2012-07-27-03-05-08-mozilla-central/firefox-17.0a1.en-US.win32.installer.exe
Thank you for the excellent bug report with a regression range !
I suspect bug 767158
Blocks: 767158
Component: General → Networking
Keywords: regression
Product: Firefox → Core
(Assignee)

Updated

5 years ago
Assignee: nobody → mcmanus
(Assignee)

Comment 2

5 years ago
will69 - do you know what kind of authorization your proxy is doing? Specifically

1] do you know if it is NTLM or Negotiate?

2] is it full windows integrated auth? Do you normally have to type in your credentials to authenticate to the proxy or is it (before this bug) transparent to you if you are logged in?

thanks
(Assignee)

Comment 3

5 years ago
3] (probably redundant, but what the heck) is your windows computer attached to a domain controller that is being used to determine the auth for the ISA proxy?
(Reporter)

Comment 4

5 years ago
1] I’m afraid I need some help to find out. Is there a plugin I can use?
2] It is fully integrated. Before this bug, I did not have to type in my credentials.
3] The computer is a member of one domain. The user is a member of another domain. Both are in the same forest. Access to the proxy is granted to the user via a security group.
(Assignee)

Comment 5

5 years ago
will I'm afraid this is works for me. I've setup a windows ADC and I've setup a copy of wingate that uses windows integrated autthentication (negotiate::kerberos) from the ADC, and I've added my laptop to that domain..

and I can confirm that the auth is enforced and works in nightly, ff15, and msie all the same.

not sure where to go here.. can you do a packet trace (see wireshark) with both nightly and ff15 or ff16 so we can see the differences?

Comment 6

5 years ago
I think I've run into this on ff17 beta, and having just downgraded to 16.0.1, I'm sure of it.

Let me know if you need anything else for debugging purposes.  network.*auth* is set to default, except for network.negotiate-auth.trusted-uris, which is set to a domain specific value I can't really disclose.  However, unsetting network.negotiate-auth.trusted-uris doesn't do anything.

I do have FoxyProxy installed, but disabling it doesn't improve matters. 

From wireshark, my initial (working, ff16.0.1) trace says:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 XXXXXXXXXXX
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 737

Which is followed by:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org
Proxy-Authorization: Negotiate TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw==

HTTP/1.1 407 Proxy Authentication Required ( Access is denied.  )
Via: 1.1 XXXXXXXXXXXXX
Proxy-Authenticate: Negotiate TlRMTVNTUAACAAAADgAOADgAAAAFgomib3t0+h9SBh0AAAAAAAAAAMQAxABGAAAABQLODgAAAA9PAEMARQBBAE4ASQBBAAIADgBPAEMARQBBAE4ASQBBAAEAGgBQAFgAWQBOAFoAMAA2ADEATABEAEEAMAAwAAQAKABvAGMAZQBhAG4AaQBhAC4AYwBvAHIAcAAuAGEAbgB6AC4AYwBvAG0AAwBEAFAAWABZAE4AWgAwADYAMQBMAEQAQQAwADAALgBvAGMAZQBhAG4AaQBhAC4AYwBvAHIAcAAuAGEAbgB6AC4AYwBvAG0ABQAYAGMAbwByAHAALgBhAG4AegAuAGMAbwBtAAAAAAA=
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 0     

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org
Proxy-Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAIAAAAAYABgAmAAAAA4ADgBIAAAAEAAQAFYAAAAaABoAZgAAAAAAAACwAAAABYKIogUBKAoAAAAPTwBDAEUAQQBOAEkAQQByAG8AYgBpAG4AcwBtADUAVwA1AEMAMgA2ADAAQQA0ADQAMQAyADkARQBdpiGV1aa44gAAAAAAAAAAAAAAAAAAAABcYnHGzWxuf2pjeRA5bWVzd/pbSh8Lkbw=

HTTP/1.1 200 Connection established
Via: 1.1 XXXXXXXXXXXXX, 1.1 XXXXXXXXXXXXX
Connection: Keep-Alive
Proxy-Connection: Keep-Alive

And then a non-working version (17.0b1), using the same profile:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/17.0 Firefox/17.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 XXXXXXXXXXXXX
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 737   

Followed by, well, nothing.
(Assignee)

Comment 7

5 years ago
this is still wfm, this time using TMG 2010 which is the successor to ISA.

Michael-Mozilla, it looks like you've got a double depth proxy of some sort based on 

HTTP/1.1 200 Connection established
Via: 1.1 XXXXXXXXXXXXX, 1.1 XXXXXXXXXXXXX

any chance you can test with just 1 proxy (I know maybe you can't).

will69, any chance you've got 2 layers of proxies too?
(Assignee)

Comment 8

5 years ago
Michael-Mozilla,

Since this remains worksForMe, can you make an HTTP log for me with the failing scenario?

It should be similar to https://developer.mozilla.org/en-US/docs/HTTP_Logging except the NSPR_LOG_MODULES like should contain negotiateauth:5

i.e.

set NSPR_LOG_MODULES=timestamp,nsHttp:5,nsSocketTransport:5,nsHostResolver:5,=negotiateauth:5

and set NSPR_LOG_FILE to something which you will upload.

Feel free to read it and redact if 100% necessary - or send it to me privately (pmcmanus@mozilla.com) if that works better. If you do redact please point those out for me.

hopefully with that we can figure out why it isn't sending the request with the Proxy-Authorization that gets challenged.
(Assignee)

Updated

5 years ago
Flags: needinfo?(michael-mozilla)

Comment 9

5 years ago
Patrick,

Have sent you the nspr log privately.  Unfortunately I can't test with just one proxy; I have no control over my local bureaucracies' proxies :(.
Flags: needinfo?(michael-mozilla)
(Reporter)

Comment 10

5 years ago
Patrick,

we do indeed have a proxy hierarchy. However, web browsers are configured to access Microsoft ISA server 2006 SP1 running on Microsoft Windows Server 2003 with integrated NLB (1 virtual IP, 3 servers). ISA is configured to do "integrated" auth.

I sent you an NSPR log privately, too. (Cannot easily do a WS scan.)
(Assignee)

Comment 11

5 years ago
micheal-mozilla, will69,

do you guys have "broken" dns at your desktop? i.e. most DNS is handled by the proxy and your desktop can only successfully resolve a few local names?

that would be consistent with the log and would explain why I can't reproduce. (I do have proxies setup but I also have working internet connectivity).
(Reporter)

Comment 12

5 years ago
Yes, DNS traffic is restricted: Resolving internet hosts is not possible.
(Assignee)

Comment 13

5 years ago
(In reply to will69 from comment #12)
> Yes, DNS traffic is restricted: Resolving internet hosts is not possible.

Awesome - the problem is likely in that error path then.
(Assignee)

Comment 14

5 years ago
If I break resolution on my local computer I can reproduce the problem, and verify a fix. will69, michael-mozilla - thank you so much for the logs; they were key to finding the problem.

Test nightlies are available here if you wish to confirm the fix.

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-b30093f4dfd3/try-win32/
(Assignee)

Comment 15

5 years ago
Created attachment 671915 [details] [diff] [review]
patch 0

Christian, normally I would put this in honza's queue - but he is out for the week and as we probably want to port this to beta (where the regression originates) before it hits release every day counts. Its a tiny change, hope you can help.

The offending code is part of this patch - 
https://hg.mozilla.org/mozilla-central/rev/959f9da9f85e

The impact of it is that a DNS failure became a hard error when it should have not stopped processing.
Attachment #671915 - Flags: review?(cbiesinger)
Comment on attachment 671915 [details] [diff] [review]
patch 0

Review of attachment 671915 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.
Attachment #671915 - Flags: review?(cbiesinger) → review+
(Assignee)

Comment 17

5 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/073b904aed0c
(Assignee)

Updated

5 years ago
status-firefox16: --- → unaffected
status-firefox17: --- → affected
status-firefox18: --- → affected
status-firefox19: --- → fixed
(Assignee)

Comment 18

5 years ago
Comment on attachment 671915 [details] [diff] [review]
patch 0

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 767158 (introduced in ff17)
User impact if declined: Some users won't be able to browse the web at all with firefox. impacted users would be using a proxy/firewall that requires windows integrated authorization (aka kerberos, aka sspi, sometimes aka "negotiate") and prevents internet DNS inside the firewall
Testing completed (on m-c, etc.): problem reproduced (and fixed) and correlated with 2 different reporter logs.
Risk to taking this patch (and alternatives if risky): low.. 
String or UUID changes made by this patch:  none
Attachment #671915 - Flags: approval-mozilla-beta?
Attachment #671915 - Flags: approval-mozilla-aurora?

Comment 19

5 years ago
hello,
this patch will he integrated in FF17? because I have the same problem on bluecoat proxies with authentication negotiate / kerberos / ntlm.
The problem is solved in FF19 nightly (19.0a1 (2012-10-16))
Thanks.
(Assignee)

Comment 20

5 years ago
(In reply to PaulD from comment #19)
> hello,
> this patch will he integrated in FF17? because I have the same problem on
> bluecoat proxies with authentication negotiate / kerberos / ntlm.
> The problem is solved in FF19 nightly (19.0a1 (2012-10-16))
> Thanks.

PaulD, Thanks for the info. Backporting is what the approval? flags on the patch are for - I have nominated the patch for porting to 18 and 17. The release team regularly triages all patches with those nominations and decides whether or not they should be backported.

Comment 21

5 years ago
Hi Patrick,

Can confirm fixed with your nightly build.
Attachment #671915 - Flags: review+
(Reporter)

Comment 22

5 years ago
Tryserver build fixes the problem. Thanks a lot Patrick.
https://hg.mozilla.org/mozilla-central/rev/073b904aed0c
Status: UNCONFIRMED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
Should this have a test?
Flags: in-testsuite?

Updated

5 years ago
tracking-firefox17: --- → +
tracking-firefox18: --- → +
Comment on attachment 671915 [details] [diff] [review]
patch 0

[Triage Comment]
Approving for Aurora/Beta because this is a low-risk fix to a new regression. If we see any fallout from this, we should strongly consider just backing out bug 767158. Please land as soon as possible.
Attachment #671915 - Flags: approval-mozilla-beta?
Attachment #671915 - Flags: approval-mozilla-beta+
Attachment #671915 - Flags: approval-mozilla-aurora?
Attachment #671915 - Flags: approval-mozilla-aurora+
(Assignee)

Comment 26

5 years ago
  https://hg.mozilla.org/releases/mozilla-beta/rev/36eb48b20b27
status-firefox17: affected → fixed
(Assignee)

Comment 27

5 years ago
https://hg.mozilla.org/releases/mozilla-aurora/rev/9364fc859c40
status-firefox18: affected → fixed

Updated

5 years ago
Duplicate of this bug: 804574
(Assignee)

Comment 29

5 years ago
backout of 767158 and 785050 from ff17 beta
  https://hg.mozilla.org/releases/mozilla-beta/rev/757f408c1494

Updated

5 years ago
status-firefox17: fixed → affected
(Assignee)

Comment 30

5 years ago
I'm not sure why scoobidiver changed this from fixed to affected for ff17, but since it is tracked for ff17 I will be clear:

785050 was a in place fix for a regression from 767158.

There was nothing wrong with 785050, but due to further problems stemming from 767158, 767158 and 785050 were backed out for ff17 beta.

Therefore there was a problem and that problem is still fixed (i.e. no regression).
Setting status-firefox17 back to fixed based on comment 30.
status-firefox17: affected → fixed
(Assignee)

Comment 32

4 years ago
bug 8040605 requires backing out 767158 from beta 18.. this bug is a spotfix to 767158 so it should come out of 18 too, but as the root problem is backed out too 18 changes to unaffected.
status-firefox17: fixed → unaffected
status-firefox18: fixed → unaffected

Comment 33

4 years ago
Was it backed out from Beta and Aurora to fix bug 804605?
(Assignee)

Comment 34

4 years ago
(In reply to Scoobidiver from comment #33)
> Was it backed out from Beta and Aurora to fix bug 804605?

It was, but you don't need to reopen this bug as a separate item -it was just a follow fix and when the code goes back into the tree it will all go in together. (its off all branches right now.)
You need to log in before you can comment on or make changes to this bug.