Closed Bug 785050 Opened 12 years ago Closed 12 years ago

ISA proxy negotiate auth broken (regression)

Categories

(Core :: Networking, defect)

Product:
Core
Component:
Networking
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla19
Tracking Flags:
Tracking Status
firefox16 --- unaffected
firefox17 + unaffected
firefox18 + unaffected
firefox19 --- fixed

People

(Reporter: will69, Assigned: mcmanus)

References

(Blocks 1 open bug)

Details

(Keywords: regression)

Attachments

(1 file)

patch 0
1.41 KB, patch
jduell.mcbugs
: review+
Biesinger
: review+
akeybl
: approval-mozilla-aurora+
akeybl
: approval-mozilla-beta+
Details | Diff | Splinter Review 
Requesting any web page yields HTTP error 407 (proxy authentication required) and status 12209 (The ISA Server requires authorization to fulfill the request. Access to the Web Proxy service is denied).

STR: Use a MS ISA server as your proxy with a default configuration of Firefox 17.

Workaround: Set "network.negotiate-auth.allow-proxies" from "true" to "false" in "about:config" (just a double click)

Regression range:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=20db7c6d82cc&tochange=8b96a33ecbd2

Download:

GOOD: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/07/2012-07-26-05-28-03-mozilla-central/firefox-17.0a1.en-US.win32.installer.exe

FAIL: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/07/2012-07-27-03-05-08-mozilla-central/firefox-17.0a1.en-US.win32.installer.exe
Thank you for the excellent bug report with a regression range !
I suspect bug 767158
Blocks: 767158
Component: General → Networking
Keywords: regression
Product: Firefox → Core
Assignee: nobody → mcmanus
will69 - do you know what kind of authorization your proxy is doing? Specifically

1] do you know if it is NTLM or Negotiate?

2] is it full windows integrated auth? Do you normally have to type in your credentials to authenticate to the proxy or is it (before this bug) transparent to you if you are logged in?

thanks
3] (probably redundant, but what the heck) is your windows computer attached to a domain controller that is being used to determine the auth for the ISA proxy?
1] I’m afraid I need some help to find out. Is there a plugin I can use?
2] It is fully integrated. Before this bug, I did not have to type in my credentials.
3] The computer is a member of one domain. The user is a member of another domain. Both are in the same forest. Access to the proxy is granted to the user via a security group.
will I'm afraid this is works for me. I've setup a windows ADC and I've setup a copy of wingate that uses windows integrated autthentication (negotiate::kerberos) from the ADC, and I've added my laptop to that domain..

and I can confirm that the auth is enforced and works in nightly, ff15, and msie all the same.

not sure where to go here.. can you do a packet trace (see wireshark) with both nightly and ff15 or ff16 so we can see the differences?
I think I've run into this on ff17 beta, and having just downgraded to 16.0.1, I'm sure of it.

Let me know if you need anything else for debugging purposes.  network.*auth* is set to default, except for network.negotiate-auth.trusted-uris, which is set to a domain specific value I can't really disclose.  However, unsetting network.negotiate-auth.trusted-uris doesn't do anything.

I do have FoxyProxy installed, but disabling it doesn't improve matters. 

From wireshark, my initial (working, ff16.0.1) trace says:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 XXXXXXXXXXX
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 737

Which is followed by:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org
Proxy-Authorization: Negotiate TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw==

HTTP/1.1 407 Proxy Authentication Required ( Access is denied.  )
Via: 1.1 XXXXXXXXXXXXX
Proxy-Authenticate: Negotiate TlRMTVNTUAACAAAADgAOADgAAAAFgomib3t0+h9SBh0AAAAAAAAAAMQAxABGAAAABQLODgAAAA9PAEMARQBBAE4ASQBBAAIADgBPAEMARQBBAE4ASQBBAAEAGgBQAFgAWQBOAFoAMAA2ADEATABEAEEAMAAwAAQAKABvAGMAZQBhAG4AaQBhAC4AYwBvAHIAcAAuAGEAbgB6AC4AYwBvAG0AAwBEAFAAWABZAE4AWgAwADYAMQBMAEQAQQAwADAALgBvAGMAZQBhAG4AaQBhAC4AYwBvAHIAcAAuAGEAbgB6AC4AYwBvAG0ABQAYAGMAbwByAHAALgBhAG4AegAuAGMAbwBtAAAAAAA=
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 0     

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org
Proxy-Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAIAAAAAYABgAmAAAAA4ADgBIAAAAEAAQAFYAAAAaABoAZgAAAAAAAACwAAAABYKIogUBKAoAAAAPTwBDAEUAQQBOAEkAQQByAG8AYgBpAG4AcwBtADUAVwA1AEMAMgA2ADAAQQA0ADQAMQAyADkARQBdpiGV1aa44gAAAAAAAAAAAAAAAAAAAABcYnHGzWxuf2pjeRA5bWVzd/pbSh8Lkbw=

HTTP/1.1 200 Connection established
Via: 1.1 XXXXXXXXXXXXX, 1.1 XXXXXXXXXXXXX
Connection: Keep-Alive
Proxy-Connection: Keep-Alive

And then a non-working version (17.0b1), using the same profile:

CONNECT bugzilla.mozilla.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/17.0 Firefox/17.0
Proxy-Connection: keep-alive
Host: bugzilla.mozilla.org

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 XXXXXXXXXXXXX
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 737   

Followed by, well, nothing.
this is still wfm, this time using TMG 2010 which is the successor to ISA.

Michael-Mozilla, it looks like you've got a double depth proxy of some sort based on 

HTTP/1.1 200 Connection established
Via: 1.1 XXXXXXXXXXXXX, 1.1 XXXXXXXXXXXXX

any chance you can test with just 1 proxy (I know maybe you can't).

will69, any chance you've got 2 layers of proxies too?
Michael-Mozilla,

Since this remains worksForMe, can you make an HTTP log for me with the failing scenario?

It should be similar to https://developer.mozilla.org/en-US/docs/HTTP_Logging except the NSPR_LOG_MODULES like should contain negotiateauth:5

i.e.

set NSPR_LOG_MODULES=timestamp,nsHttp:5,nsSocketTransport:5,nsHostResolver:5,=negotiateauth:5

and set NSPR_LOG_FILE to something which you will upload.

Feel free to read it and redact if 100% necessary - or send it to me privately (pmcmanus@mozilla.com) if that works better. If you do redact please point those out for me.

hopefully with that we can figure out why it isn't sending the request with the Proxy-Authorization that gets challenged.
Flags: needinfo?(michael-mozilla)
Patrick,

Have sent you the nspr log privately.  Unfortunately I can't test with just one proxy; I have no control over my local bureaucracies' proxies :(.
Flags: needinfo?(michael-mozilla)
Patrick,

we do indeed have a proxy hierarchy. However, web browsers are configured to access Microsoft ISA server 2006 SP1 running on Microsoft Windows Server 2003 with integrated NLB (1 virtual IP, 3 servers). ISA is configured to do "integrated" auth.

I sent you an NSPR log privately, too. (Cannot easily do a WS scan.)
micheal-mozilla, will69,

do you guys have "broken" dns at your desktop? i.e. most DNS is handled by the proxy and your desktop can only successfully resolve a few local names?

that would be consistent with the log and would explain why I can't reproduce. (I do have proxies setup but I also have working internet connectivity).
Yes, DNS traffic is restricted: Resolving internet hosts is not possible.
(In reply to will69 from comment #12)
> Yes, DNS traffic is restricted: Resolving internet hosts is not possible.

Awesome - the problem is likely in that error path then.
If I break resolution on my local computer I can reproduce the problem, and verify a fix. will69, michael-mozilla - thank you so much for the logs; they were key to finding the problem.

Test nightlies are available here if you wish to confirm the fix.

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-b30093f4dfd3/try-win32/
Attached patch patch 0 — — Splinter Review 
Christian, normally I would put this in honza's queue - but he is out for the week and as we probably want to port this to beta (where the regression originates) before it hits release every day counts. Its a tiny change, hope you can help.

The offending code is part of this patch - 
https://hg.mozilla.org/mozilla-central/rev/959f9da9f85e

The impact of it is that a DNS failure became a hard error when it should have not stopped processing.
Attachment #671915 - Flags: review?(cbiesinger)
Comment on attachment 671915 [details] [diff] [review]
patch 0

Review of attachment 671915 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.
Attachment #671915 - Flags: review?(cbiesinger) → review+
Comment on attachment 671915 [details] [diff] [review]
patch 0

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 767158 (introduced in ff17)
User impact if declined: Some users won't be able to browse the web at all with firefox. impacted users would be using a proxy/firewall that requires windows integrated authorization (aka kerberos, aka sspi, sometimes aka "negotiate") and prevents internet DNS inside the firewall
Testing completed (on m-c, etc.): problem reproduced (and fixed) and correlated with 2 different reporter logs.
Risk to taking this patch (and alternatives if risky): low.. 
String or UUID changes made by this patch:  none
Attachment #671915 - Flags: approval-mozilla-beta?
Attachment #671915 - Flags: approval-mozilla-aurora?
hello,
this patch will he integrated in FF17? because I have the same problem on bluecoat proxies with authentication negotiate / kerberos / ntlm.
The problem is solved in FF19 nightly (19.0a1 (2012-10-16))
Thanks.
(In reply to PaulD from comment #19)
> hello,
> this patch will he integrated in FF17? because I have the same problem on
> bluecoat proxies with authentication negotiate / kerberos / ntlm.
> The problem is solved in FF19 nightly (19.0a1 (2012-10-16))
> Thanks.

PaulD, Thanks for the info. Backporting is what the approval? flags on the patch are for - I have nominated the patch for porting to 18 and 17. The release team regularly triages all patches with those nominations and decides whether or not they should be backported.
Hi Patrick,

Can confirm fixed with your nightly build.
Tryserver build fixes the problem. Thanks a lot Patrick.
https://hg.mozilla.org/mozilla-central/rev/073b904aed0c
Status: UNCONFIRMED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
Should this have a test?
Flags: in-testsuite?
Comment on attachment 671915 [details] [diff] [review]
patch 0

[Triage Comment]
Approving for Aurora/Beta because this is a low-risk fix to a new regression. If we see any fallout from this, we should strongly consider just backing out bug 767158. Please land as soon as possible.
Attachment #671915 - Flags: approval-mozilla-beta?
Attachment #671915 - Flags: approval-mozilla-beta+
Attachment #671915 - Flags: approval-mozilla-aurora?
Attachment #671915 - Flags: approval-mozilla-aurora+
backout of 767158 and 785050 from ff17 beta
  https://hg.mozilla.org/releases/mozilla-beta/rev/757f408c1494
I'm not sure why scoobidiver changed this from fixed to affected for ff17, but since it is tracked for ff17 I will be clear:

785050 was a in place fix for a regression from 767158.

There was nothing wrong with 785050, but due to further problems stemming from 767158, 767158 and 785050 were backed out for ff17 beta.

Therefore there was a problem and that problem is still fixed (i.e. no regression).
Setting status-firefox17 back to fixed based on comment 30.
status-firefox17: affected → fixed
bug 8040605 requires backing out 767158 from beta 18.. this bug is a spotfix to 767158 so it should come out of 18 too, but as the root problem is backed out too 18 changes to unaffected.
status-firefox17: fixed → unaffected
status-firefox18: fixed → unaffected
Was it backed out from Beta and Aurora to fix bug 804605?
(In reply to Scoobidiver from comment #33)
> Was it backed out from Beta and Aurora to fix bug 804605?

It was, but you don't need to reopen this bug as a separate item -it was just a follow fix and when the code goes back into the tree it will all go in together. (its off all branches right now.)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: