Closed Opened 15 years ago Closed 5 years ago

# NTLM Proxy authentification dialog pops up over and over

x86
Windows XP
Not set
normal

## Tracking

### ()

RESOLVED DUPLICATE of bug 486508

## Attachments

### (10 files)

 53.53 KB, application/x-zip Details 34.03 KB, application/x-zip-compressed Details 63.51 KB, application/zip Details 1.11 KB, patch Details | Diff | Splinter Review 258.18 KB, text/plain Details 116.41 KB, text/plain Details 383.70 KB, text/plain Details 50.95 KB, text/plain Details 2.44 KB, text/plain Details 13.91 KB, image/png Details
I'm using Ff in a corporate environement, where we connect to the interne throught a proxy. This used to work fine in Ff 1.0.7, in 1.5 however I constantly need to click on the ok button from the auth dialog. Hence to load bugzilla.mozilla.org I needed to push that button 8 times.
This is a regression from 1.0.7.
*** Bug 318286 has been marked as a duplicate of this bug. ***
I think CORE/HPPT:Network is the right component, ain't sure so I'm ccing network people in order to triagge correctly this bug.
Assignee: nobody → darin
Component: General → Networking
Product: Firefox → Core
QA Contact: general → benc
Version: 1.5 Branch → 1.8 Branch
*** Bug 318578 has been marked as a duplicate of this bug. ***
This should be fixed because it make Firefox 1.5 useless in corporate space hence requesting the blocker flags.
Flags: blocking1.8.1?
Flags: blocking1.8.0.1?
When I read the first part of the description below I could have sworn I was reading a description of my find (#319336). But mine is a little different.

In my case I don't get the required auth dialog so I can't get out. FF 1.07 was AOK so is {shudder} IE 6.0!

I suspect the two may be related somehow.

Attached file Nspr logs
This is the nspr logs obtained with  nsHttp:5, nsSocketTransport:5, nsHostResolver:5. I've changed the name of the proxy. The recorded session dit the following :
<ul>
<li> ctrl-l and then bugzilla.mozilla.org
<li> clicked a few time on user/password confirmation dialog.
</ul>
This only affects NTLM proxy auth, right? Normal basic proxy auth seems to work fine here.
(In reply to comment #7)
> This only affects NTLM proxy auth, right? Normal basic proxy auth seems to work
> fine here.


Summary: Proxy authentification dialog pops up over and over → NTLM Proxy authentification dialog pops up over and over
Need trunk-baked patch before we can take for 1.8.0.1
Flags: blocking1.8.0.1? → blocking1.8.0.1-
This appears to be much more severe than the noted bug.  From what I can tell, NTLM authentication on windows is broken in firefox 1.5 on windows.

I have an internal website which uses NTLM authentication; it's actually using mod_ntlm2 under apache 2.0.54.  There is NO proxy involved in my case.

Internet Explorer 6.0 authenticates against it perfectly; works all the time.

Konqueror 3.4.2 on linux authenticates against it perfectly; works all the time.

Firefox 1.5 on windows initially authenticates against it, after clicking on approximately a half-dozen different locations within the site (top level is protected by NTLM auth), it starts endlessly prompting for username and password.

Firefox 1.07 & 1.5 on linux exhibits same behaviour as firefox 1.5 on windows, but takes closer to a couple dozen different clicks to exhibit the behavior.

I did not test on firefox 1.07 windows.
Ross: Can you please also post a NSPR log file to help us understand if the problem you are seeing is identical to that reported by Ludovic?

Some info on NSPR logging here:
http://www.mozilla.org/projects/netlib/http/http-debugging.html

You can capture this log using a release build of Firefox.

Thanks!
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.9alpha
Attached file NSPR log as requested
Here's the NSPR log as you requested.  I connected to the website, authenticated, browsed to a couple different links, got the repeating authentication prompt, filled it in and hit OK 3 times, then hit cancel on the 4th, then exited firefox.

This was using firefox 1.5 on windows.

Any more information you'd like, let me know.  This will cause me innumerable headaches if I deploy my intranet site as it is.
*** Bug 336502 has been marked as a duplicate of this bug. ***
So what can be done about this?
Flags: blocking1.8.1? → blocking1.8.1+
The log file shows the cached username and password being rejected by the server.  I'm not sure why that is happening.  Are you sure this isn't a bug in mod_ntlm2?
(In reply to comment #15)
> The log file shows the cached username and password being rejected by the
> server.  I'm not sure why that is happening.  Are you sure this isn't a bug in
> mod_ntlm2?
>

It seems pretty unlikely to be a bug in mod_ntlm2, since I have experienced the same issue through a Microsoft ISA Server proxy.
(Unfortunately I don't have control over this server, and its proxy authentication is currently disabled, so I cannot generate any logs)
Either a bug in firefox, or firefox is implementing the spec and not the reality.  IE, Safari, Konqueror & Opera all work fine against a server running mod_ntlm2.
OK, good to know.  Unfortunately, it just isn't clear to me what is going wrong because the log file doesn't actually include the NTLM payloads.  If someone is willing to capture a tcpdump of the network traffic and share that with me, then I might be able to resolve this bug.
As per your request, here are captures. Done with latest version of wireshark.
One additional comment- NTLM is a challenge/response authentication method, and I've never yet seen a failure on the initial authentication.  I've also noted that how many authentications succeed before firefox starts failing can vary significantly.  I think either the cached credentials are becoming corrupted, or there are different routines handling the initial NTLM authentication versus subsequent reauthentications, and there's a bug in the reauthentication routine. FWIW.
Thanks for the traces, Ross.  It still isn't entirely clear to me why Firefox is failing to authenticate, but there are some differences in the flags sent by IE and Firefox.  It's hard to compare the final request from Firefox to that of IE since they would anyways differ as a result of being fed different nonces by the server.  I really wish this NTLM protocol were properly documented and standardized :-(
Target Milestone: mozilla1.9alpha → mozilla1.8.1beta2
Minusing for 1.8.1 given the lack of progress and that we did ship 1.8 with it.
Flags: blocking1.8.1+ → blocking1.8.1-
For me it happens really often to go to customer sites that use NTLM and ISA Server as proxy server with my PC (registered on my company Windows domain).
I noticed that I, as not Windows domain authenticated at the customer site, have problems. People working there, authenticated on the domain, do not suffer this bug (or maybe they're prompted once to authenticate to the proxy).
I will do my best to check this on a better way....just my 1cent...
In my company I still have this validation problem evern using FF 2.0. The only version of FF that still works is 1.0.7. This problem makes impossible to upgrade FF.
(In reply to comment #24)
> In my company I still have this validation problem evern using FF 2.0. The only

Could you provide a test environment software configuration so the developers could try to understand what is going on ?

One other thing you could do is try to figure out the regression window so it can be easier to see what change broke the NTLM stuff.
I've already provided wireshark captures of IE doing the "right thing" and firefox doing the "wrong thing".  In point of fact, for me this bug isnt even involving a proxy; I'm simply pointing firefox at a website which uses mod_ntlm authentication  for apache backending to our Active Directory site. I had been solving this with IETab extension, but as of firefox 2.0, I'm seeing firefox commonly crash when it goes to open an IE tab (not always, but commmonly).  At this point I dont reguard firefox 2.0 as usable because of that combination of bugs.
(In reply to comment #26)
> I've already provided wireshark captures of IE doing the "right thing" and
> firefox doing the "wrong thing".

Would it be possible to provide captures for the same test case in both FF1.0.7 (i.e. before the regression occurred), and FF1.5 (where the problem was first noticed). I think this would give a better apples-to-apples comparison, and be easier to interpret than FF vs. IE captures.

I know that this doesn't fully address your issue Ross, since you see similar (but not identical) symptoms with 1.0.7, but hopefully it has a common root cause.
Me and my friend have wireless user accounts at a university. My version of XP is 'home addition' and my friend has 'Pro'. The miracle is that when my friend wanted to log into his account he logged into IE because i told him firefox has problems. when he logged into IE he had an option that i didnt have in my IE and that was: when he was prompted for the username and password there was an option if he wanted to save the username and passowrd. he ticked the box and click ok and then everything ran smoothly for him. he never had to re-enter the password for IE again. I told him to open firefox and he wasnt prompted for the password then either. I couldnt believe it and couldnt figure out why I didnt have the option he did. Other students in my uni got user accounts, some had XP Pro and others home addition and i noticed that all the Xp Pro users had the option that my firend had but all the Home users didnt and when all the pro isers tick the 'save username and password' box, they had no problems with firefox at all. I hope this info helps in finding a solution to this problem. is there anyway we can get the professional version of IE for Home Edition? (",)
Using Firefox 2.0 with Squid, the same apply with digest authentication when the nonce expires.

It is very annoying on website that "refresh themselves" like Gmail: you soon get a tons of proxy authentication prompts.
A colleague pointed me to the following blog post (http://andri.andriani.web.id/2006/05/proxy-di-firefox-15x), which points back to a post on the Mozillazine forums (http://forums.mozillazine.org/viewtopic.php?p=2250158#2250158).

These posts give a possible workaround for this NTLM authentication problem. I haven't had a chance to test it yet, but the suggestion is to edit the Firefox preferences (type about:config in the address bar), and set the network.negotiate-auth.allow-proxies option to false.


I can confirm that the workaround I posted about above (comment #30) works for this bug in my situation, that is Firefox 2.0, NTLM authenticating to a Microsoft ISA Server proxy.


I confirm too that setting the network.negotiate-auth.allow-proxies option to false in Firefox 2.0 fixes the NTLM authentication problem with Microsoft ISA Server proxy.

I'm running squid with ntlm authentication against w2k AD. After upgrading Firefox to 2.0.0.3 (fc7) I still receive multiple ntlm prompts especially with gmail. Looks like this was supposed to be fixed in Firefox 1.8.x Does anyone know the status of this bug? (The network.negotiate-auth.allow-proxies workaround does *not* work in my case.)
The error message reported by Samba ntlm_auth suggest firefix looses track of which connection the authentication is being done on, sending the response to the server challenge on a new connection.

Unlike the standard HTTP authentication methods which is message based, NTLM (and Negotiate/Kerberos) is connetion oriented just masquerading themselves as HTTP authentication, and all steps of the authentication process must be carried out on the same connection (client negotiate packet, server challenge packet, and client response packet). Once the three packet handshake is complete the server has authenticated the TCP connection, and any further requests on the same TCP connection is implicitly authenticated.

I am getting this bug.  See post in Mozilaazine forum for info and screen captures.  Operating on a Squidy proxy system.

Post is at http://forums.mozillazine.org/viewtopic.php?t=540718
sorry, i'm not working on this bug.  re-assigning to default owner, etc.
Assignee: darin.moz → nobody
Status: ASSIGNED → NEW
QA Contact: benc → networking
Here's steps to reproduce:

1. install squid

2. before the line "acl all src 0.0.0.0/0.0.0.0" add the following lines:

auth_param ntlm program /usr/lib/squid/fakeauth_auth                            auth_param ntlm children 5                                                      auth_param ntlm keep_alive on                                                   acl auth_users proxy_auth REQUIRED                                              http_access allow auth_users

3. start squid

5. go to www.google.com.au, you will be prompted for authentication, enter
'foo\bar' and 'foo'

6. go to https://59.167.195.44 the user is 'moz' and the password is 'dev'

7. you will now be presented with the NTLM login dialog again.

Looks to me that FireFox is forgetting the proxy-authentication credentials
when doing a WWW-authenticate over HTTPS.

The instructions in comment #37 exhibit the problem when using
Firefox on Mac OS X.  I tested using Firefox on Linux and did
not encounter the problem.
Scratch #38, the problem exists in Linux too.
Comment on attachment 280872 [details] [diff] [review]
Fixes broken NTLM Proxy Authentication

There doesn't seem to be a nice way to do this.  A connection:close is issued inside a SSL CONNECT and when the connection is closed the authentication isn't being deleted.

I would have liked to clean it up at connection close but the nsHttpConnectionMgr class has no reference to the nsHttpChannel which owns the credentials.
Attachment #280872 - Attachment is patch: true
Attachment #280872 - Attachment mime type: application/octet-stream → text/plain
This patch is fine on iceweasel-2.0.0.6/Debian GNU/Linux and firefox-2.0.0.7/MS Windows XP.
But I caught a compiler error on iceweasel-2.0.0.11-1/Debian GNU/Linux testing with the latest patch  https://bugzilla.mozilla.org/attachment.cgi?id=280872
At first gcc claims at line 2208;  'ident-Clear();' ident is not pointer.

I changes it to 'ident.Clear();', but I caught the other error
nsHttpChannel.cpp:2208: error: passing 'const nsHttpAuthIdentity' as this argument of 'void nsHttpAuthIdentity::Clear()' discards qualifiers.

gcc version is 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)

To fix this, we need to change argument type from 'const nsHttpAuthIdentity &ident' to 'nsHttpAuthIdentity &ident' in nsHttpChannel::GenCredAndSetEntry(). (in nsHttpChannel.cpp and nsHttpChannel.h)

These two changes makes a compile fine.

Same error occurs with the same patch on iceweasel 2.0.0.6 with gcc 4.2.3 20071123.

> nsHttpChannel.cpp: In member function nsresult nsHttpChannel::GenCredsAndSetEntry(nsIHttpAuthencitator*, PRBook, const char*, const char*, PRInt32, const char*, const char*, const char*, nsHttpAuthIdentity&, nsCOMPtr<nsISupports>&, char**);
>nsHttpChannel.cpp:2208: error: base operand of -&gt; has non-pointer type nsHttpAuthIdentity

I try changing it to ident.Clear() and remove 'const' declaration of argument then it is  compiled fine.
Is this patch good for  >= 2.0.0.6?
Hiroshi, I didn't have time to complete testing of the bug fix I did back in September,
I don't think it covers off all cases from memory.  Probably we need to get the regular
maintainer to look into this problem.

Does this bug depends on <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=230190">230190</a>??
It seems to be another issue.  The issue here is that when doing WWW-Authenticate inside a
Proxy-Authenticated issue FF/Moz does not correctly clean up the session credentials when
the connection is closed.

It then attempts to use the WWW-Authenticate credentials on the next connection, this fails
because the next connection needs the Proxy-Authentication credentials.
Hi, I am not sure if this is related but a customer is reporting that when using a proxy and makes a request to a HTTPS site "CONNECT menthod" with NTLM Transparent auth he gets prompted (but not all the time).

We had a look at the PCAP and noticed the following.

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9) Gecko/2008052906 Firefox/3.0\r\

Working Request:

No.     Time            Source                Destination           Protocol Info
10387 09:00:13.581999 10.12.138.100         10.10.140.50          HTTP     CONNECT www.etracker.de:443 HTTP/1.1 , NTLMSSP_AUTH, User: LV\UDVT0035

Within the NTLMSSP
Domain name: LV
Length: 4
Maxlen: 4
Offset: 72
User name: UDVT0035
Length: 16
Maxlen: 16
Offset: 76
Host name: MP
Length: 4
Maxlen: 4
Offset: 92

but then a few packets later we see

Failed Request:

No.     Time            Source                Destination           Protocol Info
10623 09:00:13.694000 10.12.138.100         10.10.140.50          HTTP     CONNECT www.etracker.de:443 HTTP/1.1 , NTLMSSP_AUTH, User: L\U

NTLMSSP
Domain name: L
Length: 4
Maxlen: 4
Offset: 72
User name: U
Length: 16
Maxlen: 16
Offset: 76
Host name: M
Length: 4
Maxlen: 4
Offset: 92

So it looks like details are trucated.

I've seen the following faulty behavior in Firefox 2.0.0.13:
* For several URLs (I've seen up to 10 on a single connection)
* Firefox sends a request with no authentication
* The proxy responds with a 407 requsting authentication
* Firefox sends a request for one URL with the NTLM hello (type 1) data.
* The proxy responds with a 407 that contains the NTLM challenge (type 2) data.
* Firefox sends a request for a different URL with another NTLM hello.
* The proxy isn't expecting another NTLM hello and fails to authenticate.

I'm not sure if this is a Firefox problem or an authentication module problem, but the following changes need to be made:

* Proxy 407 and site 401 responses should be processed immediately, so that further unauthenticated requests are not sent on the same socket. This is a waste of bandwidth.

* NTLM authentication needs to be performed correctly, ensuring that the hello message is always sent and is always followed up with a response to the challenge. If the client needs to abort the NTLM authentication sequence for any reason, it needs to close the connection so that the server doesn't get into a bad state.
I think this is the same issue as bug 439463 and bug 366562, I'm trying to link all these bugs together so that this issue gets a bit more attention. I don't know if there's a way of "sponsoring" a bug fix? My boss is very keen to get this issue looked at and fixed and asked me to find out if there's any way we can pay/donate to Mozilla to get the ball rolling?
This topic definately needs some attention payed by someone from the Mozilla engineering team!

https://bugzilla.mozilla.org/show_bug.cgi?id=318253, created 30.11.2005, Status: New
https://bugzilla.mozilla.org/show_bug.cgi?id=366562, created 10.01.2007, Status: Unconfirmed
https://bugzilla.mozilla.org/show_bug.cgi?id=501361, created 30.06.2009, Status: Unconfirmed
https://bugzilla.mozilla.org/show_bug.cgi?id=486508, created 02.04.2009, Status: Unconfirmed

In my opinion, the Mozilla team did a really good job in all the way, Firefox developed – but here is a glitch and that thing is really obvious – if Firefox is used in a business environment together with NTLM authentication.
I'm using 3.6 (Beta 4), the proxy auth. dialog appeared two times when i opened this page, it appears with every new tap and with the Add-ons manager !!

I don't experience this problem with FF 3.5.6 ....
(In reply to comment #51)
> I'm using 3.6 (Beta 4), the proxy auth. dialog appeared two times when i opened
>
> I don't experience this problem with FF 3.5.6 ....

Have you tried the latest nightly of 1.9.2? A patch landed yesterday afternoon that fixed an NTLM bug in the security changes we made recently.
Yes, I've tried the latest build this morning... the problem is still there but there is a change in behavior !

It ask me for authentication every time a new component load in the page, and it do not ask me again for authentication if I opened a another website that have the same components that I already authenticated them in the first page..

for example: If I opened a page with background and some png banners and text, the browser will ask me 3 times to do the authentication, then if I opened another website that have a background, text and a flash it will ask for authentication only once !!
Version: 1.8 Branch → Trunk
(In reply to comment #53)
> Yes, I've tried the latest build this morning... the problem is still there but
> there is a change in behavior !
>
> It ask me for authentication every time a new component load in the page, and
> it do not ask me again for authentication if I opened a another website that
> have the same components that I already authenticated them in the first page..
>
> for example: If I opened a page with background and some png banners and text,
> the browser will ask me 3 times to do the authentication, then if I opened
> another website that have a background, text and a flash it will ask for
> authentication only once !!

What's your network config Ahmed? Are you accessing the web through a proxy? If you are, a patch landed today that might help with that.
Yes, I'm accessing web behind an IIS proxy..

I'll update to today's build and I'll give you my feedback

Thanks :)
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b6pre) Gecko/20091218 Namoroka/3.6b6pre

unfortunately the problem is still there :(

I'm in a university that uses an IIS proxy server, I access that server using an IP number.

Why this problem appears only in Namoroka ?? while the 1.9.1 and 1.9.3pre a1 have no problems at all !!
Now Mozilla/5.0 Gecko/20091218 Minefield/3.7a1pre have the same problem, while the Mozilla/5.0 Gecko/20091216 Minefield/3.7a1pre is doing fine !
(In reply to comment #56)
> Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b6pre) Gecko/20091218
> Namoroka/3.6b6pre
>
> unfortunately the problem is still there :(
>
> I'm in a university that uses an IIS proxy server, I access that server using
> an IP number.
>
> Why this problem appears only in Namoroka ?? while the 1.9.1 and 1.9.3pre a1
> have no problems at all !!

Are you using single sign-on or do you expect to be prompted at least once for your credentials?

network.automatic-ntlm-auth.allow-proxies
> Are you using single sign-on or do you expect to be prompted at least once for

I expect to be prompted 'only' once at the start of a new browsing session.

>
> network.automatic-ntlm-auth.allow-proxies

Value: true

I have short video for this bug.. I'll try to upload it as soon as possible...
(In reply to comment #59)
> > Are you using single sign-on or do you expect to be prompted at least once for
>
> I expect to be prompted 'only' once at the start of a new browsing session.
>
> > What's the value of this pref in your about:config -
> >
> > network.automatic-ntlm-auth.allow-proxies
>
> Value: true
>
> I have short video for this bug.. I'll try to upload it as soon as possible...

Sounds like you are running into this bug, which is likely separate from the problems we had with the security patch. If you have the time, some log output might help to diagnose the problem.

https://developer.mozilla.org/en/HTTP_Logging

NSPR_LOG_MODULES=nsHttp:3,negotiateauth:4,NTLM:4
Platform should be changed to Windows All, and the Target Milestone should be 1.9.2
This is not going to make 1.9.2, but it might make it into an interim update if we can track down the cause.
Target Milestone: mozilla1.8.1beta2 → mozilla1.9.3
I've uploaded a video on YouTube shows the problem:
http://www.youtube.com/watch?v=fQw13U9mP2Q
This bug should be marked as a "know issue" for FireFox 3.6 (Gecko 1.9.2) ...
I never had the problem before, but it has started with Firefox 3.6 (typically with https://gmail.com) and wonder whether this issues are related:
* Bug 366562
* Bug 439463
* Bug 486508
Visitors of my website with Firefox 3.6 see a authentification pop-up at every loading page.
However, the website isn't protected by any password or SSL, and most of visitors don't use any proxy.
The window doesn't appears with previous versions of Firefox Windows or with Firefox Mac, IE7, IE8 Chrome or Opera.

You can see it here : http://lililabaleineverte.free.fr
> The window doesn't appears with previous versions of Firefox Windows or with
> Firefox Mac, IE7, IE8 Chrome or Opera.

Sorry, the problem exist in Firefox 2.6 Windows and Mac...
(In reply to comment #67)
> Visitors of my website with Firefox 3.6 see a authentification pop-up at every
> However, the website isn't protected by any password or SSL, and most of
> visitors don't use any proxy.
> The window doesn't appears with previous versions of Firefox Windows or with
> Firefox Mac, IE7, IE8 Chrome or Opera.
>
> You can see it here : http://lililabaleineverte.free.fr

The problem is the source specified for the snowstorm.js script (ftp://ftpperso.free.fr), not a Firefox issue...
Was extended protection installed on the system? (http://www.microsoft.com/technet/security/advisory/973811.mspx)

We are noticing problems with Firefox IWA auth when that is installed. We are noticing this while doing federated authentication over ADFS
Target Milestone: mozilla1.9.3 → mozilla2.0
Duplicate of this bug: 602768
Blocks: 250014
Guys, whats the progress of this bug? I can confirm comment #64. Very annoying and a show-stopper for firefox rollout in our corporate environments.

problem persists in yesterdays ftp://ftp.mozilla.org/pub/firefox/nightly/latest-mozilla-central/firefox-4.0b8pre.en-US.win32.installer.exe build.

thanks

stefan
Duplicate of this bug: 610202
reproducible in mozilla 1.9.1, tested on Seamonkey 2.0.10
This regression is evident in Fx 4 + we're about to go RC. I can already see an email going around many corporations (including mine) warning people against using Firefox 4. The application is unusable in this state. Any chance this could be *at least* soft blocking? I'd propose it myself but I'm not familiar with the current version numbers + procedure.
We are experiencing a similar issue as this bug in our corporate environment. We've rolled out Firefox and are getting consistent widespread reports of proxy authentication popups with certain sites. As far as I can tell it occurs when our users access a site which we have white listed on our proxy server to not require authentication (Facebook, Yammer, G+, etc) and it has content on it from a third party site which requires authentication. Firefox asks for credentials repeatedly and throws 407 authentication required errors but will not accept the credentials that are input.

User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

I've tested and this is occurring for all versions of Firefox from 4.x to 7.x we currently use. Other browsers in use are not affected (IE, Chrome, Opera).
twisted.au: I completely agree with you. We are experiencing same regression with Firefox and NTLM authentication. In latest Firefox releases there are too many authentication window pops up (crrently tested with 9.x and 10.x versions on Windows2003/Windows7). It is over and over while browsing in same window. We are deciding to use IE in our environment or just move to Opera or Chrome.
Please can you fix this annoyning regression? many thanks.
I'm here again to ask a solution to this problem that is even worst in version 9.0 and 10.0. Now is really hard to read many pages while you have to confirm the credentials over and over. Thanks, Gabriele.
(In reply to Gabriele Di Maio from comment #78)
> I'm here again to ask a solution to this problem that is even worst in
> version 9.0 and 10.0. Now is really hard to read many pages while you have
> to confirm the credentials over and over. Thanks, Gabriele.

Just trying to confirm, the problem regressed in 9.0? Was 8.0 working fine, or?.. Please provide some detail on what has changed.
The problem never completely disappeared since FF 1.0.7 (!) and from version 9.0 it is increased. I don't remember exactly what was the situation in version 8.0.

I hope this help.

Thanks, Gabriele.
Guys, thank you for reviving this thread. It should be processed quickest, problem still persist during many major versions of Firefox.
Can we get some exact details on how networking/NTLM proxy is setup in cases where this increased?
Also, this might be a pain but if someone can trace back to a particular build for the recent regression using nightly zips that would be a huge help.

http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/
> Can we get some exact details on how networking/NTLM proxy is setup in cases
> where this increased?

These are my NTLM settings:

network.auth.force-generic-ntlm             default  Boolean False
network.automatic-ntlm-auth.allow-proxies   user set Boolean False
network.automatic-ntlm-auth.trusted-uris    default  String
network.ntlm.send-lm-response               default  Boolean False

For the "NETWORK" parameters I need to know an easy mode to report all of them (since they are a lot!).

Gabriele
Also see Comment 58, it includes some regression range. But I'm not sure what actually regressed at that time as the problem is already quite old and has changed over time.
Sorry, I meant comment 57
We are currently using Firefox 8.0.1 for our devs, and also expirience this issue. It doesn't appear any better under Firefox 8.

Our NTLM settings are:
network.auth.force-generic-ntlm             default  Boolean  False
network.automatic-ntlm-auth.allow-proxies   user set Boolean  True
network.automatic-ntlm-auth.trusted-uris    default  String   <contains internal sites>
network.ntlm.send-lm-response               default  Boolean  False
network.proxy.autoconfig_url                locked   string  <contains link to pac file>
network.proxy.type                          locked   integer  2

I'll try the same settings on a nightly build and post the results. A good site to test this on is to log onto (free registration) newscientist.com. The error appears straight away.
OK, I've been doing testing to find exactly when this bug crept in.
The bug first occurs in nightly build 20050510 1.0+, checking the build before it, the error doesn't exist (20050509 1.0+)

I managed to track it down to a single file that changed between builds: compreg.dat

I then copied the file from 20050509 to 20050510 and the error disappears. So what ever changed in that particular file between those 2 builds has carried on ever since. Hope this helps in narrowing down what is, a lot of changes have happened since then, but I would bet that that particular code has managed to propagate through every version of Firefox since then.

The url to the builds that I'm reffering to are:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005/05/2005-05-09-07-trunk/firefox-1.0+.en-US.win32.zip (this works correctly)
and
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005/05/2005-05-10-07-trunk/firefox-1.0+.en-US.win32.zip (this doesn't work - first manifestation of the bug)
Site I was using to test: http://www.yammer.com/
Note: Problem only occurs if you are using NTLM authentication. Problem won't occur if the proxy is straight pass-through.
My initial conclusions on the compreg.dat being the issue were a bit premature. I've done some further testing between the two, and the problem is something within firefox.exe itself. I ran windiff on the two versions to pull up a list of files that were different between the two versions and slowly swapped files in and out to narrow down where the issue lay.

From here on in I think it may require a programmer to investigate as IANAP so wouldn't be too sure what I'm looking for within the code.
Bonsai link to all checkins in that timeframe: http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2005-05-09&maxdate=2005-05-10+23%3A59%3A59&cvsroot=%2Fcvsroot
I think this should be the correct link as the Mozilla 1.8 branch for Firefox 1.1/1.5 was created later (2005-08-12)
Just having a look at that link, there are two potential updates that could've affected the behaviour:

1. Make LDAP attributes used by the addressbook customizable via preferences (bug 119291). r=bienvenu@nventure.com, sr+a=shaver@mozilla.org

2. fix for 293199: do not automatically set the secure mode (to use port 636) if the user requested the startTLS operation - in this case, we want to use the regular (389) port and negotiate SSL on that connection

3. fix for 291993: find the NSPR libraries in the correct location in the dist build tree and make prldap have a run time dependency on them

Though I'd be more inclined to think it may be related to the second and third change looking at the description.
What about:

2005-05-09 23:09	darin%meer.net 	mozilla/netwerk/protocol/http/src/nsHttpTransaction.cpp 	1.92 	4/1  	fixes bug 248827 "Support HTTP/1.1 408 response code [was: 408 request timeout on a used, persistent, keep-alive connection is mistakenly used as the response on a subsequent request]" r=biesi sr=bz a=shaver
2005-05-09 23:09	darin%meer.net 	mozilla/netwerk/protocol/http/src/nsHttpConnection.cpp 	1.58 	11/0

Sounds to me like this is the commit that might have caused this regression.
But IANAP...
Apologies, this is a me too. Has there been any progress on this issue? I'm trying to access StackOverflow from behind a corporate proxy and this problem is making it extremely onerous/unuseable. Page (e.g. http://stackoverflow.com/questions/9915885/workflow-pick-activity-not-sending-reply-until-action-complete/9917629#comment12657746_9917629) is loading and then 407's on multiple GET sockets.ny.stackexchange.com. Is there a work around that can be applied until it is resolved? Currently using Firefox 11.0 on release update channel.

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Cache-Control: no-cache
Host: sockets.ny.stackexchange.com
Origin: http://stackoverflow.com
Pragma: no-cache
Proxy-Connection: keep-alive
Sec-WebSocket-Key: hWpTzq7Rou1t6Wq/PZKYXg==
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0

Cache-Control: no-cache
Connection: close
Content-Length: 2196
Content-Type: text/html
Expires: 0
Proxy-Authenticate: Negotiate
NTLM
Basic realm="WebMarshal Proxy Server"
Proxy-Connection: close
Server: WebMarshal Proxy
Via: 1.1 PWPXY04
X-WebMarshal-RequestID: 229BB050-9ABF-400A-82C6-EE083EC37435
We need to get some programmers/devs looking at this one. I have tried to get it onto the enterprise stream to be looked at, but I haven't had much luck. IE and Chrome both see the particular error, but ignore it and don't pass it onto the users.

This is a very old bug, and from looking at some of the comments here appears to be a show stopper for wider implementation in a lot of organisations.
Hi, I'm new to buzilla/mozilla but am a dedicated tester who is encountering this and would love to help fix it, however IANAP.  I can help with logs and testing.

I am using http://www.innovation.ch/personal/ronald/ntlm.html as my reference for how authentication should work.

I can very clearly reproduce and confirm the following (and can provide wireshark logs):
1) FF starts a TCP/IP connection
2) FF sends a GET request with no authorization
3) Proxy sends a 407 message requesting authentication
4) FF sends a GET request with a type 1 message
5) Proxy sends a 407 message with type 2
TCP/IP connection is closed (by FF?)
6) FF starts new TCP/IP connection
7) FF sends a GET request with type 3 message  <-- this is the error
I think (not tested)
8) Proxy responds that authentication failed
9) FF knows that authentication failed so prompts user for credentials
* This can be with web server and 401 messages rather than proxy with 407, the result is the same.

I see two basic problems:
A) If the TCP/IP connection is closed and a new one opened the authentication must start again with a type 1 message.  FF is sending a type 3 on a new connection which is clearly against the spec.

B) There is no reason I know of that the TCP/IP connection should have been closed.  In a quick comparable test with IE9 it never closed a connection mid-authentication.  This will affect how frequent the bug is seen.

This problem is random, it can happen on any GET request that requires authentication where the TCP/IP connection is broken midway.  It is easy for me to reproduce by forcing authentication on every request and loading a page which will cause many GET requests (eg with many images).
Hi

Found reason in my case:
I am using Squid 3.1 with NEGOTIATE + NTLM authentication via AD/Kerberos/Samba.

I've changed order of authentication schemes and turned on keepalive for auth - NEGOTIATE first, then NTLM. No problems anymore.
The issue though is that this is not a problem on any other browser - IE, Chrome, Opera or Safari, and only on Firefox.
For me Firefox 7.0.1 is the last working version without this issue.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

For me the issue started in version 8.0 and is still in the current version.
It is a major blocker in our corporate environment and is preventing the roll out of any newer version.
Please provide a solution or fix for this issue.
This happens to me as well, but as https://bugzilla.mozilla.org/show_bug.cgi?id=318253#c94 stated only on Stackexchange pages. Firefox was rolled out already, and I can't use any older version.
I have Debian 6 with squid 3.1.20 with samba and auth NTLM for  4000 users. Since  long time ago (Firefox 8) that I have this problem only with Firefox Browsers.

Any ideas for this issue?
Same problem here with Squid 3.1.20 + Ntlm Authentication. I just installed Firefox14b7 and the issue seems to be fixed (no more pop-up showing up)
You should try that version.
(In reply to jp.lelievre from comment #102)
> Same problem here with Squid 3.1.20 + Ntlm Authentication. I just installed
> Firefox14b7 and the issue seems to be fixed (no more pop-up showing up)
> You should try that version.

have tried 14b8 (latest beta) but without success. same asking dialog popup again, again and again.
The same problem with Firefox 13.0.1 on Ubuntu.
With the same credentials when using wget - it works, with Firefox/Chrome - doesn't.
I've tried 'Use system proxy settings', 'Manual' and 'Automatic proxy configuration' - the same problem.

wget works, because it using Basic Authorization

Here is wget way:
User-Agent: Wget/1.13.4 (linux-gnu)
Accept: */*
Connection: Close
Proxy-Connection: Keep-Alive
Proxy-Authorization: Basic dDQ2NTE4NTphZGxqc2tpMA==

HTTP/1.1 302 Found

Here is Firefox way:

Firefox request the data:
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Proxy-Connection: keep-alive

Server responses:
HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NTLM
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Via: 1.1 xxx
Cache-Control: no-cache, proxy-revalidate
Content-Length: 2297
Connection: close

Firefox sending the authorization:
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Proxy-Connection: keep-alive
Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=

Server returns:
HTTP/1.1 407 Proxy Authentication Required
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Via: 1.1 xxx
Cache-Control: no-cache, proxy-revalidate
Content-Length: 2314
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Date: Tue, 17 Jul 2012 09:27:05 GMT

and again:
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Proxy-Connection: keep-alive

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NTLM
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Via: 1.1 xxx
Cache-Control: no-cache, proxy-revalidate
Content-Length: 2309
Connection: close
Date: Tue, 17 Jul 2012 09:27:06 GMT
I've tried to force somehow Firefox to use Basic Authentication, but I couldn't.
Changing following settings in about:config didn't help either:
network.automatic-ntlm-auth.allow-proxies => false
network.negotiate-auth.allow-proxies => false
So I had to use a different proxy.
Going to look at this, at last! :)
Assignee: nobody → honzab.moz
Anyone here on this bug - I want to ask for your help.  If you can, please provide a log like this:

1. set/export NSPR_LOG_MODULES=timestamp,negotiateauth:5,nsHttp:5
2. set/export NSPR_LOG_FILE=c:\path\to\a\log-file
3. start firefox and reproduce the problem, the simpler way to do it the better, smaller logs are easier to read
4. close firefox

Other info that would be useful are any changes in related preferences (about:config, filter for "-auth").

I understand there are already logs attached, but they are very old and also don't contain negotiateauth module output that is important to have.

I'm asking for help here since it is not simple for me to reproduce locally with my testing WS2003+ISA2006.  Actually, any help on configuring ISA for NTLM or Squid+ntlm_auth might be useful here as well so I can reproduce and fix/test locally (feel free to write directly to my bugzilla mail).

Thanks every one.  I really want to push this very old bug finally forward.
Attached file NTLM log
I've attached the log. Tried two times, 3rd time I've cancelled.
Honza,

Thanks for looking at this issue.

On firefox 11 on Windows XP, I need to manually add an exception for evey site I visit. However, once the exception has been added, the pop-up does not appear again.

Of course, some pages have ressources on multiple sites. Hence I may need to add several exceptions for a single page.

Also, I believe the ISA proxy may generate a new random certificate, and I suppose you I to do it all over again when this occurs.

Finally, the overall user experience is not very good, because loading CSS or javascript does not raise the popup (such as https://addons.mozilla.org/en/firefox/ loads js from https://static-ssl-cdn.addons.mozilla.net/). Hence I must inspect the page, in order to find out which complementary servers I need exceptions for.

Also, I have set the environment variables you asked for, but I am not generating any logs. Do I have to install a plugin?
(In reply to bronek from comment #109)
> I've attached the log. Tried two times, 3rd time I've cancelled.

Thanks!  This is a perfect example of a well made log.

I cannot reproduce with ISA server 2006 configured to accept only NTLM as per [1] and Firefox on win7 and linux (with ntlm_auth) boxes, none of them part of AD on the ISA box, though the logs (your and mine) are pretty the same, except where expected I'm getting 200 CONNECTED but you are getting 407.

Can I ask you for a PCAP?  Maybe it can give more clues.  Feel free to send it to my bugzilla email directly.

(In reply to Régis Décamps from comment #110)
> Honza,
>
> Thanks for looking at this issue.
>
> On firefox 11 on Windows XP, I need to manually add an exception for evey
> site I visit.

If you mean exception for an invalid certificate, then that is a very different bug then this one (we [NSS] don't share cert store with Windows).  There is a bug for that, but I don't have a number by hand now.
Attached file NSPR LOG
The requested NSPR Log File
additional to https://bugzilla.mozilla.org/show_bug.cgi?id=318253#c112

I added a Log as well, so you have more than one. It took a while as I had to ask for permission first. I removed internal information, all of the internal addresses, hopefully. I cannot give you a PCAP, Also I cannot give you the Proxy Server configuration. However, if you want specific information from proxy auto configuration file or of my client, I might be able to give you these.

About the info you requested in the initial comment:
all -auth config settings are on standard
The version I created this log on is Firefox 14.0.1
Andreas, can you please describe exactly what is your problem?  Your log is different from the log from bronek.  bronek has problem to authenticate to his proxy (seems to be ISA server).  You, on the other hand, are able to connect (authenticate) to your proxy (McAfee gateway).  But then you are trying to connect hosts like sockets.ny.stackexchange.com that require 401 authentication.  I also see a lot of requests for prompt from user.  Since there is a lot of requests (the log is very long and complicated) and I currently have no tools to track simply down what belongs to what, I cannot say what the problem is.

1. start firefox
2. connect to just a single site that usually requests a repeated prompt from you
3. close firefix
4. send the log again

Also, I presume the credentials you are using to auth are credentials of your windows account you are currently logged in, right?  You are just putting your windows account credentials to the prompt, is that so?  If yes, then just add some of the sites you are connecting to to network.automatic-ntlm-auth.trusted-uris pref (in about:config) in form like:

http://sockets.ny.stackexchange.com, http://some.other.site

This should prevent prompts for logging in to those sites and automatically send the default credentials to them.

Thanks.
I did just what you described. All requests in this log are from this page only. I open superuser.com, part of the stackexchange network. This in turn pops up 10 popup windows asking for user credentials.

These are the two popup messages I get:
Authentication necessary

Authentication necessary
http://sockets.stackexchange.com requires a username and a password. Output of the website: "McAfee Web Gateway"

About the automated sending of ntlm auth: where does it send the authentication to? to the proxy only? The user credentials I use locally have nothing to do with the user credentials on the site. The validity of my local user credentials end at the proxy and have should not go beyond that.

on the site itself I connect to, the only thing on the page I can see that has the sockets-string in it is on line 28,

<script type="text/javascript">
StackExchange.realtime.init('ws://sockets.ny.stackexchange.com');
});
</script>
Andreas:
That site uses HTML5 WebSockets, which have mixed support.  IE does not support WebSockets and not all proxies do.  In the logs you posted search for the word "upgrade" which is what I think is failing.
Please try this website (http://www.websocket.org/echo.html) and use Connect (without TLS).  If you have repeated NTLM pop-ups on that site then your problem has nothing to do with Bug 318253.

Honza:
I thought I had a repeatable way of reproducing this but I don't seem to be able to after spending some time on it (including attempting on FF8 which I know had the problem).  I will continue to look into it, but see my comment 96 for details of what I was seeing and what I think the issue is.
FWIW the websockets auth popup issue is also reported in bug 766817, so if it's a separate bug we can followup over there.  If not, make sure to DUPE it when this is fixed.
Blocks: 766817
This bug is still on my radar on top position.  I will return to it in two months, tho.

Anyone else feel free to analyze the logs and continue here in the meantime.  I believe there are more bugs here then just a single one.
Assignee: honzab.moz → nobody
Blocks: 779467
This issue also seems to be the same one which I reported in bug 779467.  Additionally, the most recent upgrade to Firefox (15.0 Funnelcake Jul 2012 mozilla13 - 1.0) now has this bug, where the earlier version did not.
Earllier version most certainly also have this bug. I'm seeing it dailing using FF 10.0.6esr. It's hard to remember which was the last version I didn't have this issue with. Might be FF 8 or 9.
Sorry.  By earlier version, read: the version just before this latest one didn't have the problem.

From what I've read, it appears that this bug has been coming and going since FF 4.  V3.6 didn't have the bug, as far as I can tell.  Don't know when Firefox lost the bug again and I'm unable to perform regression testing here at work to determine just when it was reintroduced because, ironically, the regression software appears not to be able to handle a proxy server. :-(
HI FF 14 resolved problem for my! But update to 15 Release had returned it :(
Please test FF14 in your environment  and post here!
(In reply to newerdog from comment #122)
> HI FF 14 resolved problem for my! But update to 15 Release had returned it :(

Id tested with Forefront TMG.
FF14  work fine also with SQUID 3.2.1
(In reply to newerdog from comment #125)
> FF14  work fine also with SQUID 3.2.1

Do you know what kind of authentication you use? NTLM or Basic?
see also bug 650091
Using Firefox 15.0.1 (Release Channel)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1

On a corporate network which uses AD to validate for network access. Page loads fine, however doing a "hard refresh" causes credentials prompt for every item on the page.
To Kim Sullivan  NTLM of course!
To   Shane Troughton   But 14 FF forks fine ! try it pls
Yes it does work in FF 14! :)
Anyone know if this has been fixed in 15.0.2?
Sorry for the long response, I was fighting with other bugs which were preventing me to upload anything. In attachment 666505 [details] you'll see the tcpdump when Firefox dialog pops up over and over.
I cannot post my log for privacy reasons, but I'm also following bug 779467 and I think that it is a duplicate of this one.

Here are the messages I have in the error console - I will also attach them as text file.

signon.debug

to true: in the console, when the password is requested, I have these messages (while none of them do appear with the same preference set in 15.0.1 version):

Pwmgr Prompter: Prompt bound to an existing one in the queue, callback = [xpconnect wrapped nsIAuthPromptCallback]

Pwmgr Prompter: getAuthTarget is for proxy auth

Pwmgr Prompter: ===== asyncPromptAuth called =====

Pwmgr Prompter: ===== initialized =====

Pwmgr Prompter: Prompt bound to an existing one in the queue, callback = [xpconnect wrapped nsIAuthPromptCallback]

Pwmgr Prompter: getAuthTarget is for proxy auth

Pwmgr Prompter: ===== asyncPromptAuth called =====

Pwmgr Prompter: ===== initialized =====

Pwmgr Prompter: found 0 matching logins.

Pwmgr Prompter: getAuthTarget is for proxy auth

Pwmgr Prompter: ===== promptAuth called =====

Pwmgr PrompFactory: _doAsyncPrompt:run - performing the prompt for '1|moz-proxy://[my proxy address]:8080|moz-proxy://[my proxy address]:8080'

Pwmgr PrompFactory: _doAsyncPrompt:run dispatched

Pwmgr Prompter: getAuthTarget is for proxy auth

Pwmgr Prompter: Adding new prompt to the queue, callback = [xpconnect wrapped nsIAuthPromptCallback]

Pwmgr Prompter: getAuthTarget is for proxy auth

Pwmgr Prompter: ===== asyncPromptAuth called =====

Pwmgr Prompter: ===== initialized =====

Login Manager: onStateChange accepted: req = http://forum.mozillaitalia.org/, flags = 0x30004

My question is: has something been changed in the authentication system from version 16 up? There wasn't any problem until version 16. This issue also occurs with Aurora and Nightly.
Attached file See comment 133
Attachment #666881 - Attachment description: See https://bugzilla.mozilla.org/show_bug.cgi?id=779467#c16 → See comment 133
(In reply to Simone Lando from comment #133)
> I cannot post my log for privacy reasons, but I'm also following bug 779467
> and I think that it is a duplicate of this one.
Yes, I'm almost 100% sure that it is.  I created it because this particular thread was started in 2005 and I thought it was dead.  My thread should probably be added to the list of duplicates.
FYI not sure if this is a dup of https://bugzilla.mozilla.org/show_bug.cgi?id=766817

But I have just posted a test case there. The test produces a authentication popup which doesn't happen in Chrome(and shouldn't happen as the proxy does not do authentication). It also shows that proxy'ing websockets is broken even when the proxy server supports it. Tests with FF 15.0.1
this(In reply to Jarrod Lee Petz from comment #136)
> FYI not sure if this is a dup of
> https://bugzilla.mozilla.org/show_bug.cgi?id=766817
>

no. this bug predates websockets.
Issue still persists in latest Aurora 18.0a2 (2012-10-21).

For those who require hard refreshes of a website, but can't due to this annoying bug, here's a workaround:

Clear the cache by the press of a button and then hit F5.

I hope to see this fixed soon, as i tend to accidentally hit CRTL+F5 (or shift + mouse refresh) way too often ^^
on ctrl f5 we clear all persistent connections. That's probably related to the root cause.
This is just a shot in the dark, but can a couple of the folks who are experiencing this issue with ctrl-f5 try a windows build from:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-6b110cd9090b/try-win32/

that includes this patch:
https://hg.mozilla.org/try/rev/a2633d065b25

again - this is basically worksforme, so its just an idea worth testing. The patch changes how we handle clearing the persistent connection pool on a forced reload (i..e ctrl f5).

report back after you try it.. and thanks for the help :)
(In reply to Patrick McManus [:mcmanus] from comment #140)
> This is just a shot in the dark, but can a couple of the folks who are
> experiencing this issue with ctrl-f5 try a windows build from:
>
>
please disregard the url in comment 140. That build contains an additional patch I should not have included in this test.

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.com-631ac7fc3b0a/try-win32/

(that build is still in progress as I write this comment. If it doesn't load, try again in an hour or so.)
In this build (FF 19) I tried clicking CTRL + F5 to many times and have not found bug with authorization frames (but sow another bugs with IBM Websphere portal:).

(In reply to Patrick McManus [:mcmanus] from comment #141)
> (In reply to Patrick McManus [:mcmanus] from comment #140)
> > This is just a shot in the dark, but can a couple of the folks who are
> > experiencing this issue with ctrl-f5 try a windows build from:
> >
> >
> please disregard the url in comment 140. That build contains an additional
> patch I should not have included in this test.
>
>
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.
> com-631ac7fc3b0a/try-win32/
>
> (that build is still in progress as I write this comment. If it doesn't
> load, try again in an hour or so.)
(In reply to Patrick McManus [:mcmanus] from comment #141)
> (In reply to Patrick McManus [:mcmanus] from comment #140)
> > This is just a shot in the dark, but can a couple of the folks who are
> > experiencing this issue with ctrl-f5 try a windows build from:
> >
> >
> please disregard the url in comment 140. That build contains an additional
> patch I should not have included in this test.
>
>
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mcmanus@ducksong.
> com-631ac7fc3b0a/try-win32/
>
> (that build is still in progress as I write this comment. If it doesn't
> load, try again in an hour or so.)

This gets rid of those annoying popups. thanks! I will test it some for other possible issues.
Patrick,
could you please provide another try-build with this patch of yours? Seems like the previous one is gone.
I'd like to test this against some of our corporate systems where a bug that could be the same like the one at hand shows up. (Without a proxy, but IIS with NTLM enabled)
(In reply to Thomas B. Rücker from comment #144)
> Patrick,
> could you please provide another try-build with this patch of yours? Seems
> like the previous one is gone.
> I'd like to test this against some of our corporate systems where a bug that
> could be the same like the one at hand shows up. (Without a proxy, but IIS
> with NTLM enabled)

The patch for the try build was checked in as bug 779467 on 11/4 - so a recent nightly will include it.

It wasn't duped to this bug because it did not seem to resolve all instances of this, but it may help you.
I can report back that on today's nightly I have the following situation regarding this bug (against an IIS with NTLM enabled, no proxy):
- win32 nightly - no authentication pop-ups
- linux x86_64 nightly - still loads of authentication pop-ups

Is the bug fix for bug 779467 active for windows builds only?
Thomas, thanks - that's good information.

779467 is for all platforms, but linux and windows use totally different ntlm code so that's good information about this bug.
Just a side-note from the poster of duplicate bug 779467: I didn't want to complain because the problem is so much better now but I do still get the occasional authentication pop-up at start-up.  This may be a problem with some left-over setting in my about:config page that I made when I was searching for a solution to this but the intermittent nature of the problem makes me doubt this.  I'll let you know if I find anything but it's going to be a case of change one thing and wait a week or until the pop-up returns sort-of-thing.
HI, I recon that problem doesnt resovled yet. Aut popup appear  when i trying open this page http://www-01.ibm.com/support/docview.wss?uid=swg21200013
Newerdog,

I went to that website and got the same problem.  I guess it's not just me and this problem still needs a little work.
I can confirm that the site in comment #149 gives a proxy auth popup using FF 17.0.5 ESR on Windows, so bug 779467 did not resolve all instances of this problem.
Same here, I do get the pop-up on the site mentioned in comment 149. More annoying are the random authentication pop-ups my users get all over the web. We use squid proxy with NTLM authentication against windows AD domain. 99% of the time firefox silently authenticates with the proxy as it should. But it randomly fails and hits users with the dreaded pop-up. I can't seem to find a pattern to the failures. I would be willing to provide any logs etc that would be helpful.
Same here;

- Firefox 24;
- Windows 8 64 bits;
- Squid 3.1
- Kerberos and Basic Auth.

Wireshark shows me:
1- Firefox sends a GET with Kerberos auth;
2- Squid server reply with "HTTP/1.0 407 Proxy Authentication Required (text/html)";
3- Firefox sends a new request, but with basic auth;
4- Squid server reply with "HTTP/1.0 407 Proxy Authentication Required (text/html)" again;
5- Firefox sends a new request, but with Kerberos auth again;
6- Squid server reply with "HTTP/1.0 407 Proxy Authentication Required (text/html)" again;
7- Firefox sends a new request, but with basic auth, again;
8- Squid server say OK, and the odyssey ands.
Same for me as well:
- Firefox 25
- Client: Windows 2003 x64
- Proxy: squid-3.1.10-19.el6_4.x86_64
- Authentication: NTLM (using ntlm_auth from samba-winbind-clients-3.6.9-151.el6_4.1.x86_64 on proxy server)

On Client I am using local (non-domain) account which forces Firefox to ask for another credentials. And firefox asked for it many times within single webpage request (3-4 times). After opening sub-site of webpage I've been asked again for credentials.

With IE, only single credentials request is sufficient.
For me this issue is solved.

My environment:
- squid proxy with ntlm enabled authentication (with samba 3 controller) - i.e. pure NTLM authentication without Negotiate of kerberos ticket or so on
- computers which are not members of corporate domain (i.e. users have to enter its own credentials into browser)
- behavior exactly like described in my previous post (multiple login requests)

In deeper look communication capture I saw following behavior:
- firstly browser tried to log-in using local windows user credentials - this failed which is OK
- but after that the proxy requested three-times (!) the credentials before it allows the request itself
and this was caused by enabled proxy ntlm keepalive mechanism (auth_param ntlm keep_alive on) - it looks like first three authentication requests (one using local account and two requested from user by browser) were handled using single authentication thread (like triple-attempts authentication).

After disabling ntlm keepalive in squid proxy (== auth_param ntlm keep_alive off), there is FINALLY only one authentication request from proxy:
- firstly browser tried to log-in using local windows user credentials - this failed which is OK
- after that the proxy requested one-time for the credentials before it allows the request itself

this results that Firefox is asking only one time for user credentials, and all further requests from that users are seamlessly accepted without any next authentication challenge. All the time I was blaming that Firefox is responsible for that annoying login pop-ups due wrong implementation of NTLM authentication. And I was wrong. Sorry for that.

this long story is finally resolved for me.

hopefully this helps to someone else
My question though to Michal Bruncko's solution is why does this only occur with Firefox? Internet Explorer, Chrome and Safari all handle the proxy correctly without any changes needing to be done.

To me this indicates either a bug with how Firefox handles proxies or the ntlm proxy handover not properly implemented in FF.

As an aside, if you use a third party proxy plugin (such as FoxyProxy) seem to handle it. According to our network team who look after our proxy, the issue is basically that FF can't handle 407 Authentication requests and bounces back an authentication box when it receives this message.
> My question though to Michal Bruncko's solution is why does this only occur with Firefox? Internet Explorer, Chrome and Safari all handle the proxy correctly without any changes needing to be done.

not really in my case. I have realized during the testing:
- that when comparing both communications of Firefox and IE for NTLM handling, both behaviors looks completely same (same packet types, same challenges from proxy side and same responding replies from browsers)
- also IE in my case requested credentials from user exactly three times (I am not sure why I have not pointed to this before), but the difference from user point of view is that:
--> all next two credential requests from IE are already filled by IE according the first login request (even if I have not checked "save credentials" on first request). It seems that IE is automatically storing user credentials during existing session.
--> there are no more next credential request from IE after that at all. In comparison to Firefox where I was asked several times later - i.e. when I requested new page or so on (but of course I can check this behavior from communication capture side to confirm that both looks same as well).

so in summary I think the main difference between browsers (in my case) is "only" in behavior in interaction with end-user: once the IE determines that credentials filled by user are acceptable by proxy, it decides to use it all the time during the session without any further ask for them (except the second and third request, but they were also filled with credentials of first request).
In comparison if I not check "save credentials" in Firefox, the Firefox asked me for them without pre-filled one's from previous ask. Also after beginning three-times credentials request, I have been asked for them intermittently during the whole session.
Is there any difference if you start with a clean cache?

I haven't looked at the traffic traces, but it is known that if the first visited page is already partially in the browser cache then you get one auth request per persistent connection in virtually all browsers.

The initial password request(s) is very likely a different issue from intermittent requests during the session. Please separate the two issues.
> Is there any difference if you start with a clean cache?
For testing purposes I have changed starting page to blank to avoid any undesirable results. All the time I used different testing page. Also I thought that this could be some kind of threading issue (when Firefox is requesting web-objects simultaneous way) so I decided to request simple object only - url's pointing to image instead of whole web pages with lot of various object that needs to be requested.
And also as I said, I have looked on captures what happen on background and what was really requested by browsers.
Ok, the last unsure thing about Firefox is answered:
"Also after beginning three-times credentials request, I have been asked for them intermittently during the whole session."

I moved back to old proxy configuration (== auth_param ntlm keep_alive on) and make new test with IE an Firefox.

1. opening one URL with picture - both browsers asked me three times for credentials at the beginning. and after that I got requested picture
2. after requesting two-more URL's with pictures, I got them in both browsers without credentials request (in capture there is no "407 proxy required" from proxy)
3. then I decided to open web page (simple web page containing of two additional objects). after requesting the second additional object, the "http 407" comes from proxy side. and here comes the difference:
--> IE is using last used credentials provided by end-user and it is successful without any ask from user.
--> Firefox is again trying to use credentials of local windows user firstly(!) and not the credentials already provided by user. and of course this authentication fails and Firefox is requesting another credentials again from user via another login box. After filling correct credentials I got requested response.

In summary it looks like the Firefox is trying to use local windows user credentials instead of using credentials already provided by user, when it got next "http 407" from proxy. But the interesting thing is that this annoying behavior is here only if "auth_param ntlm keep_alive on" is set on proxy side. It seems like the Firefox is not trusting the entered credentials if they were not accepted at the beginning (where I have to type it three times) - but maybe it is because of something else.

If someone is interesting in both (IE and FF) captures, please ask me for them via email.
One last thing:

what are the difference in Firefox authentication behavior in both "auth_param ntlm keep_alive" cases:
- when the keep-alive is set to "on", the proxy sends HTTP header "Proxy-Connection: keep-alive" within each _first_ "http 407" response.
- when the keep-alive is set to "off", the proxy sends HTTP header "Proxy-Connection: close" within each _first_ "http 407" response.
- but within proxy "http 407" sub-response including "NTLMSSP_CHALLENGE" the "Proxy-Connection" header is always set to "Keep-Alive" - this makes sense as the NTLM authentication itself is requires this.

Summary for Firefox:
- the Firefox is always trying to use local windows user credentials firstly nevertheless of the value of "Proxy-Connection" header in 407 response. It is doing this all the time, no matter if you're starting session or simply browsing web-pages in existing session
- if the "Proxy-Connection" is set to "Keep-Alive" in 407 responses, Firefox explicitly ask user for credentials no matter if you already provided credentials before or not.
- if the "Proxy-Connection" is set to "Close" in 407 responses, Firefox is trying to use cached credentials (after the first try with local windows user) - if they were already explicitly provided by user

Summary for Internet Explorer (tested with v8.0 in Windows 2003):
- the Internet Explorer is trying to use local windows user credentials only at the beginning of the session, no more tries later (IE simply decides that it makes no sense to use it again if authentication failed with them at beginning)
- if any further 407 response comes from proxy during existing session, IE always use cached credentials provided by user
- Internet Explorer authentication behavior is same (as described in previous two points) nevertheless the "Proxy-Connection" header is set to "Keep-Alive" or "Close".
And that's the success of IE in such scenarios :)
Thanks Michal for your comments and logs. Hopefully we can get some action on this bug. It's been preventing wide spread adoption of FF within my organization for several years now, and no doubt many other enterprise organizations as it was first logged in 2005 (now 9 years ago).
(In reply to joopbraak from comment #92)
>
> 2005-05-09 23:09	darin%meer.net
> mozilla/netwerk/protocol/http/src/nsHttpTransaction.cpp 	1.92 	4/1  	fixes
> bug 248827 "Support HTTP/1.1 408 response code [was: 408 request timeout on
> a used, persistent, keep-alive connection is mistakenly used as the response
> on a subsequent request]" r=biesi sr=bz a=shaver
> 2005-05-09 23:09	darin%meer.net
> mozilla/netwerk/protocol/http/src/nsHttpConnection.cpp 	1.58 	11/0
>
> Sounds to me like this is the commit that might have caused this regression.

I don't know how to back out a commit or build Firefox, but maybe somebody else can do this and post the results?
Target Milestone: mozilla2.0 → ---
Depends on: 747113
It looks same my problem and others users.
Now, in Firefox 30 the solution of it not work and surf on Internet is very complicated because Firefox ask and ask user and password authentication.
In Linux I can't surf because NTLM is blocked. Many people like me is affected by it.
https://bugzilla.mozilla.org/show_bug.cgi?id=747113
With Google Chrome not happen it and like Firefox user I suggest that does [1] but now the solution is broken and nobody can surf. Or happen  NTLM Proxy authentification dialog pops up over and over.

[1] http://firefoxmania.uci.cu/firefox-me-pide-contrasena-muchas-veces-que-hago/
I'd like to add my frustration with this.  Using latest Firefox.

I've got an 800-user client who want to turn on authentication on their Squid proxy, but this "feature" causes it to be unusable with auth turned on.

I don't quite understand why we didn't see this when the client was running ISA/TMG, but do with squid, however IE and Chrome don't exhibit this behaviour.  I believe that, in general, the issue is related to sites CONNECTed to using SSL, but we have seen it (much less often) on plain HTTP as well.
Duplicate of this bug: 750621
Confirmed this bug with FF30 on Win7. Never thought I'd have to tell a customer to "Use internet explorer because Firefox won't work.."
I've been testing this on FF31.0 - the problem seems to have disappeared in 31.0 for us
Duplicate of this bug: 764656
Still have this issue with FF33.0 on Windows 7.
Version 36-37(Beta) doesn't work with NTLM until "network.auth.force-generic-ntlm-v1" is set to true
(In reply to Michal Bruncko from comment #155)
> For me this issue is solved.
>
> My environment:
> - squid proxy with ntlm enabled authentication (with samba 3 controller) -
> i.e. pure NTLM authentication without Negotiate of kerberos ticket or so on
> - computers which are not members of corporate domain (i.e. users have to
> enter its own credentials into browser)
> - behavior exactly like described in my previous post (multiple login
> requests)
>
> In deeper look communication capture I saw following behavior:
> - firstly browser tried to log-in using local windows user credentials -
> this failed which is OK
> - but after that the proxy requested three-times (!) the credentials before
> it allows the request itself
> and this was caused by enabled proxy ntlm keepalive mechanism (auth_param
> ntlm keep_alive on) - it looks like first three authentication requests (one
> using local account and two requested from user by browser) were handled
> using single authentication thread (like triple-attempts authentication).
>
> After disabling ntlm keepalive in squid proxy (== auth_param ntlm keep_alive
> off), there is FINALLY only one authentication request from proxy:
> - firstly browser tried to log-in using local windows user credentials -
> this failed which is OK
> - after that the proxy requested one-time for the credentials before it
> allows the request itself
>
> this results that Firefox is asking only one time for user credentials, and
> all further requests from that users are seamlessly accepted without any
> next authentication challenge. All the time I was blaming that Firefox is
> responsible for that annoying login pop-ups due wrong implementation of NTLM
> authentication. And I was wrong. Sorry for that.
>
> this long story is finally resolved for me.
>
> hopefully this helps to someone else

Michal, buenas tardes, solo quería agradecerle, ya que gracias a esta publicación
pude resolver un problema en una implementación de squid sobre centos, el cual me insumió demasiados días.

Cuando las credenciales iniciales con las que se presenta el usuario no son correctas y el navegador solicita
las nuevas credenciales por medio de una ventana emergente, a pesar de ingresar las correctas nunca eran validadas
correctamente.

Con solo agregar en el archivo squid.conf "auth_param ntlm keep_alive off" como Michal lo indica y restartear el
servicio, todas las validaciones se realizaron correctamente.

Muchas Gracias "Michal Bruncko" por tu aporte.
This bug is appearing in FF ESR V38.0.1. As far as I can tell there is no consistency between versions on whether it will work or not. As previously mentioned FF is the only browser displaying a 407 error behind a proxy, occasionally some versions work and others don't.
To reiterate, Internet Explorer (8,9,10,11,12), Chrome (all versions), Safari (all versions) all work with no errors. Even the new MS Spartan/Edge variant works. If users call up our service desk with proxy issues, general fix is to uninstall Firefox and get them using IE instead.

This is quite poor Mozilla, this bug has been in existence now for 10 years, first reported on 30/11/2005, with seemingly minimal traction apart from extensive reporting and logs from many users.
After 1-2 years without any problems ==>  with FF >=37 the ntlm-problem came back. network.auth.force-generic-ntlm-v1 didn´t solve the problem. Can I send some debug-information which can help to solve? Thanks !
(In reply to ulrich.kiefer from comment #175)
> After 1-2 years without any problems ==>  with FF >=37 the ntlm-problem came
> back. network.auth.force-generic-ntlm-v1 didn´t solve the problem. Can I
> send some debug-information which can help to solve? Thanks !

Is it the same problem described in the first comment - user name, password dialog is popping up all the time? Sorry the bug is old and there are tones of comments.
Which mashing are you using Linux, Windows?

Can you send me http log:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging
Hello Dragana

I've emailed you the http log file with the error and also a screenshot from firebug showing the error.
Interestingly I’ve noticed that all the 407 errors seem to be on images, basically the response headers are identical for all the 407 errors.
There seem to be two different, intertwined issues being discussed here.

The trace in attachment #227329 [details] shows a simple failure to authenticate, perhaps due to the NULL domain, or something else entirely.  The popup is because the server has refused the authentication, and that much *seems* normal and OK (we need to work out why the authentication is refused of course).

But the recent discussion here is about proxy authentication.  There are other bugs around NTLM proxy authentication, and I'm confident there is a real issue here, somewhere :-)
I have receive 2 logs. Sorry for the delay i will try to get to it today or tomorrow. There is an issue in the logs
In my case the authentication pop-up is due to an invalid NTLMSSP_AUTH reuse.

First connection to the website is Ok:
CONNECT -> 407 "Auth Required"
CONNECT + NTLMSSP_NEGOTIATE -> 407 + NTLMSSP_CHALLENGE
CONNECT + NTLMSSP_AUTH -> 200 "Connection established"

Immediately subsequent connection is not Ok since it send an NTLMSSP_AUTH header (the same as the previous connection) in the first request:
CONNECT + NTLMSSP_AUTH (with same values as in previous request) -> 407 "Auth Required" => user prompted for credentials

I will try to upload the corresponding Wireshark screenshot.
Packet capture file available on request (contains credentials).
Attached image NTLMSSP_AUTH_reuse.PNG
See comment 180 (NTLMSSP_AUTH reuse)
Depends on: 486508
(In reply to Dragana Damjanovic [:dragana] from comment #179)
> I have receive 2 logs. Sorry for the delay i will try to get to it today or
> tomorrow. There is an issue in the logs

Any progress on this?  I suspect this is either a dup or depends on bug 486508.

I'm suspecting that the NTLM state isn't reset in the right place, based on there being cached credentials.
(In reply to Honza Bambas (:mayhemer) from comment #107)
> Any help in configuring Squid+ntlm_auth might be useful here as well so I can reproduce and
> fix/test locally (feel free to write directly to my bugzilla mail).
>
> Thanks every one.  I really want to push this very old bug finally forward.

Squid+ntlm_auth is much easier to set up if you use the fixed --password option to ntlm_auth, so it need not connect to Samba/AD:

auth_param ntlm children 20 startup=0 idle=1
auth_param ntlm keep_alive on
(In reply to Andrew Bartlett from comment #183)
> Squid+ntlm_auth is much easier to set up if you use the fixed --password
> option to ntlm_auth, so it need not connect to Samba/AD:
>
> auth_param ntlm program /data/samba/samba4/prefix/bin/ntlm_auth
> auth_param ntlm children 20 startup=0 idle=1
> auth_param ntlm keep_alive on

for Squid+ntlm_auth toward Samba domain controller I am recommending you to turn off keep_alive (auth_param ntlm keep_alive off) as I mentioned in my comment #155 . Turning off keep_alive for ntlm in squid.conf helps me to solve this popping up authentication window in browser.
Of course if any other combination of environment (Windows AD instead of Samba, or Kerberos instead of NTLM, or similar) turning of keep_alive parameter couldn't help.
Having enabled keepalive parameter in squid configuration could be misleading, because in case of apache+ntlm authentication you MUST have keepalive enabled on webserver/virtualhost configuration in order to correct working of NTLM authentication. But keepalive option in squid seems having a bit different purpose like in webserver scenario. Keepalives in webserver case is needed to keep tcp session open between client and webserver to be able to exchange several NTLM messages needed to get NTLM auth working. But keepalive option in Squid is rather NTLM session-related than TCP connection-related and could be turned off in some cases (like the mine).
(In reply to Michal Bruncko from comment #184)
> (In reply to Andrew Bartlett from comment #183)
> > Squid+ntlm_auth is much easier to set up if you use the fixed --password
> > option to ntlm_auth, so it need not connect to Samba/AD:
> >
> > auth_param ntlm program /data/samba/samba4/prefix/bin/ntlm_auth
> > auth_param ntlm children 20 startup=0 idle=1
> > auth_param ntlm keep_alive on
>
> for Squid+ntlm_auth toward Samba domain controller I am recommending you to
> turn off keep_alive (auth_param ntlm keep_alive off) as I mentioned in my
> comment #155 . Turning off keep_alive for ntlm in squid.conf helps me to
> solve this popping up authentication window in browser.

My hope is that when we fix this, that will no longer be required, and NTLM keepalive will be possible, as that will reduce the load on the DC.

(In reply to mikael.andres from comment #181)

I think your issue is probably what I have a proof of concept patch in bug 486508 for.  It should allow forward progress after a failure.
Assignee: nobody → abartlet
Status: NEW → ASSIGNED
(In reply to Andrew Bartlett from comment #182)
> (In reply to Dragana Damjanovic [:dragana] from comment #179)
> > I have receive 2 logs. Sorry for the delay i will try to get to it today or
> > tomorrow. There is an issue in the logs
>
> Any progress on this?  I suspect this is either a dup or depends on bug
> 486508.
>
> I'm suspecting that the NTLM state isn't reset in the right place, based on
> there being cached credentials.

Both logs has a failed ssl handshake (do not have much ssl logging in the logs, but at some point, before the handshake is finished, the connection is reset)
Assignee: abartlet → nobody
Status: ASSIGNED → NEW
Would it help to get logs for the same sites from Chrome to compare the responses between the browsers?
I have try with this options:
auth_param ntlm keep_alive off
auth_param negotiate keep_alive off

but, not working for me :-/
(In reply to Jimi_X from comment #187)
> Would it help to get logs for the same sites from Chrome to compare the
> responses between the browsers?

As Andrew suggested this is probably duplicate of bug 486508. I will mark it as dup.

There will be a solution in bug 486508, and once it is fix it would be good if you can test it.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 486508
Will do
Duplicate of this bug: 319336
Duplicate of this bug: 757406
After update to FF 48, started popup mesage for NTLM authorization on proxy.
it's not proxy related.
I think this appeared after Windows Anniversary update. I know that I haven't changed "network.automatic-ntlm-auth.trusted-uris", but now I started to get login dialog for those URLS.
upgraded to 48.0.1 jus a moment ago - same result.
It has something to so with Version 48. The issue started with this Version. I've no Win Anniversary on my machine. I tried Version 46 and Version 47: it works like expected. As soon as I installed V48 the issue is back...
I'm using Kerberos Auth in my web app and this app now works only with FF < 48 :(
We're seeing similar behavior only after updating to Firefox 48.x.
You need to log in before you can comment on or make changes to this bug.