1548804 - proxy authentication dialogue box repeatedly pops up asking to authenticate after upgrading to 67.0 b10+

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P1)

Description

[Arthur K. (he/him)]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0 SeaMonkey/2.57

Steps to reproduce:

I updated from 66.0.3 x64 to 67.0 b14, b15 & b16.

Actual results:

I've been running 66.0.3 x64 with manual proxy config set up. I have to enter username and password to authenticate through the proxy provider (torguardvpnaccess.com) @ port 6060. "Use this proxy server for all protocols" is selected. "Proxy DNS when using SOCKS v5" is checked.

After first authentication, the authentication dialogue box pops up repeatedly as if not having accepted the initial authentication. It will do this endlessly no matter how many time username and password are entered.

Rolling back to 66.0.3 fixes it.

Expected results:

After entering username and password to authenticate once, it should not ask ever again.

Comment 1

[Arthur K. (he/him)]

By the way, I had Error Console open and it didn't show anything unusual or any errors.

Comment 2

[[:philipp]]

hi, would it be possible for you to narrow down the regression range with the tool from https://mozilla.github.io/mozregression/?

Comment 3

[Arthur K. (he/him)]

(In reply to [:philipp] from comment #2)

hi, would it be possible for you to narrow down the regression range with the tool from https://mozilla.github.io/mozregression/?

Well, without using Mozgression, I tested from 67.0b3 > b10. Last good beta was b9. b10 is when it broke. I know it's not Mozgression granular but it's a start.

Comment 4

[[:philipp]]

if it first regressed in 67.0b10, perhaps it has to do with the fix for bug 1538737?
(https://mzl.la/2VH7EEq would be a list of all the changes going into beta 10)

Comment 5

[Arthur K. (he/him)]

(In reply to [:philipp] from comment #4)

if it first regressed in 67.0b10, perhaps it has to do with the fix for bug 1538737?
(https://mzl.la/2VH7EEq would be a list of all the changes going into beta 10)

I installed Mozgression locally but I'm trying to figure out what to set in the Bisection Wizard "Bsic Configuration" window. What should I set to test between build 76.0b9 (build 20190408123043) and 67.0b10 (build 20190411084603)? It's been a bit since I've used it but when I ran using: Firefox, 64, shippable, repo=mozilla-beta, it's falling over. A little help?

Comment 6

[[:philipp]]

you can leave the "basic configuration" window in mozregression on its defaults (Firefox, 64, shippable, repo=blank), then in the second screen point it towards your existing profile and at "build selection" set last known good build to 66 (release) and first known bad build to 2019-04-10 (date), tool should then start narrowing it down from there on until you hopefully end up at a single changeset...

Comment 7

[Arthur K. (he/him)]

(In reply to [:philipp] from comment #6)

you can leave the "basic configuration" window in mozregression on its defaults (Firefox, 64, shippable, repo=blank), then in the second screen point it towards your existing profile and at "build selection" set last known good build to 66 (release) and first known bad build to 2019-04-10 (date), tool should then start narrowing it down from there on until you hopefully end up at a single changeset...

Thanks for the help. Ok, if I did it right per your instructions using 66.0's build ID (20190314174725) as first good and 67.0b10's build ID (20190411084603) as first bad, this is what I get.

Last good changeset: e31357c7759379d2279b6883cb09c91997bfaa5d
First bad changeset: 50d64901b71fe8e6ed7d9730467bf4b4d6060f01

Hope it helps.

Comment 8

[Arthur K. (he/him)]

Pushlog from the first bad build: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=bd1e28b0143bdcff0798b0e6a4f54791c41192e8&tochange=50d64901b71fe8e6ed7d9730467bf4b4d6060f01

Comment 9

[Honza Bambas (:mayhemer)]

Seeing:
https://bugzilla.mozilla.org/show_bug.cgi?id=1538737
https://bugzilla.mozilla.org/show_bug.cgi?id=1538456 (not likely to be the regressing bug)

Comment 10

[Pascal Chevrel:pascalc]

Marking as new as we have a regression range. Honza, Selena, could you evaluate the criticality of this bug with regards to our upcoming 67 release and prioritize accordingly? Thanks

Comment 11

[Neha Kochar [:neha]]

Adding ni for Nhi for comment 10.

Comment 12

[Nhi Nguyen (:nhi)]

We should try to fix this for 67.

Honza, could you take a look?

Comment 13

[Valentin Gosu [:valentin] (he/him)]

Here Connection() seems to be null:
https://hg.mozilla.org/mozilla-central/annotate/997e98493dbcd55c95b4a6eb4a44f563cd6174e9/netwerk/protocol/http/TunnelUtils.cpp#l228

Comment 14

[Honza Bambas (:mayhemer)]

Arthur, I will ask you for capturing a log, it will likely quickly reveal the problem: https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging#Start_logging_using_command_line_arguments

It might contain some private data, so feel free to send to my bugzilla email (zipped or xzipped) rather than exposing here. Please refer the bug# in subject. Thanks.

Comment 15

[Arthur K. (he/him)]

(In reply to Honza Bambas (:mayhemer) from comment #14)

Arthur, I will ask you for capturing a log, it will likely quickly reveal the problem: https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging#Start_logging_using_command_line_arguments

It might contain some private data, so feel free to send to my bugzilla email (zipped or xzipped) rather than exposing here. Please refer the bug# in subject. Thanks.

Logs sent to you privately. Hope they help.

Comment 16

[Honza Bambas (:mayhemer)]

Attachment: Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

Comment 17

[Honza Bambas (:mayhemer)]

(In reply to Honza Bambas (:mayhemer) from comment #16)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=30be989f08242894a89cd598e63c5a093a2f1d3b

Comment 18

[Honza Bambas (:mayhemer)]

Arthur, I have a test build for you, here is the installer for Windows x64. It's based on Nightly, so it will create a new profile, separate from your current one. You will have to setup the proxy again.

Please let me know if this fixes the problem. Thanks!

Comment 19

[Arthur K. (he/him)]

(In reply to Honza Bambas (:mayhemer) from comment #18)

Arthur, I have a test build for you, here is the installer for Windows x64. It's based on Nightly, so it will create a new profile, separate from your current one. You will have to setup the proxy again.

Please let me know if this fixes the problem. Thanks!

I wasn't at home where I use my VPN but with this Nightly you built, I set it up fresh on my work PC, used my VPN credentials, and it works ok. No repeated nagging to re-auth credentials when visiting cnn.com. WhatIsMyIP shows me as coming from Mexico. Do you want me to test at home too or is this good enough?

I am cautiously stating this WFM.

Comment 20

[Honza Bambas (:mayhemer)]

If you are willing to, do any testing you believe may confirm this does fix what it has to and doesn't break anything else.

Thanks a lot!

Comment 21

[Arthur K. (he/him)]

(In reply to Honza Bambas (:mayhemer) from comment #20)

If you are willing to, do any testing you believe may confirm this does fix what it has to and doesn't break anything else.

Thanks a lot!

Won't be home until after 4PM CST so it might be a while.

Comment 22

[Arthur K. (he/him)]

(In reply to Honza Bambas (:mayhemer) from comment #20)

If you are willing to, do any testing you believe may confirm this does fix what it has to and doesn't break anything else.

Thanks a lot!

Confirming on my home PC that with your test build, no issues with proxy. All good.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3502dda7734c
Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

Comment 24

[Honza Bambas (:mayhemer)]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

Beta/Release Uplift Approval Request

• User impact if declined: We don't remember proxy credentials properly, so on every new connection having different origin attributes (which is nearly all the time) we open the credentials dialog on and on.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: I believe at least a simple testing would be great to perform:

The performed browsing test steps should be:

• normally browse (go directly to urls, use bookmarks, history)
• search via the search bar
• check tiles on the home page are getting thumbnails
• may be good to try to switch the first and the second step in a new browsing session

If the same is performed in PB mode, then expect (where expected) to get another proxy creds prompt

1. have a proxy (configured whichever way - manually or via PAC) requiring a Basic authentication

• perform the browsing test steps
• expected: the browser asks for the proxy credentials only once; there should be no errors then

2. have a proxy requiring NTLM+Basic (as fallback) authentication set up (squid can be configured to do so) + let it accept any credentials or configure it to accept credentials from Windows logged in users (default credentials); any credentials will probably be easier

a. configure the browser to allow sending the default credentials to the proxy

• perform the browsing test steps
• expected: no proxy creds prompt should be raised at all and no errors connecting end servers

b. configure the browser to NOT send the default credentials

• perform the browsing test steps
• expected: the browser asks for the proxy credentials only once; there should be no errors then

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): I tend to say low as this really should be the last place we deal with the proxy credentials caching to be fixed.
• String changes made/needed: none

Comment 25

[Honza Bambas (:mayhemer)]

The reason why this bug manifests is following:

• the first bug in the proxy-creds series (bug 1520125) saves the credentials for the very first request (detect portal) which has empty OA assigned and can then be found in the cache; that case hits the only code path (fixed) looking for that cached entry
• the second one (bug 1538737) involving NTLM, hits more code paths that search for the cached entry, still, the first cached entry is for a request that has empty OA (detect portal)
• with the steps to reproduce this bug we only create creds entry (entries) from requests with non-empty OA suffix; that prevents all other requests from finding any creds entry and makes us pop the prompt up

Comment 26

[Arthur K. (he/him)]

Is the fix going to be uplifted to 67 RC1 or RC2?

Comment 27

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/3502dda7734c

Comment 28

[Rares Doghi, Desktop QA]

Hi guys, I tried reproducing this issue in the older versions of Beta but after setting a proxy in Firefox I would only get asked once for the credentials, after which I could navigate without issues, I did however encounter this issue on a Nightly build from April 4th where I would just use the credentials and it would immediately ask for them again, but this issue did not occur in our latest Nightly build or Beta.

Arthur can you please take a look and re-test this issue on your end using one of our latest Nightly builds ? you can find the build here: https://nightly.mozilla.org/

Comment 29

[Arthur K. (he/him)]

(In reply to Rares Doghi from comment #28)

Hi guys, I tried reproducing this issue in the older versions of Beta but after setting a proxy in Firefox I would only get asked once for the credentials, after which I could navigate without issues, I did however encounter this issue on a Nightly build from April 4th where I would just use the credentials and it would immediately ask for them again, but this issue did not occur in our latest Nightly build or Beta.

Arthur can you please take a look and re-test this issue on your end using one of our latest Nightly builds ? you can find the build here: https://nightly.mozilla.org/

I went to the link to get it but it was a stub installer. Yeah, no. So instead I tested with 68.0a from https://ftp.mozilla.org/pub/firefox/nightly/2019/05/2019-05-01-04-21-12-mozilla-central/ and it did not repro. Not sure why it didn't repro though.

Comment 30

[Julien Cristau [:jcristau]]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

67 is on release, updating uplift approval flag.

Comment 31

[Rares Doghi, Desktop QA]

I will mark this issue verified as Fixed on Firefox 68 based Comment 29 and that I was unable to reproduce this issue in our latest Nightly.

Comment 32

[Pascal Chevrel:pascalc]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

The issue is not easily reproducible by QA, the patch landed on nightly only 2 days ago which doesn't give us time to evaluate if it caused new regressions and it has no automated tests. It seems too risky to me for an uplift in our last RC. I suggest that we let it ride the trains to 68, if there are more reports about this issue and that we feel more confident that no new regressions were caused in 68 early betas, feel free to nominate it again for a potential 67 dot release. Thanks.

Comment 33

[Arthur K. (he/him)]

(In reply to Pascal Chevrel:pascalc from comment #32)

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

The issue is not easily reproducible by QA, the patch landed on nightly only 2 days ago which doesn't give us time to evaluate if it caused new regressions and it has no automated tests. It seems too risky to me for an uplift in our last RC. I suggest that we let it ride the trains to 68, if there are more reports about this issue and that we feel more confident that no new regressions were caused in 68 early betas, feel free to nominate it again for a potential 67 dot release. Thanks.

Whilst I agree that it's very late in the RCs, I would think it wise to at least have the patch ready for an immediate dot release should it prove to be a problem. In terms of difficulty of reproducing, absence of evidence is not evidence of absence.

Comment 34

[Arthur K. (he/him)]

I don't know that it makes much of a difference but after updating to 67.0RC2 and disabling "always use private browsing mode", it's not repeatedly popping up the login dialogue box. Just entering it one time works.

Comment 35

[czietz]

(In reply to Pascal Chevrel:pascalc (PTO until June 2) from comment #32)

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin
[...] if there are more reports about this issue [...]

In that case let me mention that I'm affected by that bug, too. It is particularly annoying in private browsing mode because in non-private mode at least the signon.autologin.proxy preference kicks in and and causes the dialog not to appear: https://github.com/mozilla/gecko-dev/blob/master/toolkit/components/passwordmgr/LoginManagerPrompter.jsm#L607

However, the authentication dialog popping up dozens of times per website(!) makes private mode totally unusable for me since the update to Firefox 67. Please don't let users live with that severe bug until Firefox 68 but fix this as soon as possible in a 67.x release!

Comment 36

[Honza Bambas (:mayhemer)]

czietz, I think you want to flip network.auth.private-browsing-sso in about:config. I know that PB mode should allow autologin to proxies by default (want to file a bug for it), but this is a workaround for your issue (it is also a very different bug, by the way), if I understand it correctly, before that is fixed.

Comment 37

[czietz]

Honza, sorry, but flipping network.auth.private-browsing-sso to true does not change anything with respect to the problem: the proxy authentication dialog still pops up multiple times in private browsing mode.

It's not that private browsing mode is the problem per se. The bug mentioned above is the root cause, regardless of private mode. It's just that outside of private mode the bug is alleviated by the password manager automatically returning the saved password every time an auth dialog is due to appear. In private mode this is blocked (see my first comment) and therefore the dialog keeps popping up, rendering private mode almost unusable.

Comment 39

[Honza Bambas (:mayhemer)]

(In reply to czietz from comment #37)

In private mode this is blocked (see my first comment) and therefore the dialog keeps popping up, rendering private mode almost unusable.

So, is your bug the same as this one and you are just asking for uplift or is there another bug we should file and fix? In the former case, it's not up to me. In the latter case, please open a new bug and cc me. Thanks.

Comment 40

[Honza Bambas (:mayhemer)]

OK, got it after I've read bug 1554859. czietz, just to confirm - when you update to Firefox Beta 68, the problem is fixed?

This seems to have rather bad impact. After you confirm that 68 works for you with otherwise same setup, I will ask release drivers to possibly reconsider an uplift, but changes are not high.

Thanks!

Comment 41

[czietz]

I'm sorry, since this is a company computer and software is rolled out by the IT department, I cannot update to Firefox 68 beta (nor can I downgrade to a Firefox version before 67, where the bug wasn't present, either). I'm currently stuck at Firefox 67 with its unusable (to me) private browsing mode. At home, where I could try 68 beta, I don't have a proxy server so that I cannot test it there, either. Again, sorry.

However, I'm fairly confident that I'm affected by this exact bug and therefore the fix already in 68 beta would solve it. I just would like to see it released as early as possible, like in a 67.x release.

Comment 42

[Honza Bambas (:mayhemer)]

Thanks for a quick answer. Then it will be harder to push on release drivers if we don't have a confirmation this is fixed in 68.

I can test locally if you tell me few details, like what authentication the proxy is setup for.

Comment 43

[czietz]

After finding out that I can install Firefox without administrative privileges, I was indeed able to confirm that the bug is fixed in Version 68.0b4 (Windows, 64-bit). With that beta, I only get one proxy authentication dialog when entering private browsing mode, same as with Firefox versions before 67.

Comment 44

[Julien Cristau [:jcristau]]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

see comment 24

Comment 45

[Honza Bambas (:mayhemer)]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

Beta/Release Uplift Approval Request

• User impact if declined: It was discovered (comment 35 and bug 1554859) that this bug very negatively affects Private Browsing when there is a proxy requiring authentication set up. We pop the credentials dialog on nearly every new request or navigation. There is no known workaround for this.

There are two confirmations (comment 43 and again bug 1554859) that 68 having this fix doesn't manifest the problem.

There are two dependencies (bug 1538737,bug 1520125) that are both present on 67, so there is no need for any other uplifts to land this bug.

• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce: (I think external user confirmation should suffice here)
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a very isolated fix affecting only proxy authentication, the patch is large only because it adds few more LOGs for making diagnoses easier.
• String changes made/needed: none

Comment 46

[Honza Bambas (:mayhemer)]

I will provide a patch to apply on m-r.

Comment 47

[Honza Bambas (:mayhemer)]

Attachment: for release/67

Comment 50

[Pascal Chevrel:pascalc]

Comment on attachment 9064452 [details]
Bug 1548804 - Remove origin suffix isolation for proxy credentials when setting authentication cache entry, r=valentin

We have a bunch of duplicates showing that this issues is impacting users now and the fix on beta proved effective, approved for 67.0.2.

Comment 51

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-release/rev/dab76a57636a

Comment 52

[Cristian Fogel [:cfogel]]

Hey Arthur,
Mind checking with the current 67.0.2 build as well?

Thank you!

Comment 53

[Arthur K. (he/him)]

(In reply to Cristian Fogel, QA [:cfogel] from comment #52)

Hey Arthur,
Mind checking with the current 67.0.2 build as well?

Thank you!

Presently at work but I can test on my home machine later after 5PM CST. My home machine has 67.0.2 build1 already. Did much change between build1 and build2?

Comment 54

[Arthur K. (he/him)]

(In reply to Cristian Fogel, QA [:cfogel] from comment #52)

Hey Arthur,
Mind checking with the current 67.0.2 build as well?

Thank you!

Confirming fixed in Private Browsing Mode.