1786881 - Firefox Developer Edition 105 - broken proxy auth.

Status: VERIFIED FIXED

(Core :: Networking, defect, P1)

Description

[Anrdei]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:104.0) Gecko/20100101 Firefox/104.0

Steps to reproduce:

System - Ubuntu 22.04
Update Firefox Developer Edition from 104b9 version to 105b2.

Actual results:

Noticed that browser is no longer authorized in my proxy. Also with full resetting the browser or reinstall.
Firefox shows the proxy authorization window by circle but never authorizes, no matter what you type in form fields.
Roll back to 105b1 - confirmed same behavior
Roll back to 104b9 - this version has normal proxy auth

Expected results:

Correct proxy auth same as version 104b9

Comment 1

[u712301]

I have the same issue with Firefox Developer Edition 105b2 on Windows.
104b9 an before are fine.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Hi, would one of you be able to try bisecting this with the mozregression tool so we can try to identify the culprit? That would be extremely helpful for getting this resolved promptly. Thanks!
https://mozilla.github.io/mozregression/

Comment 3

[Kershaw Chang [:kershaw]]

An http log would be also very helpful.
You could send the log to [e-mail] if there is any privacy concern. Thanks.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox105 (beta) and tracked for firefox106 (nightly). However, the bug still isn't assigned.

:ghess, could you please find an assignee for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit auto_nag documentation.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:valentin, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[Valentin Gosu [:valentin] (he/him)]

I think we can still wait for a response from the reporter.

Comment 7

[Timea Zsoldos [:zstimi/tzsoldos], Desktop QA]

Attachment: Proxy.png

I've tried to reproduce this issue using an affected Firefox Devedition build 105.0b4 and a Firefox Devedition 104.0b9 build, on Ubuntu 22 and on Windows 10, with the following steps:

1. Start the Firefox Devedition 104.0b9 build.
2. Set in about:preferences on Network settings page the "Manual proxy configuration" with myproxy.example.com and port:8080
3. Check "Also use this proxy for HTTPS"
4. No proxy for: .wikipedia.org, .youtube.com
5. Update from 104.0b9 to 105.0b4
   Results in both cases wikipedia.org is authorized and youtube.com not authorized, in this case Firefox shows the loading circle but never authorizes, see the attached Proxy.png screenshot.
   I verified on older builds but found the same results.
   Let me know if we can help with more info!

Comment 8

[Anrdei]

Attachment: Screenshot from 2022-08-31 14-03-17.png

Moz regression-gui results

Comment 9

[Anrdei]

Hi all! Run mozregression-gui tool at work, where internet is not available without a proxy:

2022-08-31T13:49:00.274000: INFO : Narrowed integration regression window from [64a7bac0, efd9f190] (3 builds) to [8ca49298, efd9f190] (2 builds) (~1 steps left)
2022-08-31T13:49:00.290000: DEBUG : Starting merge handling...
2022-08-31T13:49:00.290000: DEBUG : Using url: https://hg.mozilla.org/integration/autoland/json-pushes?changeset=efd9f190d5062d057ebe3a6adeb6ba235461cb62&full=1
2022-08-31T13:49:00.290000: DEBUG : redo: attempt 1/3
2022-08-31T13:49:00.290000: DEBUG : redo: retry: calling _default_get with args: ('https://hg.mozilla.org/integration/autoland/json-pushes?changeset=efd9f190d5062d057ebe3a6adeb6ba235461cb62&full=1',), kwargs: {}, attempt #1
2022-08-31T13:49:00.291000: DEBUG : urllib3.connectionpool: Resetting dropped connection: hg.mozilla.org
2022-08-31T13:49:00.292000: DEBUG : urllib3.connectionpool: Starting new HTTPS connection (3): hg.mozilla.org:443
2022-08-31T13:49:03.390000: DEBUG : urllib3.connectionpool: https://hg.mozilla.org:443 "GET /integration/autoland/json-pushes?changeset=efd9f190d5062d057ebe3a6adeb6ba235461cb62&full=1 HTTP/1.1" 200 None
2022-08-31T13:49:03.423000: DEBUG : Found commit message:
Bug 1772201 - Fix intermittent test failure, r=necko-reviewers,dragana

1. Make the compressor output errors based on the input headers
2. Make sure the weak ptr is always released on socket thread

Differential Revision: https://phabricator.services.mozilla.com/D153741

2022-08-31T13:49:03.423000: DEBUG : Did not find a branch, checking all integration branches
2022-08-31T13:49:03.424000: INFO : The bisection is done.
2022-08-31T13:49:03.430000: INFO : Stopped

Comment 10

[Valentin Gosu [:valentin] (he/him)]

Thank you!

Comment 11

[Kershaw Chang [:kershaw]]

Hi Anrdei,

Could you try to capture a http log?

Thanks.

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1772201

Comment 13

[Anrdei]

(In reply to Kershaw Chang [:kershaw] from comment #11)

Hi Anrdei,

Could you try to capture a http log?

Thanks.

Hi!
Sent logs from Firefox 105b1 to [e-mail] with subject "Mozilla Firefox bug 1786881 105b1 logs"
Sorry for delay)

Logs terminal exports:

$ export MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5,cookie:5
$ export MOZ_LOG_FILE=~logs/105b1/log.txt

Comment 14

[Kershaw Chang [:kershaw]]

Thanks for your help!

In order to help us understand the root cause, could you try to capture the log again with Firefox 104.b9?
I'd like to see how the proxy connection is established.

Thanks again.

Comment 15

[Anrdei]

(In reply to Kershaw Chang [:kershaw] from comment #14)

Thanks for your help!

In order to help us understand the root cause, could you try to capture the log again with Firefox 104.b9?
I'd like to see how the proxy connection is established.

Thanks again.

Hi!
Sent logs from Firefox 104b9 to [e-mail] with subject "Mozilla Firefox bug 1786881 104b9 logs"

Comment 16

[Kershaw Chang [:kershaw]]

Hi Anrdei,

Unfortunately, we don't have a suitable environment to test this code, so may I ask you to download a test build and test if the problem still exists?
The test build is created from this try push.

Thanks.

Comment 17

[Anrdei]

(In reply to Kershaw Chang [:kershaw] from comment #16)

Hi Anrdei,

Unfortunately, we don't have a suitable environment to test this code, so may I ask you to download a test build and test if the problem still exists?
The test build is created from this try push.

Thanks.

Ho-ho! Proxy-authorization was successful from this build!
Good work guys! You all are awesome :)

Comment 18

[Kershaw Chang [:kershaw]]

Attachment: Bug 1786881 - Reuse connection for connection based auth, r=#necko

Comment 19

[Ryan VanderMeulen [:RyanVM]]

(In reply to Anrdei from comment #17)

Ho-ho! Proxy-authorization was successful from this build!
Good work guys! You all are awesome :)

Thank you very much for the testing! These problems can be really tough to track down without the exact right environment for reproducing the problem. Looks like we should have a fix in DevEdition by some time next week.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5af24f8fa239
Reuse connection for connection based auth, r=necko-reviewers,valentin

Comment 21

[Kershaw Chang [:kershaw]]

Comment on attachment 9292601 [details]
Bug 1786881 - Reuse connection for connection based auth, r=#necko

Beta/Release Uplift Approval Request

• User impact if declined: Connection based authentication (e.x., NTLM) is broken.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: N/A
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This patch just reverts a change added in bug 1772205.
• String changes made/needed: N/A
• Is Android affected?: Yes

Comment 22

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/5af24f8fa239

Comment 23

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9292601 [details]
Bug 1786881 - Reuse connection for connection based auth, r=#necko

Approved for 105.0b7. Thanks again for the report, Anrdei!

Comment 24

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/c460006ce993

Comment 25

[Cristian Baica [:cbaica], Release Desktop QA]

Hello Anrdei,
The issue has been uplifted to beta builds and will be riding the 105 train to release. Could you please perform a quick check using a build for your OS from this location to verify that everything is in order before going to release with this?

Comment 26

[Anrdei]

Took the version from this path:
/pub/firefox/candidates/105.0b7-candidates/build1/linux-x86_64/
Version (105b7) working well with my proxy

Thank you all for your tireless work! Firefox Dev edition really important for my work on the web and with the web, thx :)
Glad I was able to help out a little)

Comment 27

[Cristian Baica [:cbaica], Release Desktop QA]

Closing the issue as verified fixed based on the previous comment.
Thank you so much Anrdei for helping us with the verification!