Closed Bug 1609410 Opened 4 years ago Closed 3 years ago

421 response code from `icloud.com` not handled well with a proxy

Categories

(Core :: Networking: HTTP, defect, P2)

Product:
Core
Component:
Networking: HTTP
Version:
68 Branch
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla80
Tracking Flags:
Tracking Status
firefox80 --- affected

People

(Reporter: svw, Assigned: kershaw)

Details

(Whiteboard: [necko-triaged][ntlm])

Attachments

(9 files)

Logfiles.7z
340.04 KB, application/x-7z-compressed
Details
log.txt-main.7z
2.14 MB, application/octet-stream
Details
Bug 1609410 - Allow restart of an HTTP transaction with 421 HTTP end-server response made through a newly authenticated NTLM proxy connection by unsticking the connection, r=kershaw
47 bytes, text/x-phabricator-request
Details | Review
log.txt-main.6464.moz_log
6.37 MB, application/octet-stream
Details
log.txt-main.11096.7z
387.82 KB, application/octet-stream
Details
Bug 1609410 - Clear used proxy identity in nsHttpChannelAuthProvider to prevent authentication prompt pop-up on transaction internal restart, r=kershaw
47 bytes, text/x-phabricator-request
Details | Review
logs_04082020.7z
859.35 KB, application/octet-stream
Details
Logs06082020.7z
1.03 MB, application/octet-stream
Details
Bug 1609410 - When receiving 421, don't restart the transaction that has a sticky connection
48 bytes, text/x-phabricator-request
Details | Review
Attached file Logfiles.7z —

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

In our company network one of our employees tried to log into icloud.com. The website loads but the login screen doesn´t show up. Instead he recieves a Networkerror message.

Actual results:

We use a Mcafee Proxy in our company network. After analyzing the issue we recognized that with internet explorer and firefox on a laptop without using a proxy the website loads the way it should be. Thats because while the connection is established the following link : "setup.icloud.com/setup/ws/1/validate?clientBuildNumber=1923Hotfix2&clientMasteringNumber=1923Hotfix2&clientId=7969427a-aee8-49d1-88d0-5802bb246184" recieves a 421 statuscode in internet explorer and firefox without a proxy. When useing a proxy firefox receives a 200 statuscode and the connection can´t be established. But the proxy does transmit the correct statuscode (421).

Expected results:

While using a proxy firefox should recieve the correct statuscode (as mentioned above) as the proxy transmits the correct one.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Networking
Product: Firefox → Core
Component: Networking → Networking: HTTP
Priority: -- → P2
Summary: Network error for icloud.com → 421 response code from `icloud.com` not handled well with a proxy
Whiteboard: [necko-triaged]

I can see the proxy is using NTLM authentication. Not sure this is important, but tentatively marking as NTLM related bug.

Whiteboard: [necko-triaged] → [necko-triaged][ntlm]

svw, can you please provide HTTP logs for the (minimal, if possible) use case according https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging?

Thanks.

Flags: needinfo?(svw)
Attached file log.txt-main.7z —

Sorry it took me so long to provide the needed information.
I uploaded the logfile you requested.

Flags: needinfo?(svw)
Flags: needinfo?(honzab.moz)

Thanks. The proxy is requiring NTLM authentication, the auth is successful, we get 200 Connected. The end-server sends back 421, we see it and set the transaction to restart, but we actually don't restart it because (I believe - this is unconfirmed) the NS_HTTP_CONNECTION_RESTARTABLE caps flag is not set (or NS_HTTP_STICKY_CONNECTION unset) on that transaction after we have finished the HTML auth with the proxy (by getting 200 Connected).

I believe we can set the NS_HTTP_CONNECTION_RESTARTABLE (or unset the STICKY) flag at https://searchfox.org/mozilla-central/rev/557a0e222dd104c5d805ba344c45d6abc27d3db0/netwerk/protocol/http/nsHttpTransaction.cpp#2541.

I can have an experimental build to test with.

Assignee: nobody → honzab.moz
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Flags: needinfo?(honzab.moz)

Thank you for your feedback.
Should we test your experimental build?
Where can we download it?

I didn't get to working on this bug so far, sorry. I will add comments when anything to test with is available.

Sorry for such a lag, this was considered a lower priority bug.

svw, here is a test build based of Firefox Nightly that you can test with. Please let me know how it goes.
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NfQrwmHMTV2dhXECea24sA/runs/0/artifacts/public/build/install/sea/target.installer.exe

Flags: needinfo?(svw)
Pushed by honzab.moz@firemni.cz:
https://hg.mozilla.org/integration/autoland/rev/d6d8e1c341f5
Allow restart of an HTTP transaction with 421 HTTP end-server response made through a newly authenticated NTLM proxy connection by unsticking the connection, r=kershaw,necko-reviewers

https://hg.mozilla.org/mozilla-central/rev/d6d8e1c341f5

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
status-firefox80: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla80

We tested the provided nightly version. When we try to open icloud.com, a proxy authentication window displayed and a manualy login fails (proxy settings in firefox on auto-detect). And the browser reported a error message (connectionerror). If we use our workaround and set no authentication on the proxy settings for setup.icloud.com, then the website opens.

Flags: needinfo?(svw)

(In reply to svw from comment #13)

Created attachment 9162125 [details]
log.txt-main.6464.moz_log

We tested the provided nightly version. When we try to open icloud.com, a proxy authentication window displayed and a manualy login fails (proxy settings in firefox on auto-detect). And the browser reported a error message (connectionerror). If we use our workaround and set no authentication on the proxy settings for setup.icloud.com, then the website opens.

Thanks. What was exactly the setting for logging you used? It seems to be missing some information, seems like you used nsHttp:4 instead of nsHttp:5.

Status: RESOLVED → REOPENED
Flags: needinfo?(svw)
Resolution: FIXED → ---
Attached file log.txt-main.11096.7z —

The setting was at nsHTTP:3. Here are the new logging with nsHTTP:5.

Flags: needinfo?(svw)

Thanks for the log. I will have to write a test for this code path to catch all things we are still missing here. Unfortunatelly, NTLM et al are very undertested.

Status: REOPENED → ASSIGNED

I see two things to do here:

  • drop Proxy-Authrorization request header on the restarted transaction to not confuse the proxy with a no longer valid value and rather reathenticate
  • tell the HTTP authenticator of the http channel to reset the proxy auth identity and start over.

svw, please check this test build, based of the current Nightly as before:
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BH0qbs_4Te2Xu_ZowMQk1w/runs/0/artifacts/public/build/install/sea/target.installer.exe

Thanks!

Flags: needinfo?(svw)
Attached file logs_04082020.7z —

Thanks for the new Testversion.
Now when we try to open icloud.com, the proxy authentication window displayed only one time. After a manually login the window doesn't popup anymore, but the website doesn't load completely (see the screenshot). If we clear the cache, the proxy authentication window will no longer appear.

Flags: needinfo?(svw)

Thanks.

The request:

POST /setup/ws/1/validate?clientBuildNumber=2013Project42&clientMasteringNumber=2013B29&clientId=1b1524d3-30ce-422f-a8de-a159f5bad847 HTTP/1.1
Host: setup.icloud.com

loops, we are getting 421 on and on, but proxy auth works as expected. This is then yet a different bug.

I don't see any request in the log from comment 21 that would popup the auth dialog for proxy auth.

Note that the icloud server receiving the POST is talking HTTP/1.1 and there seems to be no alternative way to talk to the server. So I'm quite puzzled what happens here.

I would be interested in a log when you exclude setup.icloud.com from proxy authentication. I'd like to compare with a "correct" flow.

Flags: needinfo?(svw)
Attached file Logs06082020.7z —

If we exclude setup.icloud.com from proxy authentication, the website opens correctly. In the attechment you will find the logs with exlude proxy authentication and the normally situation with proxy authentication. The proxy does not block setup.icloud and the client also comes with the correct user. But the HTTP-Status was 421, as you wrote from comment 22.

Flags: needinfo?(svw)
Keywords: leave-open

(In reply to svw from comment #25)

Created attachment 9168402 [details]
Logs06082020.7z

Thanks. In the log I can see the server responses with 421 to the second, restarted request. So, the only problem that my patch is causing is the loop. I don't have much time right now to attend this bug, though.

Thanks for all the logs and responses, I'll get back to this bug sometime later.

Pushed by honzab.moz@firemni.cz:
https://hg.mozilla.org/integration/autoland/rev/3f59e3e8b36d
Clear used proxy identity in nsHttpChannelAuthProvider to prevent authentication prompt pop-up on transaction internal restart, r=kershaw,necko-reviewers
Assignee: honzab.moz → nobody
Status: ASSIGNED → NEW

The leave-open keyword is there and there is no activity for 6 months.
:jstutte, maybe it's time to close this bug?

Flags: needinfo?(jstutte)

Hi Kershaw, is there anything left todo here? Thanks!

Flags: needinfo?(jstutte) → needinfo?(kershaw)

(In reply to Jens Stutte [:jstutte] from comment #30)

Hi Kershaw, is there anything left todo here? Thanks!

Yes, the problem is not fixed yet. I'll look at this.

Assignee: nobody → kershaw
Severity: normal → S4
Flags: needinfo?(kershaw)

Hello Reporter,

Could you use this build to test again?
Please also try to get the http log if you still see the same problem.

Thanks.

Flags: needinfo?(svw)

Hello,
it seems to be good in your build.
The test in Firefox ESR 78.7.1 was successfully, too.

Flags: needinfo?(svw)

Hello,
sorry for the false information.
We have call the website without proxy authentification.
If we call the website with proxy authentification, only in the nightly version it works correct.

Thank you very much.

Pushed by kjang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f45dc6a2bafc
When receiving 421, don't restart the transaction that has a sticky connection r=necko-reviewers,dragana
Keywords: leave-open
Status: NEW → RESOLVED
Closed: 4 years ago → 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: