Open Bug 1982072 Opened 4 months ago Updated 1 month ago

Some sites do not open in 140esr

Categories

(Core :: Networking, defect)

Product:
Core
Component:
Networking
Version:
Firefox 129
Type:
defect
Points:
?

Tracking

()

Status:
UNCONFIRMED

People

(Reporter: hekoqoto, Unassigned, NeedInfo)

Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0
Firefox for Android

Steps to reproduce:

After updating ff from 128esr to 140esr, some sites stopped opening, for example, qwen3.app, temp-mail.io...

It tries to connect for a long time, several minutes, then it writes that the connection timed out.

They can open 1 time out of 100, for example.

I changed all the settings, both doh and http3.

I cleared the cache, created a new profile, tried on another computer and provider.

Everything works through the proxy plugin.

Everything also works in ff 128esr and in other browsers, chrome, chromium, vivaldi...

I checked ff versions before 128 esr and after, and found that this problem appeared in 129.0.

As far as I understand, the network parts of doh, dns, etc. were changed in this version.

All these sites work through cloudflare. Perhaps this is somehow related to this.

This also applies to the mobile version.

In Russia, sometimes sites are blocked, but these sites are not blocked and open in other browsers without additional plugins.

I left a message for support https://support.mozilla.org/ru/questions/1527455 and webcompat https://webcompat.com/issues/170455, but this did not give any results.

Perhaps this message will be useful.

Best regards,

Actual results:

Websites do not open.

Expected results:

Websites should open.

Group: firefox-core-security

The Bugbug bot thinks this bug should belong to the 'Firefox for Android::Browser Engine' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Browser Engine
Product: Firefox → Firefox for Android
Component: Browser Engine → General
Product: Firefox for Android → Firefox
Points: --- → ?
OS: Unspecified → All
Hardware: Unspecified → All
Version: Firefox 140 → Firefox 129

All these sites work through cloudflare.

What do you mean by this?

Component: General → Networking
Flags: needinfo?(hekoqoto)
Product: Firefox → Core

(In reply to :Gijs (he/him) from comment #2)

All these sites work through cloudflare.

What do you mean by this?

Hello!

I mean that all the sites in the ticket and a few more sites work through cloudflare, and maybe this is somehow related to the lack of connection. For some reason it does not want to connect to them via https2, and after http1 it immediately tries to connect to http3, and if http3 is disabled, nothing happens.

This happens starting with version 129.

Best regards,

Redirect a needinfo that is pending on an inactive user to the triage owner.
:smayya, since the bug has recent activity, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(hekoqoto) → needinfo?(smayya)

I didn't manage to reproduce this issue on Ubuntu 22.04 using Firefox 140.3.0esr. Since the reporter already tried multiple machines, profiles and providers, this may be due to specific network conditions. Marking as qa-not-actionable for now.

QA Whiteboard: qa-not-actionable

In Russia, sometimes sites are blocked, but these sites are not blocked and open in other browsers without additional plugins.

It sounds like this is connected to the use of ECH and HTTP/3 ?
Could you check if the problem goes away after you set:
network.dns.http3_echconfig.enabled: false
network.dns.echconfig.enabled: false
in about:config

Please check with https://nightly.mozilla.org/ if possible. Thanks!

Flags: needinfo?(smayya) → needinfo?(hekoqoto)

I think I may also be experiencing this issue, or a similar one - happy to make a separate bug if required - on a windows 11 install, latest normal firefox.

It is intermittent but very noticeable when it does happen and genreally fairly reproducible.

I've got a screen-recording of this happening, whilst I have the dev tools open also.

Using mitmproxy fully alleviates the issue, as does disabling http3. Disabling/setting to false the echconfig options as described above also appears to still allow the issue to occur.

It generally recovers itself when trying again after a short time.

For ref, this impacts most significantly for me, Reddit and LinkedIn.

Happy to get captures and screenshots and whatever else needed to help debug this.

Also to add; I was able to reproduce in a fresh profile w/ no extensions.

Ah - to add also: I'm on Firefox 144.0.2 (64-bit)

Could you capture some logs as described at the following link?
https://firefox-source-docs.mozilla.org/networking/http/logging.html

Thanks!

(In reply to Valentin Gosu [:valentin] (he/him) from comment #9)

Could you capture some logs as described at the following link?
https://firefox-source-docs.mozilla.org/networking/http/logging.html

Thanks!

Yup, reproduced here: https://share.firefox.dev/4qUopI0 . Image that failed is https://preview.redd.it/tdto4cutfk0g1.jpeg?width=917&auto=webp&s=41837a328946f2f3ac46c795157ab4dc4985acb3

I did this on a fresh Firefox profile w/ no extensions, so all should be there as required?

Looks like it failed on the 8th click. Not sure why the "play" image expansion icon is missing - I don't think it's related(!)

I also captured this using the Networking Preset as opposed to the HTTP/3 preset I used above, and I focused it down on the issue using ctrl-shift-1/2 to start/stop the profiler appropriately. https://share.firefox.dev/3XoMnxm The failure image here should be https://preview.redd.it/qqm329qoll0g1.jpeg?width=1024&auto=webp&s=dcc243c57077dfcb81fe422da078ece3ca9560d1, but I see a few other failures of this family above that, and they all die at the same time.

LogMessages — (nsHttp) HttpConnectionUDP::Close [this=16f8934a000 reason=804b000e]
which causes the call to:
LogMessages — (nsHttp) HttpConnectionUDP::Close [this=16f8934a000 reason=80004004]

Flags: needinfo?(kershaw)

Hi Richard,

Thanks for the log — it’s very helpful.
I think what’s happening is that the HTTP/3 connection is in the 0-RTT state, but UDP is blocked, so the data can’t be sent out. Unfortunately, our fallback mechanism doesn’t handle this case, which causes the request to time out.

To confirm this, could you open about:config, search for network.http.http3.enable_0rtt, set it to false, and see if you can still reproduce the issue?

If it still occurs, could you capture another HTTP log with network.http.http3.enable_0rtt set to false?

Thanks!

Flags: needinfo?(kershaw) → needinfo?(Danse)

Yep, it still occurs, and here's a HTTP/3-profile log: https://share.firefox.dev/3JZ42bV .

Just to check; that's the right profile? Would you prefer another or some other custom string?

Happy to guinea pig here as it makes browsing a right pain in the proverbial.

Flags: needinfo?(Danse)

(In reply to Richard Fontaine from comment #14)

Yep, it still occurs, and here's a HTTP/3-profile log: https://share.firefox.dev/3JZ42bV .

Just to check; that's the right profile? Would you prefer another or some other custom string?

Happy to guinea pig here as it makes browsing a right pain in the proverbial.

Hi,
It seems that the profile you shared is incomplete. Many details are missing.
Could you try capturing a log file instead?
In about:logging, please select “logging to a file.” The log file will be created in your temporary folder. Once it’s ready, please send it to necko@mozilla.com.
Thanks.

Flags: needinfo?(Danse)

Apols, think I stuffed up the setup on the first run.

Here's what should be a good link: https://share.firefox.dev/43qqHEE

Flags: needinfo?(Danse)

I think the profiler isn't capturing the socket thread while profiling HTTP/3 ue to bug 1996746.
Could you use this URL to profile instead? about:logging?output=profiler&preset=http3&profiler-preset=networking
I think you also need to be using Firefox Nightly, not Firefox Release for the rust modules to be correctly captured by the profiler.

Flags: needinfo?(Danse)

I've just fresh-installed and clicked all the buttons in this one: https://share.firefox.dev/3JIMEZ1 .

An example file with the issue is: oldl4ehdox0g1.jpg

There's also an odd one in there w/ this URL: https://www.redditmedia.com/mediaembed/1ovlc92 . Though that died after 9s, and behaved differently.

Flags: needinfo?(Danse)

Hi,

Could you open about:config and check the value of network.http.http3.backup_timer_delay?
From the log you shared, it looks like UDP might be blocked, and Firefox isn’t falling back properly to HTTP/2. I wonder if that’s because the preference is set to 0.

Thanks.

Flags: needinfo?(Danse)

(In reply to Kershaw Chang [:kershaw] from comment #19)

Hi,

Could you open about:config and check the value of network.http.http3.backup_timer_delay?
From the log you shared, it looks like UDP might be blocked, and Firefox isn’t falling back properly to HTTP/2. I wonder if that’s because the preference is set to 0.

Thanks.

Hi

It's set at 100 (which I believe is the default).

I've made no changes to defaults from the nightly install that I used for that last report.

Flags: needinfo?(Danse)

Thanks for confirming.
To help us investigate further, could you please capture a Wireshark trace while reproducing the issue?
Please also include the SSL key log file, so we can decrypt HTTPS traffic in the trace.
Once done, please embed the key to the pcap file and send it to necko@mozilla.com.

Flags: needinfo?(Danse)

Hi,

I'll get on that a bit later - do you mind if I filter and only provide requests originating from the host computer?

Originating/returning to, I should add.

(In reply to Richard Fontaine from comment #22)

Hi,

I'll get on that a bit later - do you mind if I filter and only provide requests originating from the host computer?

Of course. Thanks for your help.

I've sent the requested wireshark capture. I did not also capture firefox logging/profiling stuff (which in hindsight might have helped!).

For ref; that pcap is capture-filtered on my system's MAC.

Flags: needinfo?(Danse)

Thanks for the Wireshark trace. It’s very helpful.

From the trace, it looks like the server stops responding at some point, so this appears to be a server-side issue.
Dennis, could you reach out to Reddit and ask them to take a look at this? Thanks.

Flags: needinfo?(dschubert)

FWIW I also have this suspect behaviour on LinkedIn, Reddit was just a simple and reliable reproduction case - LinkedIn's issue manifests slightly differently.

I'd also vaguely argue that the browser might handle the lack in response better, maybe?

I wonder if there could be some LAN/ISP-caused issue impacting this also?

Thanks for looking into it though, and I'll keep http3 disabled on my main usage of the browser until some fix comes in :)

You need to log in before you can comment on or make changes to this bug.