1649902 - Network requests randomly stop working

Status: RESOLVED FIXED

(Core :: Networking, defect, P2)

Description

[Max Chernoff]

Attachment: ProcExp Screenshot showing the suspicious thread

Version

79.0b2 (Firefox Developer Edition, aurora update channel)

Overview

When browsing, all network requests will randomly stop working. It's hard to predict when this will happen, but it typically occurs within the first 20 minutes of the browsing session.

The issue

When the issue occurs, no network requests work. Navigating to a webpage, opening a new tab with Ctrl-Click, or making an XHR request in the console with fetch('http://example.com') all hang indefinitely. The requests neither time-out nor show any errors in the Developer Console. However, the browser remains otherwise responsive.

Debugging

When the issue occurs, ProcExp shows that the root Firefox process (the base firefox.exe, not the render processes) is using ~25% CPU (in a 4-core system). Upon further inspection, ProcExp shows that the thread ucrtbase.dll!configurethreadlocale+0x50 is maxing out a single CPU (25% CPU utilization).

Further inspection of the stack shows that the thread is actively running. I'm not very good at debugging, but variants of xul.dll!v8::internal::RegExpParser::has_more+0x1XXXXX occupy most of the stack. When I kill this thread, nothing really happens—the browser doesn't crash, the network requests don't start succeeding, nothing. Of course, the CPU usage drops instantly.

In addition, whether or not I kill the thread above, clicking the "X" in the browser to close the window does not successfully terminate the firefox.exe processes. The browser windows close, and some of the processes terminate, but 3 firefox.exe processes remain. The root Firefox process continues to use a full thread's worth of CPU until manually terminated, either by using Task Manager or by relaunching the browser.

Reproduction

This issue is quite random to reproduce: some browser sessions it never happens, other sessions it fails in 5 minutes. Usually, if I use Google to search for things and click on a few random links, it triggers. One sure-fire way that I have found to trigger this is by using Google Earth. When I open Google Earth, all network requests in all tabs start failing within about 10 seconds. To be clear, it is not just Google properties where this issue occurs; rather, they are just simple examples.

To further reproduce the issue, I made a new, empty profile. In this new profile, I was unable to reproduce any of these issues, even when I installed the extensions present on my main profile (UBlock Origin, Privacy Badger, HTTPS Everywhere).

The issue also occurs in a "Private Browsing" window on the main profile. This likely means that it's not a weird caching issue.

Additional Debugging

I took a memory dump when this issue occurred and imported it into windbg. Again, my debugging skills aren't very good, but I have attached the windbg report below. I can send the firefox.dmp file to a developer if required, but I'd rather avoid it since it's a 500 MB file and it likely contains sensitive information (like login cookies).

System Information

This issue is quite recent: it has only occurred after the update to 79.0b2 two days ago on the Developer Edition. I am running Windows 10, 2004, build 19041.329, x64. I have attached an about:support report below.

Attached Files

https://gist.github.com/gucci-on-fleek/92064774f2c3b1092e32039967049513

Comment 1

[Valentin Gosu [:valentin]]

Seems to happen with the network.http.http3.enabled pref set.

Comment 2

[Max Chernoff]

Setting network.http.http3.enabled to false in my main profile resolves the bug.

I also tested this in a new profile and was able to reproduce the problem.

1. Create a new profile in about:profiles and launch it
2. Go to about:config and set network.http.http3.enabled to true
3. Visit any website with HTTP/3.0 enabled (google.com seems to do the trick)
4. All future network requests hang indefinitely

One odd thing is that in my main profile, the issue doesn't appear for about 2 or 3 minutes, even when browsing HTTP/3.0-enabled sites. However, in the blank profile, it occurs immediately after loading a single HTTP/3.0-enabled site. This might just be because my main profile has cached some of the requests, but it could be something else.

Comment 3

[Max Chernoff]

I should also mention that I don't remember setting network.http.http3.enabled to true. So this means one of two things:

1. I enabled HTTP/3.0 in about:config 2 or 3 months ago to test something HTTP/3.0-related, then forgot about it and left it enabled
2. The flag enabled itself in the most recent Developer Edition update

I'm not entirely sure which of these is the case, but either way, this issue is a regression from the previous Developer Edition version (presumably 78.0).

Comment 4

[Dragana Damjanovic [:dragana]]

Can you try with Nightly?

Comment 5

[Max Chernoff]

This issue appears to be resolved now. I've tested with Nightly 80.0a1 (2020-07-15/20200715215205) and I haven't noticed any issues.

I browsed Google Earth and the CloudFlare Blog for about 15 minutes each and the performance was great (I verified in the DevTools that the requests were using HTTP/3.0). I didn't notice any adverse latency, memory usage, throughput, or performance issues, and the browser's CPU and memory usage dropped to almost nil after closing all of the open tabs.

Thanks for the fix!

Comment 6

[Dragana Damjanovic [:dragana]]

Thank you for verifying, very appropriated.

Comment 7

[elvinas]

Just seen the same situation with version 96.0.
Firefox suddenly stopped responding and processing network requests. I was not able to open local LAN addreses.

Further check with Process Explorer shoewed the same picture. One of the FireFox threads was stuck with ~25% CPU consumption (1 core). Ucrtbase.dll and configthreadlocale was listed as in the screenshot by issue reporter. Restarting Firefox did not help.

Setting network.http.http3.enabled to false (I presume it is now the default) helped to regain FF functionality. Are there some regressions?