Open Bug 2055234 Opened 4 months ago Updated 28 days ago

Content process appears to enter degraded networking state after prolonged browsing; GitHub SPA no longer completes loading

Categories

(Core :: Networking, defect)

Firefox 152
defect

Tracking

()

UNCONFIRMED

People

(Reporter: martin+mozilla, Unassigned, NeedInfo)

References

Details

Attachments

(4 files, 2 obsolete files)

Attached image github.png β€”

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0

Steps to reproduce:

I'm a heavy GitHub users do the projects I maintain and contribute (mostly nixos/nixpkgs) there.

  • Use Firefox normally for several hours
  • Browse GitHub extensively (multiple repositories, pull requests, issues)

Actual results:

Eventually GitHub tabs stop loading completely and the loading indicator keeps spinning. Anything can be missing from CSS to XHR.

This is limited to a single content processes, because when I manage to create a new one GitHub browses fine, while on the old processes they remain degraded.

Expected results:

GitHub should continue issuing all required network requests and complete loading regardless of how long Firefox has been running.

The Bugbug bot thinks this bug should belong to the 'Core::Networking' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Networking
Product: Firefox → Core

Thanks for filing this bug Martin!
We need some more logs to confirm the actual root cause.

Next Steps

  1. When a tab is stuck spinning, please capture a profile: open about:logging, choose the Networking preset (timestamp,sync,nsHttp:5,nsSocketTransport:5,nsHostResolver:5), and under Logging output select Logging to the Firefox Profiler. Click Start Logging, reproduce the stuck load once, then Stop Logging and upload the resulting profile, pasting the share URL here.
  2. Still with the affected tab open and stuck, open a new tab and load about:networking#sockets and about:networking#http; a screenshot of those pages while the other tab is degraded would show whether connections/requests to github.com are piling up or stuck.
  3. A copy of about:support would help confirm the exact build/channel and any non-default networking prefs.
  4. If you get a chance, trying Troubleshoot Mode (all extensions disabled) for a similarly long session would help rule out an add-on holding connections open in that process.

No need to do all of these if only one is practical (that would help pin this down). If you are concerned about sharing this info publicly, please send them to necko@mozilla.com.

Thanks again for supporting Firefox

Flags: needinfo?(martin+mozilla)
Attached file about:support β€”
Attached image about:networking (http) (obsolete) β€”
Attached image about:networking (sockets) (obsolete) β€”

I sent two profiles to necko@mozilla.com a few minutes ago. I have more confidence in the second one being recorded properly. I correlated about:networking while it was recording that and it showed GitHub connections.

Flags: needinfo?(martin+mozilla)

Only the second report made it because I didn't understand that I needed to press "Upload". The screenshots are related to that second attempt.

Unfortunately the profile did not capture any requests to cloudflare.
Could you try submitting one again, if you are able to reproduce?

Flags: needinfo?(martin+mozilla)

https://share.firefox.dev/4h96w5X

Another attempt at loading https://github.com/NixOS/nixpkgs/pull/545717. I refreshed the tab multiple times.

The tab keeps spinning and saying "Transferring data from github.githubassets.com".

Flags: needinfo?(martin+mozilla)
Attached image about:networking (http) β€”
Attachment #9610238 - Attachment is obsolete: true
Attached image about:networking (sockets) β€”
Attachment #9610239 - Attachment is obsolete: true

Hi reporter,

We fixed a similar issue recently. Could you please download Firefox Nightly and check whether you can still reproduce this?
If you can still reproduce it, could you use about:logging to capture a networking log again? Note that the profile in comment #9 does not contain any networking logs. When uploading the profile, please make sure to check the Include hidden threads box.

Thanks.

Flags: needinfo?(martin+mozilla)

https://share.firefox.dev/44YKb3x same profile including hidden threads.

Will look into reproducing with nightly, but since it sometimes takes over a week to degrade it's kinda hard to reproduce outside continuous normal browsing.

Flags: needinfo?(martin+mozilla)

Sorry, that was just a re-upload from the previous profile.

Flags: needinfo?(martin+mozilla)

I don't think I can quickly reproduce this on nightly. But it still happens on 154.0.

https://share.firefox.dev/462U3tP

Flags: needinfo?(martin+mozilla)

I didn't notice anything odd in the networking panel - all requests that are started seem to complete.
The one thing I found very odd in the Marker Chart were the AsyncShutdown blockers for 5 minutes https://share.firefox.dev/3U4GhEe
Jens, is this expected? Or are we maybe failing to shutdown the content process, and that is preventing new channels from being created?

Flags: needinfo?(jstutte)

Those markers are expected and don't indicate a shutdown problem.

ContentParent registers itself as a blocker on xpcom-will-shutdown /
profile-before-change / quit-application when it launches the child, and drops
them in ActorDestroy():

https://searchfox.org/firefox-main/rev/092b4be38e4fa42c638ff6700e3f833f71085fd2/dom/ipc/ContentParent.cpp#3879
https://searchfox.org/firefox-main/rev/092b4be38e4fa42c638ff6700e3f833f71085fd2/dom/ipc/ContentParent.cpp#1942

The marker is emitted when the blocker is removed, with startTime set to when
it was added, so the interval is the content process' entire lifetime, not time
spent waiting for shutdown - hence also the negative start times. And in this
profile every one of those intervals ends inside the recording (id=12403 at
52.1s, 12408 at 28.1s, 12409 at 13.9s, 12412 at 1m2s, 12413 at 1m4s), i.e. those
five ContentParents did reach ActorDestroy. The two ~5s "ReleasingTimerHolder"
ones are the blob URL revocation timer. A shutdown that really got stuck would
show up as an "AsyncShutdown timeout" marker instead, and only during app
shutdown, which never happens in this session. Blockers registered while the
browser runs are inert - nothing consults the barrier until the phase starts, so
they cannot gate channel creation.

That this marker is misleading in exactly this way is being fixed in bug 2065408:
D320447 (r+) adds a separate "AsyncShutdown blocker wait" marker covering only
the interval where the barrier actually waits.

On networking I agree with your read: in the process with the spinning tab (pid
4129733) all 170 requests have a completion, and something is in flight for only
2.65s of the 65s. One caveat: a channel already open when profiling started has
no start marker, so it wouldn't appear here at all.

Martin, two things that may still help:

  • The troubleshoot mode question from comment 2 seems still open. This profile has
    31 extensions (Refined GitHub, uBlock Origin, uMatrix, Dark Reader, Stylus,
    Greasemonkey, ...), and a content script wedged in one content process would
    match "only this one process is degraded" quite well.
  • With the tab stuck, is there a request sitting in "pending" in the DevTools
    network panel, and what does about:networking#http say for the
    github.githubassets.com session?
Flags: needinfo?(jstutte)
See Also: → 2065408
Flags: needinfo?(martin+mozilla)
See Also: → 2079806
See Also: → 2059903
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: