1964030 - Intermittently, pages just stall loading - they keep on loading and loading. Page opened in Chrome load instantly.

Status: RESOLVED FIXED

(Core :: Networking, defect, P1)

Description

[Mayank Bansal]

Since maybe 2 weeks, intermittently pages will stop loading - they will keep on loading and loading. If i open the same page on Chrome, they load normally.

I have captured a partial log - log was started when i was already seeing the pageload stall. But the log does capture the moment when the pages again started loading.
Profile: https://share.firefox.dev/44fTKvS

Comment 1

[Mayank Bansal]

Attachment: about:support

Comment 2

[Mayank Bansal]

Usually if I close the browser while the pages have stalled loading, the browser wont quit cleanly. When i restart, i get the message "Another instance of firefox is running. do you want to close that".
I had filed bug 1962022 for some crashes i was seeing at teh same time. So the stalling, the dirty shutdown and the occassional crash all happen together. The crashes have been fixed since bug 1962022 though.

Comment 3

[Valentin Gosu [:valentin] (he/him)]

Thank you for the profile. It's really odd that nothing is really happening on either the cache or the socket threads.
I did see a request for http://wpad/wpad.dat at the end, so I do wonder if there's another proxy request going on in the background, or if this is just another issue similar to bug 1937367.

@JanErik, in the profile here I do see that setupTelemetry is lasting for a very long time - Is it possible that addon initialization is waiting for telemetry and in turn that's blocking network requests?
https://share.firefox.dev/4cXKnU1

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1964030 - Add NS_DISPATCH_EVENT_MAY_BLOCK for GetPACFromDHCP runnable r=#necko

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Additionally I noticed that https://phabricator.services.mozilla.com/D243076 doesn't include the NS_DISPATCH_EVENT_MAY_BLOCK when dispatching the potentially blocking runnable. I don't think that would block the entire thread pool, but it's better to be safe.

Comment 6

[Jan-Erik Rediger [:janerik]]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #3)

@JanErik, in the profile here I do see that setupTelemetry is lasting for a very long time - Is it possible that addon initialization is waiting for telemetry and in turn that's blocking network requests?
https://share.firefox.dev/4cXKnU1

Uhm ... not that I know of? That code is essentially untouched for a long-time and we haven't had any other reports about it.

Comment 7

[Jens Stutte [:jstutte]]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #5)

Additionally I noticed that https://phabricator.services.mozilla.com/D243076 doesn't include the NS_DISPATCH_EVENT_MAY_BLOCK when dispatching the potentially blocking runnable. I don't think that would block the entire thread pool, but it's better to be safe.

Well, it just directs the event to a different pool with a few more threads. But once all those threads are blocked by really blocking events, the entire pool will block.

Comment 8

[Phabricator Automation]

Comment on attachment 9485021 [details]
Bug 1964030 - Add NS_DISPATCH_EVENT_MAY_BLOCK for GetPACFromDHCP runnable r=#necko

Revision D247584 was moved to bug 1964064. Setting attachment 9485021 [details] to obsolete.

Comment 9

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1964030 - Make sure only one GetPACFromDHCP runnable is active at any time r=#necko

Dispatching multiple of them during a period when the system doesn't
return from the DhcpRequestParams call could potentially block all
background thread pool tasks.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f3a372d30948
Make sure only one GetPACFromDHCP runnable is active at any time r=necko-reviewers,kershaw

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1e64a48e3517
Revert "Bug 1964030 - Make sure only one GetPACFromDHCP runnable is active at any time r=necko-reviewers,kershaw" for causing high frequency Gtest failures complaining about TestPACMan.

Comment 12

[Atila Butkovits]

Backed out for causing high frequency Gtest failures complaining about TestPACMan.

Backout link: https://hg-edge.mozilla.org/integration/autoland/rev/1e64a48e351754157e6652d8a652dd5557a2bfc3

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=linux%2C18.04%2Cx64%2Cwebrender%2Copt%2Ctest-linux1804-64-qr%2Fopt-gtest-1proc%2Cgtest-1proc&revision=f3a372d30948539f9cf212214d8c7e8e6b4a4f3a&selectedTaskRun=H6n1hl_vT6ucktl9cdhEhQ.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=507046307&repo=autoland&lineNumber=13003

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/605517b61bbd
Make sure only one GetPACFromDHCP runnable is active at any time r=necko-reviewers,kershaw

Comment 14

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/605517b61bbd

Comment 15

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1964030 - Make sure only one GetPACFromDHCP runnable is active at any time r=#necko

Dispatching multiple of them during a period when the system doesn't
return from the DhcpRequestParams call could potentially block all
background thread pool tasks.

Original Revision: https://phabricator.services.mozilla.com/D247615

Comment 16

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined: The runnable dispatched to avoid hanging in an OS call may themselves hang blocking the user from making any networking requests.
• Code covered by automated testing: no
• Fix verified in Nightly: yes
• Needs manual QE test: no
• Steps to reproduce for manual QE testing: It's unclear exactly what causes the DhcpRequestParams call to hang in windows - probably network changes. Not easily reproducible (at least on my machine)
• Risk associated with taking this patch: Low
• Explanation of risk level: These two patches apply the NS_DISPATCH_EVENT_MAY_BLOCK flag to the runnable, so it ends in the correct thread pool, and make sure to not dispatch another one if we already have one in progress.
• String changes made/needed: none
• Is Android affected?: no

Comment 17

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/3e9224e28fe5