1931956 - sendBeacon doesn't work during pageHide

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Thorsten Dencker]

Attachment: sendBeaconTest-1.zip

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36

Steps to reproduce:

Starting with Firefox 125, sendBeacon requests are no longer reliably or not all sent during the pagehide event. See Bug 1609653 for more details.

Issue Impact:
SAP relies on sendBeacon requests to properly close backend sessions when a tab or browser is closed. Previously, we migrated from the deprecated unload event (no longer supported by Chrome) to the pagehide event. If sendBeacon requests fail to reach the server, backend sessions remain open, causing business objects to become locked. This results in other users being unable to access these objects, leading to significant delays and disruptions in business operations.

Customer Impact:
This behavior causes severe delays in business processes due to locked objects and unclosed sessions. Immediate attention and resolution are required to address this critical issue. Several customers have already raised serious complaints about this issue.

Steps to reproduce:

1. Create a Test Page

• Set up a page with a pagehide event handler on the body.
• Inside the pagehide handler, send a sendBeacon request to the server.
• Add three additional iframes on the main page, each implementing the same logic (registering a pagehide event and sending a sendBeacon request).

2. Test the Following Scenarios
   Use tools like Fiddler or server log files to verify if all sendBeacon requests are sent and reach the server under the following conditions:

• Scenario 1: Press F5 or click the Refresh button.
• Scenario 2: Close the tab (while keeping at least one other tab open, so the browser does not close entirely).
• Scenario 3: Close the entire browser.

Example:
<!DOCTYPE html>
<html>
<body>

<h2>sendBeacon during pagehide event</h2>

<p>Open an http tracing tool like Fiddler and close your browser (not only the current tab - the entire browser must be closed). The sendBeacon function won't send the HTTP request in question in FireFox.</p>

<script>

window.addEventListener("pagehide", function() {navigator.sendBeacon("/test")});

</script>

</body>
</html>

Actual results:

Actual Behavior (Observed with Firefox 125):
sendBeacon requests are not (reliably) sent during the pagehide event, leading to issues such as session locks and backend disruptions

Expected results:

Expected Behavior:
In all scenarios, every sendBeacon request (from both the main page and the iframes) should successfully reach the server.

Comment 1

[edgul]

Thanks for the report. But I am not able to reproduce with the following setup:

I configured firefox:

• to run with manual HTTP proxy going to localhost:8080.
• enabled network.proxy.allow_hijacking_localhost at about:config so localhost traffic would be sent over the proxy.

I ran mitmproxy alongside without any args (localhost:8080).

Then I served the following files with php -S localhost:8000:

index.html

<!DOCTYPE html>
<html>
  <head>
    <script>
      window.addEventListener("pagehide", () => {
        console.log("Top-level pagehide triggered");
        navigator.sendBeacon("/top-level-beacon");
      });
    </script>
  </head>
  <body>
    <iframe src="iframe.html"></iframe>
    <iframe src="iframe.html"></iframe>
    <iframe src="iframe.html"></iframe>
  </body>
</html>

iframe.html

<!DOCTYPE html>
<html>
  <head>
    <script>
      window.addEventListener("pagehide", () => {
        console.log("Iframe pagehide triggered");
        navigator.sendBeacon("/iframe-beacon");
      });
    </script>
  </head>
  <body>
    <p>I am an iframe</p>
  </body>
</html>

In all 3 scenario's you mention (refresh, tab close, window close) I observe the beacon POST requests going over mitmproxy.

Are you able to provide additional information about your setup that might be causing interference?
Are you able to reproduce with a fresh profile (fresh settings and no extensions)?

Thanks!

Comment 2

[Thorsten Dencker]

I am using Firefox 133.0 (64-bit) on Windows 11.

The issue occurs when the entire browser is closed. To reproduce the problem, start the example in the browser without any other tabs open, then close the browser window. In this scenario, the sendBeacon requests are not sent. I have tested this several times with the exact example you provided, and in all cases, the sendBeacon requests fail to send when the browser is fully closed.

Could you please test the example again and ensure that the browser is completely closed?

Comment 3

[edgul]

As you suggested, I tried again and the sendBeacon indeed goes out when the window closed is not the last window open.

If it is the last window the sendBeacon does not go out. However, there is significant discussion online about how in this scenario the pagehide event is not reliable. Neither the html spec nor the mdn make an explicit mention of this case. So it doesn't appear safe to assume it will be reliable. So I think this is your issue. FWIW, others online are talking about using visibilitychange event, maybe that provides a workaround?

That said, I am not an HTML expert and in case there is an issue here I'll pass to DOM team for further handling.

Comment 4

[Sean Feng [:sefeng211]]

Yeah, what Edgul said is correct. In addition to that, there's an effort to make it better with https://github.com/whatwg/fetch/pull/1647.

So yeah, I don't think we want to spend efforts to improve the pagehide case, and try visibilitychange event and see if it's better?

I'll just close this bug given the current state.

Comment 5

[Thorsten Dencker]

We had this discussion with the Chromium team (Google Chrome) and they fixed the issue for good reasons:

https://chromeenterprise.google/intl/en/policies/#FetchKeepaliveDurationSecondsOnShutdown
https://bugs.chromium.org/p/chromium/issues/detail?id=1035327#c21

The scenario is not about sending analytical data but about closing the session or application and releasing locks on business objects. Handling this during the visibilitychange event is not feasible, as it would inadvertently close the application simply because the user switches tabs. This has a significant impact on business users, as they cannot continue their processes if the objects remain locked.

Can we already test the concept behind https://github.com/whatwg/fetch/pull/1647.

Comment 6

[Sean Feng [:sefeng211]]

ah okay, thanks for the info!

I'll talk with the team to see what we can do here.

Comment 7

[Sean Feng [:sefeng211]]

So to summarize what could be the cause here...

The client JS had a sendBeacon call in the event handler for pageHide event, which supposed to do a network request. During the browser shutdown phase, I think there are two possibilities for it to not sent. It could be either the content process shutdown too fast, so that the IPC message for doing this network request didn't get send, or the parent process shutdown too fast, so the actual HTTP request wasn't sent.

If my understanding is correct, we need to delay the shutdown for both processes if we want to hack it. By discussing this internally, I see we can introduce a pref, so that SAP can ask their users to change this, but this requires manual intervention from users.

Or maybe a list of urls in a pref, so that we delay the shutdown when the url matches, and we can set it for SAP.

Bas, Olli, thoughts?

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

Of course one can't ever guarantee sendBeacon gets through - there might be network issues or browser or OR might crash etc.

We could add something similar what Chrome has.
Currently we seem to start the beacon but then cancel it
https://share.firefox.dev/3CJUcGU (that is a shutdown performance profile)
Necko certainly is still up and running. See pagehide in the file:// process and then https://mozilla.pettay.fi/ network request in the parent.
My testcase has navigator.sendBeacon("https://mozilla.pettay.fi");

Comment 9

[Sean Feng [:sefeng211]]

ah thanks! Looks like it's got aborted early at https://searchfox.org/mozilla-central/rev/d4235354139ae0da3b15b89943b140b510a74f2f/netwerk/protocol/http/nsHttpTransaction.cpp#220-222, which is something we landed in 120. It doesn't match exactly to Firefox 125, but it's something relatively recent, so I think it makes sense now.

Sunil, do you think there's a way that we can specialize the beacon request to allow it to be sent during shutdown? Or thoughts?

Comment 10

[Sean Feng [:sefeng211]]

I guess at least we could introduce a pref, and disable this check if the request matches certain URLs

Comment 11

[Sunil Mayya]

(In reply to Sean Feng [:sefeng] from comment #9)

ah thanks! Looks like it's got aborted early at https://searchfox.org/mozilla-central/rev/d4235354139ae0da3b15b89943b140b510a74f2f/netwerk/protocol/http/nsHttpTransaction.cpp#220-222, which is something we landed in 120. It doesn't match exactly to Firefox 125, but it's something relatively recent, so I think it makes sense now.

Sunil, do you think there's a way that we can specialize the beacon request to allow it to be sent during shutdown? Or thoughts?

Hey Sean, we introduced the check at nsHttpTransaction to prevent shutdown hangs during browser closure. If we disable this check for send-beacon this is not always guaranteed to work. A permanent reliable solution is to delay the browser shutdown for X seconds whenever we have pending keepalive requests similar to chrome. This might need some efforts and we need to decide if need to invest in this or implement fetch later (deffered fetch).

Olli, do you know why we are cancelling the request here. Could it be because of the following code:

  // The channel needs to have a loadgroup associated with it, so that we can
  // cancel the channel and any redirected channels it may create.
  nsCOMPtr<nsILoadGroup> loadGroup = do_CreateInstance(NS_LOADGROUP_CONTRACTID);
  nsCOMPtr<nsIInterfaceRequestor> callbacks =
      do_QueryInterface(mWindow->GetDocShell());
  loadGroup->SetNotificationCallbacks(callbacks);
  channel->SetLoadGroup(loadGroup);

If we dont set the notification callbacks, do we prevent the channel from being cancelled?
If yes, do you see any issue if we dont set the notification callbacks from the docshell?

Comment 12

[Sean Feng [:sefeng211]]

Just my two cents.

I think our solution doesn't have to be perfect, I know disabling it for sendBeacon is not always guaranteed to work but it should be like rolling back to what we had for years and it sounds like (according to the bug report), it seems to have been working well. So perhaps this is still the most feasible solution...

Comment 13

[Olli Pettay [:smaug][[e-mail]]]

(getting to this very very late). I don't see how setting SetNotificationCallbacks would cause the request being cancelled.
Isn't that sean mentioned the reason for this?

Comment 14

[Valentin Gosu [:valentin]]

Attachment: Bug 1931956 - Allow beacon transactions during shutdown r=#necko

Comment 15

[Valentin Gosu [:valentin]]

Attachment: Bug 1931956 - Add test for sendbeacon during shutdown r=#necko

Comment 16

[Valentin Gosu [:valentin]]

Hopefully this doesn't cause any shutdown hangs.

Comment 17

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1858238

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/f5cda475084e
https://hg.mozilla.org/integration/autoland/rev/4a3a0d4015e0
Allow beacon transactions during shutdown r=necko-reviewers,kershaw
https://github.com/mozilla-firefox/firefox/commit/83428e1040ff
https://hg.mozilla.org/integration/autoland/rev/0ad103360d17
Add test for sendbeacon during shutdown r=necko-reviewers,kershaw

Comment 19

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/4a3a0d4015e0
https://hg.mozilla.org/mozilla-central/rev/0ad103360d17

Comment 20

[BugBot [:suhaib / :marco]]

The patch landed in nightly and beta is affected.
:valentin, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
  • See https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift for documentation on how to request an uplift.
• If no, please set status-firefox144 to wontfix.

For more information, please visit BugBot documentation.

Comment 21

[Valentin Gosu [:valentin]]

I'd prefer to let this ride the trains.

Comment 22

[Valentin Gosu [:valentin]]

Attachment: Bug 1931956 - Send shutdown beacon on a warmed up connection to avoid intermittent failures r=#necko

Comment 23

[BugBot [:suhaib / :marco]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 24

[Phabricator Automation]

Comment on attachment 9642030 [details]
Bug 1931956 - Send shutdown beacon on a warmed up connection to avoid intermittent failures r=#necko

Revision D325905 was moved to bug 1989564. Setting attachment 9642030 [details] to obsolete.