1670181 - With navigator.webdriver enabled bet365.com never finishes loading, causing 100% CPU load, and prevents Firefox from shutdown

Status: RESOLVED WORKSFORME

(Web Compatibility :: Site Reports, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As I got informed today via https://github.com/mozilla/geckodriver/issues/1787 there is a problem with Marionette after loading bet365.com. When I investigated that more I noticed that it's actually a problem with that page and Firefox in several ways:

1. Loading of the website never finishes because one of the embedded Javascript files (https://www.bet365.com/defaultapi/sports-configuration?&rt=1) can't be loaded due to no reply from the server.

2. This actually causes a 100% CPU load for the current tab, which is persistent.

3. It's not possible to close Firefox. Instead I have to force-kill the process.

Here a recorded profile: https://share.firefox.dev/33JXohQ

As you can see the webcontent process is blocked by a long running JS script as executed via setTimeout.

To reproduce just try to load that website in a fresh profile.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Maybe this is related to a long running GC? I can see one entry for GC Minor in the marker table that runs all the time.

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

I was just testing this on my machine (Nightly build, Win10) and I can't reproduce the problem. The page seems to be loaded fine and I am not experiencing any UI freezing. Perhaps Mac/Linux only issue?

Honza

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

There doesn't seem to be one entry for minor GC, but a lot of them.
Almost as if there was some endless loop to create new array object.
But one should still be able to close Firefox.

Comment 4

[Andrew McCreight [:mccr8]]

Jon is a better person to ask about the GC.

Comment 5

[Jon Coppeard (:jonco)]

As Olli says, this isn't a single GC but many short ones. It seems some JS is running that constantly creates garbage. The nursery has responded by resizing to its maximum of 16MB.

The profile comes from nightly, and I'm seeing lots of memset (well, _platform_bzero$VARIANT$Haswell) going on. This is caused by the GC poisoning the nursery to catch more bugs, and only happens on nightly. Maybe constantly writing to memory like this is causing the OS to block paging memory out or something like that.

I couldn't reproduce this on MacOS.

Comment 6

[Karl Dubost💡 :karlcow]

1. Does it happen on Chrome?
2. for https://www.bet365.com/defaultapi/sports-configuration?&rt=1, I get

HTTP/1.1 500 Internal Server Error
Date: Fri, 16 Oct 2020 04:09:59 GMT
Content-Length: 0

So this doesn't seem to be an issue anymore?

When loading https://www.bet365.com/
There is one request to https://www.bet365.com/defaultapi/sports-configuration

HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, private, must-revalidate, max-age=0
Content-Type: application/json; charset=utf-8
Expires: 0
Pragma: no-cache
Set-Cookie: pstk=E2A6A754D9FC4D8D9F5FDECF43B2C585000003; Path=/; Domain=.bet365.com; SameSite=None; Secure
rmbs=3; Path=/; Expires=Fri, 16 Apr 2021 04:12:22 GMT; SameSite=None; Secure
aps03=cf=N&cg=1&cst=0&ct=99&hd=N&lng=1&oty=2&tzi=28; Path=/; Expires=Wed, 16 Oct 2030 04:12:22 GMT; SameSite=None; Secure
Date: Fri, 16 Oct 2020 04:12:22 GMT
ntCoent-Length: 2032
Content-Encoding: gzip
Content-Length: 1098

Shall we close this?

Comment 7

[Alexei]

Hi, I'm the reporter of the original issue: https://github.com/mozilla/geckodriver/issues/1787

Nothing changed from what I can tell. This is still an issue in Firefox driven by geckodriver. Visiting bet365.com hangs the tab where attempting to navigate elsewhere hangs perpetually, not subject to any timeout.

In Chrome driven by ChromeDriver, we run into similar weird tab behavior where nothing finishes loading on bet365.com and the tab seems to get into a bad state. However, navigating somewhere else does produce a timeout exception, which lets you try to recover somehow (perhaps by restarting the browser).

Comment 8

[Alexei]

I personally can't reproduce this behavior in Firefox or Chrome without Selenium.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

No, we should not close this bug. It's valid and completely locks the browser when using Marionette / geckodriver. Not sure yet what exactly causes it.

To reproduce this behavior use the command mach run --marionette, and manually load bet365.com. Once the load has been started you can see that the browser locks up, and you can't even open devtools nor any other window.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Marionette test

Simple Marionette test to reproduce this problem is attached. Save it to your local disk and run with:

./mach marionette-test --gecko-log - -vv testcase.py

Shutdown of Firefox hangs during shutdown, and it will be killed after the 15 or so seconds with the following message:

WARNING: At least one completion condition is taking too long to complete. Conditions: [{"name":"ContentParent: id=1220c6000","state":{"remoteTypePrefix":"web"},"filename":"/builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp","lineNumber":3328,"stack":["resource://gre/modules/nsAsyncShutdown.jsm:addBlocker:162","chrome://global/content/elements/browser-custom-element.js:get loadContext:359","chrome://global/content/elements/browser-custom-element.js:construct:1062","chrome://global/content/elements/browser-custom-element.js:connectedCallback:297","chrome://browser/content/tabbrowser.js:_setupInitialBrowserAndTab:381","chrome://browser/content/tabbrowser.js:init:42","chrome://browser/content/browser.js:onDOMContentLoaded:1779"]}] Barrier: profile-before-change
JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 2303: TypeError: subject.QueryInterface is not a function

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

A debug build gives the following output:

[Parent 87469, Main Thread] ###!!! ABORT: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3328
#01: NS_InvokeByIndex[/Users/henrik/code/gecko/obj/debug/dist/NightlyDebug.app/Contents/MacOS/XUL +0x1c8c0e]
[Parent 87469, Main Thread] ###!!! ABORT: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3328
Hit MOZ_CRASH() at /builds/worker/checkouts/gecko/memory/mozalloc/mozalloc_abort.cpp:33
Exiting due to channel error.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Btw I pushed a try build with that test so that we can get a crash stack. Also we could see which platforms are affected by that.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a468bcd3db92888ff60522c3a3266e3d2dcaa0bf

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

All platforms are affected. Here the crash report on Linux:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=318860382&repo=try&lineNumber=4115

Jon, I assume this mostly warrants a new bug, which can be marked as blocking this one. But I still wonder what's causing the hang here means before trying to quit the browser. Are you able to reproduce it with the Marionette script as attached above?

Comment 15

[Karl Dubost💡 :karlcow]

At least, this doesn't seem to be a webcompat bug which requires outreach.
It seems to be something related to Firefox Core + Marionette itself, no?

Where should we move this issue?

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I did some more investigation today and it is indeed a site compat issue, and we should reach out. Here the explanation:

There is nothing specific to Marionette that causes this behavior. But the website itself checks for navigator.webdriver and if that property is true it seems to exercise a different execution path compared to that one you can see when just opening the page in your browser. That is something a site can do, but shouldn't cause the browser to freeze / hang.

Steps to reproduce:

1. Create a fresh profile and open Firefox
2. Change the preference marionette.enabled to true (it would enable Marionette after a restart)
3. Open https://bet365.com

After step 3 you can observe the hang and freeze of Firefox during shutdown.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Just note that the affected website is an online gaming (casino) site. So they might indeed be interested in to not being automated. But they how they do it is just wrong. Maybe a better way would be just to serve a clear web page that states that compared to completely locking up the browser.

Comment 18

[Oana Arbuzov [:oanaarbuzov]]

I was able to reproduce the issue and after Firefox freezes, the tab crashes.
https://prnt.sc/w43ljo

Tested with:
Browser / Version: Firefox Nightly 86.0a1 (2020-12-16)
Operating System: Windows 10 Pro

Comment 19

[Oana Arbuzov [:oanaarbuzov]]

I was not able to reproduce the issue on my side anymore.

• https://www.bet365.com/ - loads correctly, and no freezes/crashes occur.
  https://prnt.sc/19xfETqIyWaV
• https://www.bet365.com/defaultapi/sports-configuration?&rt=1 - loads correctly, and no freezes/crashes occur.
  https://prnt.sc/gvsCV4uY47TA

Tested with:
Browser / Version: Firefox Nightly 104.0a1 (2022-06-28)
Operating System: Windows 10 Pro

Henrik can you still reproduce it?

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Cloudflare intercepts the load now and shows the following output:

Access denied Error code 1020

You do not have access to bet365.com.

The site owner may have set restrictions that prevent you from accessing the site. Contact the site owner for access or try loading the page again.

That's indeed a way better behavior compared to what was happening before. So I think that we can indeed close this bug now.