1830616 - Intermittent Wd | webdriver.bidi.modules.script.ScriptEvaluateResultException: Script execution failed

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=414163280&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/D-N-CdM-RaetcPebSkT8bw/runs/0/artifacts/public/logs/live_backing.log

---

TEST-PASS | /webdriver/tests/bidi/network/response_completed/response_completed.py | test_response_status[204-No Content] 
[task 2023-04-29T10:54:09.833Z] 10:54:09     INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/bidi/network/response_completed/response_completed.py | test_response_status[205-Reset Content] - webdriver.bidi.modules.script.ScriptEvaluateResultException: Script execution failed.
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - bidi_session = <webdriver.bidi.client.BidiSession object at 0x000002FB62F07DF0>
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - wait_for_event = <function wait_for_event.<locals>.wait_for_event at 0x000002FB62F1A670>
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - url = <function url.<locals>.url at 0x000002FB62F1A280>
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - fetch = <function fetch.<locals>.fetch at 0x000002FB62F1A1F0>
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - setup_network_test = <function setup_network_test.<locals>._setup_network_test at 0x000002FB62E32700>
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - status = 205, status_text = 'Reset Content'
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO - 
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -     @pytest.mark.parametrize(
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -         "status, status_text",
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -         HTTP_STATUS_AND_STATUS_TEXT,
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -     )
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -     @pytest.mark.asyncio
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -     async def test_response_status(
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -         bidi_session, wait_for_event, url, fetch, setup_network_test, status, status_text
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -     ):
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -         status_url = url(
[task 2023-04-29T10:54:09.834Z] 10:54:09     INFO -             f"/webdriver/tests/support/http_handlers/status.py?status={status}&nocache={RESPONSE_COMPLETED_EVENT}"
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -         )
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -     
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -         network_events = await setup_network_test(events=[RESPONSE_COMPLETED_EVENT])
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -         events = network_events[RESPONSE_COMPLETED_EVENT]
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -     
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -         on_response_completed = wait_for_event(RESPONSE_COMPLETED_EVENT)
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - >       await fetch(status_url)
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - 
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - bidi_session = <webdriver.bidi.client.BidiSession object at 0x000002FB62F07DF0>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - events     = []
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - fetch      = <function fetch.<locals>.fetch at 0x000002FB62F1A1F0>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - network_events = {'network.responseCompleted': []}
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - on_response_completed = <Future pending>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - setup_network_test = <function setup_network_test.<locals>._setup_network_test at 0x000002FB62E32700>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - status     = 205
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - status_text = 'Reset Content'
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - status_url = 'http://web-platform.test:8000/webdriver/tests/support/http_handlers/status.py?status=205&nocache=network.responseCompleted'
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - url        = <function url.<locals>.url at 0x000002FB62F1A280>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO - wait_for_event = <function wait_for_event.<locals>.wait_for_event at 0x000002FB62F1A670>
[task 2023-04-29T10:54:09.835Z] 10:54:09     INFO -

Comment 1

[Intermittent Failures Robot]

1 failures in 3177 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-04-24&endday=2023-04-30&tree=all

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The failures here are coming from:

[task 2023-04-29T10:53:53.737Z] 10:53:53     INFO - PID 8760 | 1682765633735	RemoteAgent	DEBUG	WebDriverBiDiConnection ca05930e-2dfa-4962-9e2d-5d675c9c8582 -> {"id":61,"method":"script.evaluate","params":{"expression":"\n                 {\n                   const controller = new AbortController();\n                   setTimeout(() => controll ...   \n                     signal: controller.signal\n                   }).then(response => response.text());\n                 }","target":{"context":"5fe9e925-8f06-457a-87a8-3a7e0625d793"},"awaitPromise":true}}
[task 2023-04-29T10:53:53.738Z] 10:53:53     INFO - PID 8760 | 1682765633735	RemoteAgent	TRACE	Received command script.evaluate for destination ROOT
[task 2023-04-29T10:53:53.739Z] 10:53:53     INFO - PID 8760 | 1682765633736	RemoteAgent	TRACE	Received command script.evaluateExpression for destination WINDOW_GLOBAL
[task 2023-04-29T10:53:53.739Z] 10:53:53     INFO - PID 8760 | 1682765633736	RemoteAgent	TRACE	Module windowglobal/script.jsm found for WINDOW_GLOBAL
[task 2023-04-29T10:53:56.740Z] 10:53:56     INFO - PID 8760 | 1682765636746	RemoteAgent	DEBUG	WebDriverBiDiConnection ca05930e-2dfa-4962-9e2d-5d675c9c8582 <- {"id":61,"result":{"realm":"e889cd22-bab9-41b3-8f4e-715b55d2012c","type":"exception","exceptionDetails":{"columnNumber":22,"exception":{"type":"error"},"lineNumber":8,"stackTrace":{"callFrames":[{"columnNumber":22,"functionName":"","lineNumber":8,"url":"http://web-platform.test:8000/webdriver/tests/bidi/network/support/empty.html"}]},"text":"AbortError: The operation was aborted. "}}}

Julian, I thought we already had a bug about increasing the timeout for the fetch helper but I miss to find it. Do you know?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Actually I found bug 1829645.

Comment 5

[Intermittent Failures Robot]

5 failures in 3411 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• mozilla-central: 2

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1
• windows11-64-2009-qr: 2
  • opt: 2
• windows11-32-2009-shippable-qr: 1
  • opt: 1
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-05-01&endday=2023-05-07&tree=all

Comment 7

[Intermittent Failures Robot]

9 failures in 4077 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• mozilla-beta: 1
• try: 3
• mozilla-central: 2

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 4
  • opt: 4
• windows11-64-2009-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-05-08&endday=2023-05-14&tree=all

Comment 8

[Intermittent Failures Robot]

13 failures in 3578 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4
• mozilla-central: 5
• try: 2
• mozilla-beta: 2

Platform and build breakdown:

• windows11-32-2009-qr: 2
  • opt: 1
  • debug: 1
• android-em-7-0-x86_64-lite-qr: 1
  • opt: 1
• windows11-64-2009-qr: 3
  • opt: 2
  • debug: 1
• windows11-32-2009-shippable-qr: 5
  • opt: 5
• windows11-64-2009-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-05-15&endday=2023-05-21&tree=all

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Might be related to bug 1832891 as well.

Comment 11

[Intermittent Failures Robot]

11 failures in 3946 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-beta: 5
• try: 4

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • opt: 1
• windows11-64-2009-devedition-qr: 4
  • opt: 4
• windows11-32-2009-shippable-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-05-22&endday=2023-05-28&tree=all

Comment 13

[Intermittent Failures Robot]

10 failures in 3471 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1
• mozilla-central: 1
• autoland: 7
• try: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• windows11-32-2009-qr: 4
  • opt: 4
• windows11-64-2009-qr: 3
  • debug: 2
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-05-29&endday=2023-06-04&tree=all

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So the number of failures have definitely decreased but with bug 1832891 we don't have a complete fix because it only handles cases when we open a new context. You can see that here:
https://treeherder.mozilla.org/logviewer?job_id=418531085&repo=autoland&lineNumber=7702-7706

The problem is that we fail on the very first wait for navigation completed when the WebDriver session gets started:
https://searchfox.org/mozilla-central/rev/2e06f92ba068e32a9a7213ee726e8171f91605c7/remote/marionette/driver.sys.mjs#467-469

(In reply to Julian Descottes [:jdescottes] from bug 1832891 comment #22)

Something which would be useful would be a way to know if a document with isInitialDocument set to true is going to navigate or not. Today we don't know for sure so we have to use the timeout. But if we could know for sure when there will be a load, we could decide whether we wait for a navigation or not and completely remove the timeout.

That comment from the other bug is still interesting for me. I really wonder if there is a way to find out if the page stays on the initial about:blank or if it is going to load a different page afterward. Olli, is there any way to find this out?

Comment 15

[Olli Pettay [:smaug][[e-mail]]]

There is no way to know that. window.open() (no params) may not load anything, as an example.
Or someone may load something to it later using relevant window's location object or so.

Comment 16

[Intermittent Failures Robot]

8 failures in 3885 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 5
• mozilla-beta: 1

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 4
  • opt: 4
• windows11-64-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-06-05&endday=2023-06-11&tree=all

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #15)

There is no way to know that. window.open() (no params) may not load anything, as an example.
Or someone may load something to it later using relevant window's location object or so.

Lets move this discussion to bug 1838381 to have it on the right place.

Comment 18

[Intermittent Failures Robot]

8 failures in 4079 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-release: 4
• mozilla-central: 1
• try: 2

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 5
  • opt: 5
• windows11-64-2009-qr: 1
  • debug: 1
• windows11-64-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-06-12&endday=2023-06-18&tree=all

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+2]]

No more failures since my patch on bug 1838381 landed.

Comment 21

[Intermittent Failures Robot]

3 failures in 3880 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-beta: 2

Platform and build breakdown:

• windows11-64-2009-qr: 1
  • opt: 1
• windows11-64-2009-devedition-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-06-19&endday=2023-06-25&tree=all

Comment 22

[Intermittent Failures Robot]

3 failures in 3520 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 3

Platform and build breakdown:

• windows11-64-2009-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-07-17&endday=2023-07-23&tree=all

Comment 23

[Intermittent Failures Robot]

1 failures in 3403 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-07-24&endday=2023-07-30&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 3076 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-08-14&endday=2023-08-20&tree=all

Comment 25

[Intermittent Failures Robot]

1 failures in 3671 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2023-12-04&endday=2023-12-10&tree=all

Comment 26

[Intermittent Failures Robot]

1 failures in 3791 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2024-03-18&endday=2024-03-24&tree=all

Comment 27

[Intermittent Failures Robot]

1 failures in 3805 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2024-04-01&endday=2024-04-07&tree=all

Comment 28

[Intermittent Failures Robot]

1 failures in 4336 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2024-04-22&endday=2024-04-28&tree=all

Comment 29

[Intermittent Failures Robot]

1 failures in 3649 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830616&startday=2024-05-06&endday=2024-05-12&tree=all