1878934 - Intermittent /webdriver/tests/bidi/network/continue_response/invalid.py | single tracking bug

Status: REOPENED

(Remote Protocol :: WebDriver BiDi, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: pstanciu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=446151296&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CCPBOFudRNKgrwPZnaC2xw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2024-02-06T14:34:57.284Z] 14:34:57     INFO - PID 16135 | 1707230097282	WebDriver BiDi	DEBUG	Connection bfbc4810-2bbe-4ae2-b53c-70b33e710985 attached to session c0ea3582-54e4-44cc-812c-dd991d7c729e
[task 2024-02-06T14:34:57.285Z] 14:34:57     INFO - PID 16135 | 1707230097283	webdriver::server	DEBUG	-> GET /session/c0ea3582-54e4-44cc-812c-dd991d7c729e/window/rect
[task 2024-02-06T14:34:57.285Z] 14:34:57     INFO - PID 16135 | 1707230097283	Marionette	DEBUG	0 -> [0,2131,"WebDriver:GetWindowRect",{}]
[task 2024-02-06T14:34:57.288Z] 14:34:57     INFO - TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/bidi/network/continue_response/invalid.py | expected OK
[task 2024-02-06T14:34:57.288Z] 14:34:57     INFO - TEST-INFO took 30003ms
[task 2024-02-06T14:34:57.298Z] 14:34:57     INFO - PID 16135 | 1707230097285	Marionette	DEBUG	0 <- [1,2131,null,{"x":100,"y":100,"width":800,"height":600}]
[task 2024-02-06T14:34:57.299Z] 14:34:57     INFO - PID 16135 | 1707230097285	webdriver::server	DEBUG	<- 200 OK {"value":{"x":100,"y":100,"width":800,"height":600}}
[task 2024-02-06T14:34:57.555Z] 14:34:57     INFO - Test harness output was not a valid structured log message
[task 2024-02-06T14:34:57.555Z] 14:34:57     INFO - Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x7f520e0dd740>
[task 2024-02-06T14:34:57.556Z] 14:34:57     INFO - Traceback (most recent call last):
[task 2024-02-06T14:34:57.556Z] 14:34:57     INFO -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/third_party/websockets/src/websockets/legacy/protocol.py", line 1329, in close_connection
[task 2024-02-06T14:34:57.556Z] 14:34:57     INFO -     await self.close_transport()
[task 2024-02-06T14:34:57.556Z] 14:34:57     INFO -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/third_party/websockets/src/websockets/legacy/protocol.py", line 1345, in close_transport
[task 2024-02-06T14:34:57.557Z] 14:34:57     INFO -     self.transport.close()
[task 2024-02-06T14:34:57.557Z] 14:34:57     INFO -   File "/usr/lib/python3.8/asyncio/selector_events.py", line 680, in close
[task 2024-02-06T14:34:57.557Z] 14:34:57     INFO -     self._loop.call_soon(self._call_connection_lost, None)
[task 2024-02-06T14:34:57.557Z] 14:34:57     INFO -   File "/usr/lib/python3.8/asyncio/base_events.py", line 711, in call_soon
[task 2024-02-06T14:34:57.558Z] 14:34:57     INFO -     self._check_closed()
[task 2024-02-06T14:34:57.558Z] 14:34:57     INFO -   File "/usr/lib/python3.8/asyncio/base_events.py", line 504, in _check_closed
[task 2024-02-06T14:34:57.558Z] 14:34:57     INFO -     raise RuntimeError('Event loop is closed')
[task 2024-02-06T14:34:57.558Z] 14:34:57     INFO - RuntimeError: Event loop is closed
[task 2024-02-06T14:34:57.638Z] 14:34:57     INFO - Closing logging queue
[task 2024-02-06T14:34:57.638Z] 14:34:57     INFO - queue closed
[task 2024-02-06T14:34:57.750Z] 14:34:57     INFO - PID 17017 | 1707230097648	geckodriver	INFO	Listening on 127.0.0.1:37177
[task 2024-02-06T14:34:57.750Z] 14:34:57     INFO - Starting runner
[task 2024-02-06T14:34:58.012Z] 14:34:58     INFO - TEST-START | /webdriver/tests/bidi/network/continue_response/request.py

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Running all the tests in this test module just took all the time, and no delay is noticeable for startup. Most likely the additionally added tests from bug 1874206 caused this problem. Also [Chrome times out on these](https://wpt.fyi/results/webdriver/tests/bidi/network/continue_response/invalid.py?label=master&label=experimental&product=firefox&product=firefox_android&product=chrome&product=chrome_android&product=safari&aligned&view=subtest.

Julian, I assume that we have to extend the timeout for this particular test. For that lets file a new bug that we can also mark as regression if that's the case. Thanks.

Comment 2

[Julian Descottes [:jdescottes]]

Yes that sounds correct

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

With bug 1879006 fixed this should no longer be an issue.

Comment 4

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=446424012&repo=autoland

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Treeherder Bug Filer from comment #4)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=446424012&repo=autoland

That's a different issue. Here the test did even not start. Maybe some hang in wpt-runner.

Comment 6

[Julian Descottes [:jdescottes]]

I think the new failure is similar to Bug 1875679

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh, actually this is Android and we should check the adb logcat! So here is the real problem:

02-07 23:00:00.186 20174 20189 I Gecko   : 1707346800186	RemoteAgent	DEBUG	WebDriverBiDiConnection 75b7d506-8d69-4e5d-b1dc-8ba92ab04479 <- {"type":"success","id":17,"result":{"realm":"774c871c-b71f-4ad1-8017-2d6da254b507","type":"exception","exceptionDetails":{"columnNumber":22,"exception":{"type":"error"},"lineNumber":8,"stackTrace":{"callFrames":[{"columnNumber":22,"functionName":"","lineNumber":8,"url":"https://web-platform.test:8443/webdriver/tests/bidi/network/support/empty.html"}]},"text":"AbortError: The operation was aborted. "}}}
[..]
02-07 23:00:02.639 20342 20342 E hostapd : Could not read interface wlan1 flags: No such device

Here we most likely have a network error which first caused an AbortError for the fetch fixture and then shows a lot of wlan issues until Firefox gets killed. This would explain why we have missing entries in the normal error log file. Overall I expect this to happen kinda rarely.

Comment 8

[Intermittent Failures Robot]

7 failures in 4009 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2
• autoland: 4
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-devedition-qr: 2
  • opt: 2
• linux1804-64-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 1
  • opt: 1
• android-em-7-0-x86_64-lite-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-02-05&endday=2024-02-11&tree=all

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As it looks like no further failures happened since the timeout has been extended.

Comment 10

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=446328882&repo=autoland

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Treeherder Bug Filer from comment #10)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=446328882&repo=autoland

That's a failure from an old job on Tuesday last week.

Comment 12

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=446328882&repo=autoland

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Lets try closing again - so that the bot is not re-opening the bug.

Comment 14

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=456446413&repo=mozilla-beta

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Treeherder Bug Filer from comment #14)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=456446413&repo=mozilla-beta

This is actually bug 1774392. I've re-classified.

Comment 16

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=456401480&repo=try

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Treeherder Bug Filer from comment #16)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=456401480&repo=try

The stack looks more like bug 1894213. I'll re-classify if that turns out to be true.

Comment 18

[Intermittent Failures Robot]

1 failures in 4059 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-04-29&endday=2024-05-05&tree=all

Comment 19

[Intermittent Failures Robot]

1 failures in 3649 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-06-03&endday=2024-06-09&tree=all

Comment 20

[Intermittent Failures Robot]

1 failures in 4577 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-06-17&endday=2024-06-23&tree=all

Comment 21

[Intermittent Failures Robot]

3 failures in 3199 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1
• mozilla-esr128: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 2
  • opt: 2
• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-07-01&endday=2024-07-07&tree=all

Comment 22

[Intermittent Failures Robot]

5 failures in 3799 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• mozilla-beta: 4

Platform and build breakdown:

• macosx1015-64-qr: 4
  • debug: 4
• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-07-15&endday=2024-07-21&tree=all

Comment 23

[Intermittent Failures Robot]

18 failures in 4124 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 16
• autoland: 1
• try: 1

Platform and build breakdown:

• macosx1015-64-qr: 17
  • debug: 17
• linux1804-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-07-22&endday=2024-07-28&tree=all

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+2]]

There are quite a lot of issues for beta. Julian, do you know what might have caused that?

Comment 25

[Julian Descottes [:jdescottes]]

This started spiking on the 20th, and early beta was removed on the 19th, so I'm guessing it has something to do with something flagged behind EARLY_BETA_OR_EARLIER, but what exactly I don't know.

Those tests are creating a lot of blocked requests, and it seems the root cause of the issue is

Parent 2793, Main Thread] ###!!! ABORT: failed to re-open freezeable shm: Too many open files: file /builds/worker/checkouts/gecko/ipc/chromium/src/base/shared_memory_posix.cc:345

I suspect the two are related. Maybe we could try to either: resume the blocked request after the invalid check, OR use dedicated tabs (I assume closing the tab will force to terminate the request?)

Comment 26

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Using dedicated tabs for each test sound like a good idea. Given that these tests are for invalid cases it sounds strange if we would do more than just the check.

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Should now be fixed by the landing of bug 1910555. Marking as incomplete given that it is a single tracking bug.

Comment 28

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=468359397&repo=mozilla-beta

Comment 29

[Julian Descottes [:jdescottes]]

(In reply to Treeherder Bug Filer from comment #28)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=468359397&repo=mozilla-beta

This failure is from July 29th, so before we landed the fix. However considering that this failure is on beta, and the fix only landed in nightly, I think we can wait until 130 becomes beta to close it again.

Comment 30

[Intermittent Failures Robot]

2 failures in 3839 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-07-29&endday=2024-08-04&tree=all

Comment 31

[Julian Descottes [:jdescottes]]

Beta is now 130, this should no longer occur.

Comment 32

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=468801269&repo=autoland

Comment 33

[Intermittent Failures Robot]

1 failures in 2154 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-08-12&endday=2024-08-18&tree=all

Comment 34

[Intermittent Failures Robot]

1 failures in 3879 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-08-26&endday=2024-09-01&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 3668 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr128: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-09-02&endday=2024-09-08&tree=all

Comment 36

[Intermittent Failures Robot]

4 failures in 4448 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform and build breakdown:

• android-em-7-0-x86_64-qr: 2
  • debug-isolated-process: 1
  • debug: 1
• android-em-7-0-x86_64-lite-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-09-09&endday=2024-09-15&tree=all

Comment 37

[Intermittent Failures Robot]

2 failures in 3574 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr128: 2

Platform and build breakdown:

• macosx1015-64-qr: 2
  • debug: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-09-30&endday=2024-10-06&tree=all

Comment 38

[Intermittent Failures Robot]

2 failures in 4579 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr128: 1
• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1
• android-em-7-0-x86_64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-10-14&endday=2024-10-20&tree=all

Comment 39

[Intermittent Failures Robot]

1 failures in 4456 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-10-21&endday=2024-10-27&tree=all

Comment 40

[Intermittent Failures Robot]

1 failures in 4456 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows11-32-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-10-21&endday=2024-10-27&tree=all

Comment 41

[Intermittent Failures Robot]

1 failures in 4515 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878934&startday=2024-11-04&endday=2024-11-10&tree=all