1722336 - Intermittent Linux webrender asan /webdriver/tests/bidi/new_session/connect.py | test_websocket_url_connect[capabilities0] - KeyError: 'webSocketUrl' - when Gecko 92 merges to Beta on 2021-08-09

Status: RESOLVED DUPLICATE of bug 1629244

(Testing :: Marionette Client and Harness, defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

+++ This bug was initially created as a clone of Bug #1721012 +++

• Central-as-beta-simulation
• How to run these simulations
• Failure log

[task 2021-07-26T14:57:19.606Z] 14:57:19     INFO - STDOUT: :553: SessionNotCreatedException
[task 2021-07-26T14:57:19.607Z] 14:57:19     INFO - STDOUT: =================================== FAILURES ===================================
[task 2021-07-26T14:57:19.607Z] 14:57:19     INFO - STDOUT: __________________ test_websocket_url_connect[capabilities0] ___________________
[task 2021-07-26T14:57:19.608Z] 14:57:19     INFO - STDOUT: session = <Session 27f57b9a-3f14-44c3-95f9-b7a9e0fa492b>
[task 2021-07-26T14:57:19.608Z] 14:57:19     INFO - STDOUT:     @pytest.mark.asyncio
[task 2021-07-26T14:57:19.609Z] 14:57:19     INFO - STDOUT:     @pytest.mark.capabilities({"webSocketUrl": True})
[task 2021-07-26T14:57:19.609Z] 14:57:19     INFO - STDOUT:     async def test_websocket_url_connect(session):
[task 2021-07-26T14:57:19.610Z] 14:57:19     INFO - STDOUT: >       websocket_url = session.capabilities["webSocketUrl"]
[task 2021-07-26T14:57:19.610Z] 14:57:19     INFO - STDOUT: E       KeyError: 'webSocketUrl'
[task 2021-07-26T14:57:19.611Z] 14:57:19     INFO - STDOUT: session    = <Session 27f57b9a-3f14-44c3-95f9-b7a9e0fa492b>
[task 2021-07-26T14:57:19.611Z] 14:57:19     INFO - STDOUT: tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py
[task 2021-07-26T14:57:19.612Z] 14:57:19     INFO - STDOUT: :11: KeyError
[task 2021-07-26T14:57:19.612Z] 14:57:19     INFO - STDOUT: =========================== short test summary info ============================
[task 2021-07-26T14:57:19.624Z] 14:57:19     INFO - ....
[task 2021-07-26T14:57:19.624Z] 14:57:19     INFO - TEST-OK | /webdriver/tests/bidi/new_session/connect.py | took 43817ms
[task 2021-07-26T14:57:19.624Z] 14:57:19     INFO - Restarting browser for new test group
[task 2021-07-26T14:57:19.697Z] 14:57:19     INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py::test_websocket_url_connect[capabilities0]
[task 2021-07-26T14:57:19.697Z] 14:57:19     INFO - STDOUT: ERROR tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_send
[task 2021-07-26T14:57:19.697Z] 14:57:19     INFO - STDOUT: ERROR tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_with_different_capability[capabilities0]
[task 2021-07-26T14:57:19.697Z] 14:57:19     INFO - STDOUT: ==================== 1 failed, 1 passed, 2 errors in 43.60s ====================
[task 2021-07-26T14:57:19.698Z] 14:57:19     INFO - ERROR | LeakSanitizer | SUMMARY: AddressSanitizer: 32646 byte(s) leaked in 309 allocation(s).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Aryx, isn't that failure just about the leak? It's expected that the webSocketUrl is not available on beta and the expectation data is set appropriately:

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/webdriver/tests/bidi/new_session/connect.py.ini

Do we know when this has been started? Has there been ran a backfill yet?

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ab524816237c246a054ef8cfb65937fe3315efa5&tochange=0c272222c17b3edd7190a24d7171c51eb2f009ba
Will check if the issue persists tomorrow.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I had a look at the Treeherder jobs for staging releases, and I cannot find the build jobs that are used to run the tests. Do you know where the builds are coming from? I would like to try with an ASAN build locally in how to get it reproduced.

Also why is it intermittent now? Do you have a query for Treeherder?

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Failure log > Task > left side > See More > Dependencies > click on the build > either click on the Treeherder at the top or the artifacts list on the right.

For the log in comment 0, that's https://firefox-ci-tc.services.mozilla.com/tasks/K5_etDSMTFmr3ZktTcvsfA

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Aryx, is this still a problem? If yes aren't these failures classified?

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

No such failure yesterday, but Wednesday's central-as-beta simulation has two. Is this bug a duplicate of bug 1629244?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

When it's about the leak then probably yes. The KeyError is actually expected and as such marked in the manifest. Looks like that was an oversight when filing the bug.

Comment 8

[Intermittent Failures Robot]

1 failures in 3131 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1722336&startday=2021-07-26&endday=2021-08-01&tree=all

Comment 9

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.