Open Bug 1856368 Opened 1 year ago Updated 10 months ago

Perma dom/media/tests/crashtests/1490700.html failure on new Ubuntu 22.04 Wayland pool

Categories

(Core :: WebRTC, defect, P3)

Product:
Core
Component:
WebRTC
Type:
defect

Tracking

()

People

(Reporter: ahal, Unassigned)

References

(Depends on 2 open bugs, Blocks 2 open bugs)

Details

This failure happens on a new platform we're standing up to get Wayland test coverage going. Here is an example log with failure:
https://treeherder.mozilla.org/logviewer?job_id=430279220&repo=try&lineNumber=8171

[task 2023-09-25T23:46:13.191Z] 23:46:13     INFO - REFTEST TEST-START | dom/media/tests/crashtests/1490700.html
[task 2023-09-25T23:46:13.194Z] 23:46:13     INFO - REFTEST INFO | RESTORE PREFERENCE pref(media.getusermedia.insecure.enabled,false)
[task 2023-09-25T23:46:13.197Z] 23:46:13     INFO - REFTEST INFO | RESTORE PREFERENCE pref(media.devices.insecure.enabled,false)
[task 2023-09-25T23:46:13.200Z] 23:46:13     INFO - REFTEST INFO | RESTORE PREFERENCE pref(dom.disable_open_during_load,true)
[task 2023-09-25T23:46:13.202Z] 23:46:13     INFO - REFTEST INFO | RESTORE PREFERENCE pref(media.navigator.permission.disabled,false)
[task 2023-09-25T23:46:13.204Z] 23:46:13     INFO - REFTEST INFO | SET PREFERENCE pref(media.navigator.permission.disabled,true)
[task 2023-09-25T23:46:13.207Z] 23:46:13     INFO - REFTEST INFO | SET PREFERENCE pref(dom.disable_open_during_load,false)
[task 2023-09-25T23:46:13.210Z] 23:46:13     INFO - REFTEST INFO | SET PREFERENCE pref(media.devices.insecure.enabled,true)
[task 2023-09-25T23:46:13.212Z] 23:46:13     INFO - REFTEST INFO | SET PREFERENCE pref(media.getusermedia.insecure.enabled,true)
[task 2023-09-25T23:46:13.226Z] 23:46:13     INFO - REFTEST TEST-LOAD | file:///home/ubuntu/tasks/task_169568508287826/build/tests/reftest/tests/dom/media/tests/crashtests/1490700.html | 810 / 4008 (20%)
[task 2023-09-25T23:46:13.358Z] 23:46:13     INFO - JavaScript error: , line 0: NotFoundError: The object can not be found here.
[task 2023-09-25T23:46:13.596Z] 23:46:13     INFO - Timecard created 1695685570.862268
[task 2023-09-25T23:46:13.598Z] 23:46:13     INFO -  Timestamp   | Delta       | Event                                  | File                         | Function
[task 2023-09-25T23:46:13.598Z] 23:46:13     INFO - =========================================================================================================================
[task 2023-09-25T23:46:13.599Z] 23:46:13     INFO -     0.000616 |    0.000616 | Constructor Completed                  | PeerConnectionImpl.cpp:389   | PeerConnectionImpl
[task 2023-09-25T23:46:13.601Z] 23:46:13     INFO -     0.001148 |    0.000532 | Initializing PC Ctx                    | PeerConnectionImpl.cpp:474   | Initialize
[task 2023-09-25T23:46:13.601Z] 23:46:13     INFO -     1.716539 |    1.715391 | Close                                  | PeerConnectionImpl.cpp:2517  | Close
[task 2023-09-25T23:46:13.602Z] 23:46:13     INFO -     2.731820 |    1.015281 | Destructor Invoked                     | PeerConnectionImpl.cpp:406   | ~PeerConnectionImpl
[task 2023-09-25T23:46:13.603Z] 23:46:13     INFO -     2.731820 |    0.000000 | {076460ab-3048-43c8-8292-08e39941d24b} | PeerConnectionImpl.cpp:407   | ~PeerConnectionImpl
[task 2023-09-25T23:46:13.603Z] 23:46:13     INFO - [Child 3212: Main Thread]: I/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:414: ~PeerConnectionImpl: PeerConnectionImpl destructor invoked for {076460ab-3048-43c8-8292-08e39941d24b}
[task 2023-09-25T23:46:17.597Z] 23:46:17     INFO - [Child 3212: Socket Thread]: I/mtransport NrIceCtx(PC:{076460ab-3048-43c8-8292-08e39941d24b} 1695685570863404 (id=17179871369 url=file:///home/ubuntu/tasks/task_169568508287826/build/tests/reftest/tests/dom/media/tests/crashtests/1453030.html)): Destroy
[task 2023-09-25T23:47:17.757Z] 23:47:17     INFO - Timecard created 1695685572.592950
[task 2023-09-25T23:47:17.759Z] 23:47:17     INFO -  Timestamp   | Delta       | Event                                  | File                         | Function
[task 2023-09-25T23:47:17.759Z] 23:47:17     INFO - =========================================================================================================================
[task 2023-09-25T23:47:17.761Z] 23:47:17     INFO -     0.000340 |    0.000340 | Constructor Completed                  | PeerConnectionImpl.cpp:389   | PeerConnectionImpl
[task 2023-09-25T23:47:17.761Z] 23:47:17     INFO -     0.000580 |    0.000240 | Initializing PC Ctx                    | PeerConnectionImpl.cpp:474   | Initialize
[task 2023-09-25T23:47:17.762Z] 23:47:17     INFO -     0.058170 |    0.057590 | Close                                  | PeerConnectionImpl.cpp:2517  | Close
[task 2023-09-25T23:47:17.763Z] 23:47:17     INFO -    65.158836 |   65.100666 | Destructor Invoked                     | PeerConnectionImpl.cpp:406   | ~PeerConnectionImpl
[task 2023-09-25T23:47:17.763Z] 23:47:17     INFO -    65.158839 |    0.000003 | {2135276a-5b48-4dc2-93a6-83044408d4a4} | PeerConnectionImpl.cpp:407   | ~PeerConnectionImpl
[task 2023-09-25T23:47:17.764Z] 23:47:17     INFO - [Child 3212: Main Thread]: I/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:414: ~PeerConnectionImpl: PeerConnectionImpl destructor invoked for {2135276a-5b48-4dc2-93a6-83044408d4a4}
[task 2023-09-25T23:47:17.766Z] 23:47:17     INFO - Timecard created 1695685572.587042
[task 2023-09-25T23:47:17.766Z] 23:47:17     INFO -  Timestamp   | Delta       | Event                                  | File                         | Function
[task 2023-09-25T23:47:17.767Z] 23:47:17     INFO - =========================================================================================================================
[task 2023-09-25T23:47:17.767Z] 23:47:17     INFO -     0.000093 |    0.000093 | Constructor Completed                  | PeerConnectionImpl.cpp:389   | PeerConnectionImpl
[task 2023-09-25T23:47:17.768Z] 23:47:17     INFO -     0.000316 |    0.000223 | Initializing PC Ctx                    | PeerConnectionImpl.cpp:474   | Initialize
[task 2023-09-25T23:47:17.769Z] 23:47:17     INFO -     0.036351 |    0.036035 | Create Offer                           | PeerConnectionImpl.cpp:1539  | CreateOffer
[task 2023-09-25T23:47:17.770Z] 23:47:17     INFO -     0.047744 |    0.011393 | Create Offer                           | PeerConnectionImpl.cpp:1539  | CreateOffer
[task 2023-09-25T23:47:17.770Z] 23:47:17     INFO -     0.583397 |    0.535653 | Close                                  | PeerConnectionImpl.cpp:2517  | Close
[task 2023-09-25T23:47:17.771Z] 23:47:17     INFO -    65.168600 |   64.585203 | Destructor Invoked                     | PeerConnectionImpl.cpp:406   | ~PeerConnectionImpl
[task 2023-09-25T23:47:17.772Z] 23:47:17     INFO -    65.168601 |    0.000001 | {04701b20-ad0d-45a9-8e6f-f9bfbe477a7d} | PeerConnectionImpl.cpp:407   | ~PeerConnectionImpl
[task 2023-09-25T23:47:17.773Z] 23:47:17     INFO - [Child 3212: Main Thread]: I/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:414: ~PeerConnectionImpl: PeerConnectionImpl destructor invoked for {04701b20-ad0d-45a9-8e6f-f9bfbe477a7d}
[task 2023-09-25T23:47:17.773Z] 23:47:17     INFO - [Child 3212: Socket Thread]: I/mtransport NrIceCtx(PC:{04701b20-ad0d-45a9-8e6f-f9bfbe477a7d} 1695685572587347 (id=17179871374 url=file:///home/ubuntu/tasks/task_169568508287826/build/tests/reftest/tests/dom/media/tests/crashtests/1468451.html)): Destroy

It's unclear whether this is a result of using Wayland, a result of upgrading to Ubuntu 22.04 or a result of a race condition due to new worker characteristics.

The test will be skipped on this new platform.

Since this is wayland I would assume we're hanging waiting for the pipewire screensharing prompt to be approved/rejected. Is it expected that there was no screenshot taken on this failure?

Flags: needinfo?(ahal)

Good question, I'm not sure. Do crashtests normally have screenshots? If there was a crash, maybe it's no longer possible?

I also wouldn't be shocked if there's some screenshot utility missing on the worker, though IIRC we got screenshots working in Mochitest for these workers.

Flags: needinfo?(ahal)
Blocks: wayland-tests, pipewire
Severity: -- → S4
Depends on: 1747206
Priority: -- → P3

(In reply to Andreas Pehrson [:pehrsons] from comment #1)

Since this is wayland I would assume we're hanging waiting for the pipewire screensharing prompt to be approved/rejected. Is it expected that there was no screenshot taken on this failure?

Yes, that's expected. Note that this is the same for XWayland which is used right now on Ubuntu.

We need to create modified xdg-desktop-portal-gnome-44.2-1.fc38.x86_64/xdg-desktop-portal-gtk-1.14.1-2.fc38.x86_64 packages which will automatically allow screen sharing/window sharing for tests. For instance we can modify the package to allow screensharing if a particular env variable is set (and also test screensharing deny). Do you want me to create such package? You'd need to install it to testsuite environment then.

Or we can hack the environment somehow to click on screen sharing prompt (which may be difficult and I don't know how to do it).

Flags: needinfo?(ahal)

I see Bug 1747206 was created for it.

I don't think we'd want to implement clicking on prompts from the test harness level, and I'm not sure how we'd do it even if we did :/

That said, I think crashtests just don't take screenshots? Here's a non-Wayland failing crashtest:
https://firefox-ci-tc.services.mozilla.com/tasks/OHj6StrsS3a6Q8PC7IElWA

Note there are no screenshot artifacts. And here's a failing Wayland mochitest-plain task:
https://firefox-ci-tc.services.mozilla.com/tasks/IYdPeHbwRCar4rbtvFDeRw

It took a screenshot just fine.

Flags: needinfo?(ahal)

Ah I see. My comments are not related to this bug - this is not cause by pipewire/screensharing. Yes screenshots can be taken fine on Wayland.

We use /usr/bin/gnome-screenshot binary to get screenshot on Wayland:
https://searchfox.org/mozilla-central/source/testing/tools/screenshot/gdk-screenshot.cpp

No longer depends on: 1747206

The discussion on screenshots is just here as it would have been a way to diagnose the issue with this failing crashtest. The main hypothesis from my side is still that we're hanging on a portal prompt, or in case the NotFoundError is from getDisplayMedia -- that something in the setup or the backend is not working as it should.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #3)

We need to create modified xdg-desktop-portal-gnome-44.2-1.fc38.x86_64/xdg-desktop-portal-gtk-1.14.1-2.fc38.x86_64 packages which will automatically allow screen sharing/window sharing for tests. For instance we can modify the package to allow screensharing if a particular env variable is set (and also test screensharing deny). Do you want me to create such package? You'd need to install it to testsuite environment then.

This SGTM, we'll of course want screen capture coverage on wayland.

(In reply to Andrew Halberstadt [:ahal] from comment #5)

I don't think we'd want to implement clicking on prompts from the test harness level, and I'm not sure how we'd do it even if we did :/

There's no clicking with a modified portal AIUI. Clicking was the hacky approach.

Depends on: 1747206

re screenshots in crashtests. The reftest harness times out the test and moves on before the time out detection that happens at a higher level can kick in, which is the code that takes a screenshot. If you change the reftest harness to not move foward to the next test (say by increasing it's internal timeout) then the higher level timeout detection works and a screenshot is taken. More discussion in bug 1892364.

See Also: → 1892364
You need to log in before you can comment on or make changes to this bug.