1707735 - 14 tests skipped on apple_silicon

Status: REOPENED

(Core :: WebRTC, task, P3)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Last week we turned on tests for Apple Silicon (OSX 11.2.3 on new Apple based hardware). We are using the simplified new test config process:
https://firefox-source-docs.mozilla.org/testing/ci-configs/index.html

As the tests are live, we are now filing bugs to help close the loop and hope to fix any issues over the next 7 weeks. As the process outlines, there are tier-3 jobs running on m-c which run these skipped tests and will expect a failure/timeout/crash- if it doesn't fail, then the job will turn orange.

here are the failing tests:
browser/base/content/test/webrtc/browser_device_controls_menus.js
browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame.js
browser/base/content/test/webrtc/browser_devices_get_user_media_multi_process.js
browser/base/content/test/webrtc/browser_devices_get_user_media_paused.js
browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js
browser/base/content/test/webrtc/browser_devices_get_user_media_screen_tab_close.js
browser/base/content/test/webrtc/browser_indicator_popuphiding.js
browser/base/content/test/webrtc/browser_notification_silencing.js
browser/base/content/test/webrtc/browser_stop_sharing_button.js
browser/base/content/test/webrtc/browser_stop_streams_on_indicator_close.js

and more failing tests here:
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_in_xorigin_frame.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_multi_process.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_paused.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_screen.js

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:mjf - a heads up on these disabled tests- find a time to triage this and take a look and see if this is expected on the new osx platform or if we can fix it in anyway.

Comment 2

[[e-mail] (:bhearsum)]

browser_devices_get_user_media_tear_off_tab.js is another test from this set that fails on apple silicon.

https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=macosx1100&test_paths=browser%2Fbase%2Fcontent%2Ftest%2Fwebrtc&revision=eac40293649628d32911c909219d514bebb8ce63 has a very recent for all of these tests.

Comment 3

[[e-mail] (:bhearsum)]

browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_in_xorigin_frame.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_multi_process.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_paused.js
browser/base/content/test/webrtc/legacyIndicator/../browser_devices_get_user_media_screen.js

New logs for these can be found on https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=macosx1100&revision=eac40293649628d32911c909219d514bebb8ce63&test_paths=browser%2Fbase%2Fcontent%2Ftest%2Fwebrtc&selectedTaskRun=ThbuiqsjQ6OfktSHXf85EQ.0

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

cancelling long overdue needinfo

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Probably not a S2
Reminder, S2 means: (Serious) Major functionality/product severely impaired or a high impact issue and a satisfactory workaround does not exist

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1707735 - Turn browser/base/content/test/webrtc tests back on on apple silicon. r?mjf

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/99bcff922822
Turn browser/base/content/test/webrtc tests back on on apple silicon. r=mjf

Comment 8

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/99bcff922822

Comment 9

[Iulian Moraru]

Backed out for causing bc failures on browser_devices_get_user_media_in_xorigin_frame.js.
This seems to only affect OS X 11 WebRender Shippable opt.

• Push with failures

• Failure log

• Backout link

[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame.js | Allow button is enabled - 
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - Buffered messages finished
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame.js | Test timed out - 
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - GECKO(1825) | MEMORY STAT | vsize 402436MB | residentFast 395MB | heapAllocated 130MB
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - TEST-OK | browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame.js | took 45087ms
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - Not taking screenshot here: see the one that was previously logged
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame.js | Found a tab after previous test timed out: https://example.com/browser/browser/base/content/test/webrtc/get_user_media_in_xorigin_frame.html - 
[task 2023-07-14T04:30:35.961Z] 04:30:35     INFO - checking window state
[task 2023-07-14T04:30:35.962Z] 04:30:35     INFO - TEST-START | browser/base/content/test/webrtc/browser_devices_get_user_media_in_xorigin_frame_chain.js

LATER EDIT: this also caused failures on browser_device_controls_menus.js: https://treeherder.mozilla.org/logviewer?job_id=422601293&repo=autoland&lineNumber=7191

Comment 10

[Iulian Moraru]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/196cda3a1052