1707748 - 4 tests skipped on apple_silicon

Status: RESOLVED DUPLICATE of bug 1753310

(Toolkit :: Performance Monitoring, task)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Last week we turned on tests for Apple Silicon (OSX 11.2.3 on new Apple based hardware). We are using the simplified new test config process:
https://firefox-source-docs.mozilla.org/testing/ci-configs/index.html

As the tests are live, we are now filing bugs to help close the loop and hope to fix any issues over the next 7 weeks. As the process outlines, there are tier-3 jobs running on m-c which run these skipped tests and will expect a failure/timeout/crash- if it doesn't fail, then the job will turn orange.

Here are the failing tests:
toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_default_options.js
toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_all_frames.js
toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_threads.js
toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_frames_without_threads.js

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:florian - as triage owner I wanted to give you a heads up about the skipped tests on apple silicon.

Comment 2

[[e-mail] (:bhearsum)]

Here's a link to some newer logs: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=macosx1100&revision=eac40293649628d32911c909219d514bebb8ce63&test_paths=toolkit%2Fcomponents%2Faboutprocesses&selectedTaskRun=ThbuiqsjQ6OfktSHXf85EQ.0

Comment 3

[Florian Quèze [:florian]]

(In reply to [e-mail] (:bhearsum) from comment #2)

Here's a link to some newer logs: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=macosx1100&revision=eac40293649628d32911c909219d514bebb8ce63&test_paths=toolkit%2Fcomponents%2Faboutprocesses&selectedTaskRun=ThbuiqsjQ6OfktSHXf85EQ.0

The log shows:

TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_default_options.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_default_options.js | took 2385ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_all_frames.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_all_frames.js | took 2954ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_frames_without_threads.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_frames_without_threads.js | took 2096ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_threads.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_threads.js | took 2769ms

Shouldn't the failures be made visible in the log if there are some left?

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Florian Quèze [:florian] from comment #3)

(In reply to [e-mail] (:bhearsum) from comment #2)

Here's a link to some newer logs: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=macosx1100&revision=eac40293649628d32911c909219d514bebb8ce63&test_paths=toolkit%2Fcomponents%2Faboutprocesses&selectedTaskRun=ThbuiqsjQ6OfktSHXf85EQ.0

The log shows:

TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_default_options.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_default_options.js | took 2385ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_all_frames.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_all_frames.js | took 2954ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_frames_without_threads.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_frames_without_threads.js | took 2096ms
TEST-START | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_threads.js
TEST-OK | toolkit/components/aboutprocesses/tests/browser/browser_aboutprocesses_show_threads.js | took 2769ms

Shouldn't the failures be made visible in the log if there are some left?

Sorry, I realize this is a bit unclear! These logs are from the failures tasks, which run all tests that are marked as skip/known fail, and pass tests that still do that. So, TEST-OK means "it is still failing". If you push to try with the annotation removed, it will fail as usual.

Comment 5

[Florian Quèze [:florian]]

(In reply to [e-mail] (:bhearsum) from comment #4)

Sorry, I realize this is a bit unclear! These logs are from the failures tasks, which run all tests that are marked as skip/known fail, and pass tests that still do that. So, TEST-OK means "it is still failing". If you push to try with the annotation removed, it will fail as usual.

Wouldn't it make sense for these tasks to still output in the logs the messages of the failures? Eg. mark them TEST-EXPECTED-FAIL but still print them in the log so that the failures can be investigated directly without needing to do a specific try push to just to see the output?

Comment 6

[Florian Quèze [:florian]]

The root cause is bug 1753310.