Split Mn job on Treeherder into unit and integration tests
Categories
(Testing :: Marionette Client and Harness, task, P2)
Tracking
(firefox132 fixed, firefox133 fixed)
People
(Reporter: whimboo, Assigned: jmaher)
References
(Blocks 1 open bug)
Details
(Whiteboard: [webdriver:external][webdriver:m13] )
Attachments
(2 files)
On bug 1916456 we added the capability to split the Mn job into multiple chunks. It was necessary because a lot of new tests were added over the past couple months exercising restart scenarios for Firefox and causing task timeouts for the Mn jobs for ASAN builds.
But as noticed this doesn't seem to be enough given that there are some tests like the windows enrollment tests that take about 15 minutes for ASAN and still cause the Mn2 job to fail with a task timeout:
https://treeherder.mozilla.org/logviewer?job_id=475926892&repo=autoland&lineNumber=105162
As such we should split the Marionette jobs into unit tests that only run the tests at:
https://searchfox.org/mozilla-central/source/testing/marionette/harness/marionette_harness/tests/unit/
And as well the integration (e2e) tests that run the tests from all the other locations. Only those we may have to chunk.
CC'ing Joel for further input.
Assignee | ||
Comment 1•28 days ago
|
||
this is a good idea, we should be able to do this via the .yml files, creating a job for the unittests by pointing directly at the manifest and adjusting the moz.build to not include the unittests as part of MARIONETTE_MANFIESTS
so the existing tasks run faster.
Assignee | ||
Comment 2•24 days ago
|
||
Updated•24 days ago
|
Updated•19 days ago
|
Updated•18 days ago
|
Assignee | ||
Comment 4•17 days ago
|
||
Comment 6•17 days ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/57a2af12c61e
https://hg.mozilla.org/mozilla-central/rev/7a6329ddf6b7
Updated•16 days ago
|
Reporter | ||
Comment 8•12 days ago
|
||
Hi Joel, thank you for the work on this bug and getting the changes landed. I had a look and I actually wonder if we should put both of the jobs into a Mn
group instead. That would give a better separation on Treeherder. What do you think?
Assignee | ||
Comment 9•8 days ago
|
||
this sounds like a good idea. Would we have: Mn(Mn Mn-swr unit unit-swr)
? I ask as the Mn(Mn)
seems repetitive, maybe there is a better symbol to use inside the Mn()
?
Reporter | ||
Comment 10•8 days ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC -8) from comment #9)
this sounds like a good idea. Would we have:
Mn(Mn Mn-swr unit unit-swr)
? I ask as theMn(Mn)
seems repetitive, maybe there is a better symbol to use inside theMn()
?
What if we do it similar to Mochitests and just use numbers for integration tests? That would result in Mn(1, 2, unit1, unit2)
. Also would it be possible to have the swr
jobs in their own group like Mn-swr(1, 2, unit1, unit2)
?
Description
•