Open Bug 984911 Opened 6 years ago Updated 6 years ago

Look into sorting of mochitest manifests

Categories

(Testing :: Mochitest, defect)

x86_64
Linux
defect
Not set

Tracking

(Not tracked)

People

(Reporter: billm, Unassigned)

Details

In bug 938019, we tried running tests in the order they're specified in the .ini files. That caused a bunch of tests to go orange because it's a different order than we currently run them, and tests don't like that. I think we have several options here.

1. Track down the sources of the test failures and fix the bugs. This would be best, but the most time consuming.

2. Require that tests appear in manifest files in order. I think we already do this for moz.build files.

3. Move the sorting from runtests.py to manifestparser.py.
What I want is sort of 2a -- Just sort the existing manifests into alphabetical order, which should preserve the existing behavior, then stop sorting in the harness. New tests will get run in whatever order they're listed in the manifest. (The only issue here is bad interactions between tests, which we have a long history of. For new tests the burden is on the test author.)
You need to log in before you can comment on or make changes to this bug.