Open Bug 1509045 Opened 6 years ago Updated 2 years ago

Some mochitest test suites are wrongly selected by filter_tasks_by_paths because the mochitest task_regex is too loose

Categories

(Testing :: General, defect, P3)

defect

Tracking

(Not tracked)

People

(Reporter: marco, Unassigned)

References

(Blocks 1 open bug)

Details

For example for the test "dom/html/test/forms/xbl/test_step_attribute.html", which has flavor "mochitest" and no subsuite, mochitest-webgl1-core is selected because the regex is "mochitest(?!-a11y|-browser|-chrome|-clip|-devtools|-gpu|-media|-screen)($|.*(-1|[^0-9])$)".
It would be good, after fixing this bug, to add assertions to desktop_unittest.py (and the equivalents for marionette and wpt) to make sure that, when MOZHARNESS_TEST_PATHS is defined and has a value and if the suite is supported by MOZHARNESS_TEST_PATHS, MOZHARNESS_TEST_PATHS[suite] is not empty (so we avoid this problem reoccuring).
Depends on: 1509048
Now that bug 1509048 is fixed, |mach try| selectors (which are currently the only consumers of the "task_regex"), have access to the full taskgraph, rather than just the labels.

That means that we should be able to replace all these task_regexes with `filter` functions. E.g:

    'marionette': {
        'aliases': ('mn',),
        'mach_command': 'marionette-test',
        'kwargs': {'tests': None},
        'task_filter': lambda t: t.attr['kind'] == 'test' && t.attr['unittest_suite'] == 'marionette',
    },

This would be much less fragile than relying on regexes.
Priority: -- → P3
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.