Create a common test harness argument parser

NEW
Unassigned

Status

enhancement
P3
normal
2 years ago
Last year

People

(Reporter: ahal, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

We've long thought it a good idea if our test harnesses standardized on command line arguments as much as possible. We attempt to name them the same as much as possible, but there is no enforcement.

We should create a common test harness argument parser that gets used by most of our test harnesses. This way the common arguments are guaranteed to stay in sync. I'd propose this lives under 'moztest.cli'. The benefit is that these arguments can now be used in places like |mach test| and |mach try| (in the future).

Under tools/tryselect, I think there is a pretty neat way of sharing arguments:
https://dxr.mozilla.org/mozilla-central/rev/2535bad09d720e71a982f3f70dd6925f66ab8ec7/tools/tryselect/cli.py#15

We could create a bunch of "groups". E.g, a 'chunking' group that contains all chunking related arguments. Specific test harness parsers could then declare which groups they want to add, e.g:

    MochitestArgumentParser(BaseTestHarnessParser):
        common_groups = ['chunking', 'debugging', 'marionette']
        arguments = {
            ...
        }

Similarly:

    ReftestArgumentParser(BaseTestHarnessParser):
        common_groups = ['chunking', 'debugging']
        arguments = {
            ...
        }

This will let us slice and dice arguments however we want, and provide an easy mechanism to share them across only the test harnesses that need them.
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.