Closed Bug 1494090 Opened 5 years ago Closed 4 years ago
Add wrapper around pytest's '-x' flag to |mach python-test|
47 bytes, text/x-phabricator-request
|Details | Review|
Passing in -x to pytest aborts after the first failure. This is *really* useful when you have multiple failures in a single test file and want to debug them one at a time. It's currently possible to do this with: $ PYTEST_ADDOPTS=-x ./mach python-test ... But we should make it an official argument. Since every test file is run in a multiprocess, we would need some extra logic in python/mach_commands.py to abort remaining test files when an error is detected (though even if we dumbly pass the argument down, it would still be valuable).
According to |pytest --help| the |-x| option is essentially a synonym for |--maxfail=1|: > -x, --exitfirst exit instantly on first error or failed test. > --maxfail=num exit after first num failures or errors. If we're going to add an argument to |mach python-test| then do we want to also add the ability to exit after more than one failure? I'd personally prefer a way to pass any command line options unrecognised by |mach python-test| directly to pytest. This was also suggested in bug 1433941, and I've seen other instances where it would be useful to add pytest options conveniently via mach.
True, I'd be happy with passing all command line flags too. But this particular one will require a bit of extra logic in the mach command since we run one pytest command per test file. Maybe we can do the argument pass through as a pre-cursor commit within this bug.
(In reply to Andrew Halberstadt [:ahal] from comment #2) > True, I'd be happy with passing all command line flags too. But this > particular one will require a bit of extra logic in the mach command since > we run one pytest command per test file. Maybe we can do the argument pass > through as a pre-cursor commit within this bug. Longer term I'd like to see us moving closer to a pytest plugin model, where we could hook our test discovery into pytest itself and avoid invoking pytest for each test file. This will improve the test logs, and would allow us to take advantage of more pytest command line options and plugins. For now, let's keep this simple as you originally suggested and just add a `-x` argument to `mach python-test` that aborts on the first failure.
Pushed by email@example.com: https://hg.mozilla.org/integration/autoland/rev/4167cec91574 Added -x option to |mach python-test| r=ahal
You need to log in before you can comment on or make changes to this bug.