Check for --disable-tests when running mach test commands

RESOLVED FIXED in Firefox 64

Status

defect
RESOLVED FIXED
3 years ago
9 months ago

People

(Reporter: zbraniecki, Assigned: egao)

Tracking

Version 3
mozilla64
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox64 fixed)

Details

Attachments

(1 attachment, 3 obsolete attachments)

I can't run xpcshell-test

When I attempt to run the example from [0]:

`./mach xpcshell-test netwerk/test/`


I'm getting a failure with 0 tests run.

When I attempt to run full `./mach xpcshell-test`, I'm getting https://pastebin.mozilla.org/8888902

I asked :stas to try the same commands and he's getting the same errors, so it's not specific to my machine, but since we're both on Linux it may be linux specific.


[0] https://developer.mozilla.org/en-US/docs/Mozilla/QA/Writing_xpcshell-based_unit_tests#Your_first_xpcshell-based_test
:gps, can you help me?
Flags: needinfo?(gps)
Much to my surprise, removing "--disable-tests" from mozconfig fixed this issue. Magic.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(gps)
Resolution: --- → INVALID
We should definitely error if you try to run a mach test command with --disable-tests.
Component: XPCShell Harness → General
Summary: Can't launch xpschell-test → Check for --disable-tests when running mach test commands
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Assignee

Updated

9 months ago
Assignee: nobody → egao
Assignee

Comment 4

9 months ago
- implemented a mozconfig checker inside runxpcshelltests.py::XPCShellTests::runTests, which checks the build.mozconfig['configure_args'] for --disable-tests option.
- changed runspcshelltest.py::XPCShellTests::buildTestList to clearly print error then exit, with additional error message hinting at --disable-tests flag. This is to cover case where compilation was done with --disable-tests flag, but user attempts to run tests by mnaually specifying another mozconfig that does not have --disable-tests flag.
Assignee

Comment 5

9 months ago
- debugging statements to run on try server

Depends on D7130
Assignee

Comment 6

9 months ago
- further debug statements

Depends on D7131
Assignee

Comment 7

9 months ago
- restored check for build.mozconfig.configure_args value, surrounded by checks to ensure it does not fail in automation environment

Depends on D7132
Attachment #9012708 - Attachment is obsolete: true
Attachment #9012707 - Attachment is obsolete: true
Attachment #9012709 - Attachment is obsolete: true
Thanks for taking this :egao. It looks like you are on the right track here.

I think you could put your check in either mach_commands.py or the test harness - your choice. Hopefully you can apply your solution to all of the mach test commands/harnesses: mach test, web-platform-test, mochitest, reftest, xpcshell-test, etc.
Assignee

Comment 10

9 months ago
An updated try run with a revised patch:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=45fb4eac5374e6623c3cf27bd41649b4474c471a

Revised patch details:

1. patch from comment 7 worked only for xpcshelltest, when invoked by: ./mach xpcshell-test <test_name>.
The revised patch removes this change, restoring testing/xpcshell/mach_commands.py to default state.

2. newer patch works at mozilla-central/python/mach/mach/main.py. As the arguments are parsed and appropriate flags are set, additional checks are done using the following criteria in order:
- if MOZCONFIG environment variable is present;
- if MOZCONFIG specified in environment contains '--disable-tests' flag;
- if mach contains 'test' subcommand;
- if mach command contains 'test' in the cli argument; then
- a message stating that tests are disabled is shown.

I have checked the patch locally with the following commands:

./mach try
./mach bootstrap
./mach mochitest
./mach test mochitest
./mach xpcshell-test
./mach test xpcshell
./mach --help
./mach try
Attachment #9012710 - Attachment description: Bug 1291335 - Check for --disable-tests when running mach test commands r?gbrown → Bug 1291335 - Check mozconfig for --disable-tests when running mach test commands r?gbrown

Comment 11

9 months ago
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b58cfd180ba7
Check mozconfig for --disable-tests when running mach test commands r=gbrown

Comment 12

9 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/b58cfd180ba7
Status: REOPENED → RESOLVED
Closed: 3 years ago9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
You need to log in before you can comment on or make changes to this bug.