Closed Bug 1459251 Opened 6 years ago Closed 6 years ago

Running |mach raptor-test| with no test specified results in a traceback

Categories

(Testing :: Raptor, defect)

Product:
Testing
Component:
Raptor
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox62 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla62
Tracking Flags:
Tracking Status
firefox62 --- fixed

People

(Reporter: ahal, Assigned: rwood)

References

Details

Attachments

(1 file)

Bug 1459251 - Running |mach raptor-test| with no test specified results in a traceback;
59 bytes, text/x-review-board-request
ahal
: review+
Details 
This should probably have a better error message. Semi-relatedly, how can one fine the list of valid test names that can be passed in? It might be good to make the 'test' argument a choices action so that they can be discovered via |mach talos-test --help|.
Making it a 'choices' action would also make sure we fail fast if an invalid test were passed in (currently it does ~20 seconds worth of mozharness setup before failing).
Thanks yes I have this fixed already in the patch I'm working on for processing output. I'll move it to a separate patch here tho thanks.
(In reply to Andrew Halberstadt [:ahal] from comment #1)
> Making it a 'choices' action would also make sure we fail fast if an invalid
> test were passed in (currently it does ~20 seconds worth of mozharness setup
> before failing).

Yeah so I use manifest parser, that builds a list of available tests first. If a test argument is passed in then I check the list of tests manifest parser created. If it's in that list then it's valid, otherwise errors out invalid test. If no test is specified then all available tests will run.
Is there a way to do what you want and still use test manifests? Maybe make the test argument a choices argument but get the choices from manifest parser...
Assignee: nobody → rwood
Blocks: 1449180
Status: NEW → ASSIGNED
Comment on attachment 8973289 [details]
Bug 1459251 - Running |mach raptor-test| with no test specified results in a traceback;

This patch fixes the case of running:

./mach raptor-test

Now it will find and run all available tests. Note, for me on OSX, if you provide an invalid test name i.e:

./mach raptor-test invalid-test-name

It only takes 2 seconds before it errors out with:

09:58:12     INFO -  manifest abort: specified test doesn't exist!
09:58:12     INFO -  raptor-main abort: no tests found
09:58:12    ERROR - Return code: 1

Which is fine IMO.
Attachment #8973289 - Flags: review?(ahalberstadt)
(In reply to Robert Wood [:rwood] from comment #6)
 
> ./mach raptor-test invalid-test-name
> 
> It only takes 2 seconds before it errors out with:
> 
> 09:58:12     INFO -  manifest abort: specified test doesn't exist!
> 09:58:12     INFO -  raptor-main abort: no tests found
> 09:58:12    ERROR - Return code: 1
> 
> Which is fine IMO.

Ah, because I already ran Raptor previously and the raptor-venv already existed, otherwise yeah I see about 20 seconds before it errors out. I think it's fine as that's only the first time if the raptor-venv doesn't exist yet.
Comment on attachment 8973289 [details]
Bug 1459251 - Running |mach raptor-test| with no test specified results in a traceback;

https://reviewboard.mozilla.org/r/241754/#review248934

Thanks!
Attachment #8973289 - Flags: review?(ahalberstadt) → review+
Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/fe2a0b79e7af
Running |mach raptor-test| with no test specified results in a traceback; r=ahal
https://hg.mozilla.org/mozilla-central/rev/fe2a0b79e7af
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox62: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: