Closed Bug 831830 Opened 8 years ago Closed 8 years ago

Run Marionette unit tests before WebAPI tests, not after

Categories

(Testing :: Marionette, defect)

All
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(b2g18 fixed)

RESOLVED FIXED
mozilla21
Tracking Status
b2g18 --- fixed

People

(Reporter: jgriffin, Assigned: jgriffin)

Details

Attachments

(1 file)

On marionette-webapi test runs, we end up running all of the WebAPI tests, and then the Marionette unit tests.  We should run these the other way around.
I always thought that these should be two separate jobs entirely. It's kind of weird that on desktop we have only Mn tests but on B2G Mn means "marionette+webapi".

* Is there any particular reason we run them together?
* How hard would it be to run them separately?
Going to run this on try to make sure we don't get any weird behavior.
Attachment #703393 - Flags: review?(rwood)
I don't think we need to run them separately.  For mochitests, there are a set of mochitest 'unit tests' (harness sanity, special powers, etc) that we run before the rest of the tests.  It's just a way of catching problems in the harness that may prevent later tests from failing.
Comment on attachment 703393 [details] [diff] [review]
Run Marionette unit tests before webapi tests,

Review of attachment 703393 [details] [diff] [review]:
-----------------------------------------------------------------

Lgtm and ran fine on my local VM.
Attachment #703393 - Flags: review?(rwood) → review+
(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> I don't think we need to run them separately.  For mochitests, there are a
> set of mochitest 'unit tests' (harness sanity, special powers, etc) that we
> run before the rest of the tests.  It's just a way of catching problems in
> the harness that may prevent later tests from failing.

When are those tests run? For each Mochitest chunk or already in the Build phase? We could save a lot of time if it would happen in the latter.
They're not run once per chunk, but just once per set, usually as part of chunk 1.
But shouldn't we run this in the build step? If something is wrong with Marionette we do not have to execute all the referenced tests which will save a lot of time on the testing machines.
It isn't possible to run them in the build step.  The only kinds of tests we can run in the build step are the C++ tests.  Marionette has a set of dependencies which currently are not met on the build slaves.
https://hg.mozilla.org/mozilla-central/rev/5e5b67c6c031
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.