Closed Bug 979512 Opened 11 years ago Closed 11 years ago

Add a command line option to split the tests and run a specified chunk

Categories

(Remote Protocol :: Marionette, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla31

People

(Reporter: davehunt, Assigned: davehunt)

References

Details

Attachments

(1 file, 3 obsolete files)

It looks like we're going to need to split the manifest when running Gaia UI tests against emulators on TBPL due to us hitting the 2 hour build limit. Rather than split up the manifest files manually and have to maintain these separate files, it would be good to control this via a command line option.

I'm thinking something like --chunk=1:4 where the tests selected to run would be split into four groups, as equal in number as possible, and the first group would be run. Subsequent (or parallel) jobs using --chunk=2:4 etc would run the remaining groups.

So long as all jobs are running on the same revision then the groups can be trusted not to overlap or omit tests.
Assignee: nobody → dave.hunt
Status: NEW → ASSIGNED
Comment on attachment 8388459 [details] [diff] [review]
Add a command line option to split the tests and run a specified chunk. v1.0

Review of attachment 8388459 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/marionette/client/marionette/runner/base.py
@@ +975,5 @@
>              if self.marionette.check_for_crash():
>                  break
>  
>      def run_test_sets(self):
> +        run_chunk, total_chunks = [int(s) for s in self.chunk.split(':')]

Can we use this_chunk instead of run_chunk to be consistent with mochitest and other harnesses?
Attachment #8388459 - Flags: review?(jgriffin) → review+
Carried r+
Attachment #8388459 - Attachment is obsolete: true
Attachment #8389137 - Flags: review+
Before landing this, I notice that mochitest uses --this-chunk and --total-chunks command line options. Do we want to simply use that rather than introduce a strictly formatted string? My initial thinking was that it would be better to keep the number of command line options to a minimum, but the two arguments would be less error prone and would be consistent with mochitests.
Flags: needinfo?(jgriffin)
dave and I discussed and agreed it made sense to use the same argument names as mochitest, for consistency's sake.
Flags: needinfo?(jgriffin)
Attachment #8389137 - Attachment is obsolete: true
Attachment #8392208 - Flags: review?(jgriffin)
Attachment #8392208 - Flags: review?(jgriffin) → review+
And other bustage:
https://tbpl.mozilla.org/php/getParsedLog.php?id=36389402&tree=Fx-Team

Please make sure this passes on Try before re-requesting checkin.
Whiteboard: [fixed-in-fx-team]
Sorry for that silly mistake Ryan, try looks good now.
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/2cd13f9fe11e
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla31
Blocks: 987045
Product: Testing → Remote Protocol
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: