Android test chunks determined by mozharness configuration

RESOLVED FIXED

Status

defect
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

unspecified

Firefox Tracking Flags

(firefox50 fixed)

Details

Attachments

(1 attachment)

android_emulator_unittest.py does not accept options like --this-chunk and --total-chunk. Instead, callers request a test suite and chunk with an option like "--test-suite=mochitest-23". The mozharness configuration maps the test suite name to a suite category (eg mochitest) and a chunk number.

That's inconvenient when increasing chunks: The config file needs to be updated with new suite mappings and callers (taskcluster and buildbot) must be updated at the same time to ensure the new suites are called.

Use of command line options like desktop_unittest's --this-chunk and --total-chunk would be more convenient and allow buildbot and taskcluster to be updated independently.
This adds support for --this-chunk and --total-chunk arguments in android_emulator_unittest.py. Old-style requests are still accepted too. Callers can request, for example:
  --test-suite=mochitest --this-chunk=3 --total-chunk=20
or
  --test-suite=mochitest-3

In the latter case, the script now parses {{suite}}-{{chunk}} rather than using the test_suite_definitions mapping from the configuration. This allows for the complete removal of test_suite_definitions, simplifying the config file substantially. (I also had to make a minor change to the mochitest-chrome config to remove test_suite_definitions, but that just made the mochitest-chrome settings more consistent, like mochitest-media, etc.)

I intend to start using --this-chunk and --total-chunk from taskcluster soon. Eventually, I think I will remove total_chunks from the mozharness configuration and rely on --total-chunk (perhaps once buildbot jobs are retired).

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf4e1bd9b448 shows how this makes it easier to update chunks on taskcluster only -- bug 1287831.
Attachment #8773525 - Flags: review?(kmoir)
Attachment #8773525 - Flags: review?(kmoir) → review+
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/06d44cf3896a
Accept total-chunk, this-chunk options in android_emulator_unittest.py; r=kmoir
https://hg.mozilla.org/mozilla-central/rev/06d44cf3896a
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.