1288501 - Android test chunks determined by mozharness configuration

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Geoff Brown [:gbrown]]

android_emulator_unittest.py does not accept options like --this-chunk and --total-chunk. Instead, callers request a test suite and chunk with an option like "--test-suite=mochitest-23". The mozharness configuration maps the test suite name to a suite category (eg mochitest) and a chunk number.

That's inconvenient when increasing chunks: The config file needs to be updated with new suite mappings and callers (taskcluster and buildbot) must be updated at the same time to ensure the new suites are called.

Use of command line options like desktop_unittest's --this-chunk and --total-chunk would be more convenient and allow buildbot and taskcluster to be updated independently.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: add --this-chunk, --total-chunk options and simplify configuration

This adds support for --this-chunk and --total-chunk arguments in android_emulator_unittest.py. Old-style requests are still accepted too. Callers can request, for example:
  --test-suite=mochitest --this-chunk=3 --total-chunk=20
or
  --test-suite=mochitest-3

In the latter case, the script now parses {{suite}}-{{chunk}} rather than using the test_suite_definitions mapping from the configuration. This allows for the complete removal of test_suite_definitions, simplifying the config file substantially. (I also had to make a minor change to the mochitest-chrome config to remove test_suite_definitions, but that just made the mochitest-chrome settings more consistent, like mochitest-media, etc.)

I intend to start using --this-chunk and --total-chunk from taskcluster soon. Eventually, I think I will remove total_chunks from the mozharness configuration and rely on --total-chunk (perhaps once buildbot jobs are retired).

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf4e1bd9b448 shows how this makes it easier to update chunks on taskcluster only -- bug 1287831.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/06d44cf3896a
Accept total-chunk, this-chunk options in android_emulator_unittest.py; r=kmoir

Comment 3

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/06d44cf3896a