1357701 - Buildbot try_parser.py needs to support running all mochitest-chrome chunks for |-u mochitest-chrome|

Status: VERIFIED FIXED

(Release Engineering :: General, enhancement)

Description

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

When I specify |-u mochitest-chrome| by trychooser, mochitest-chrome does not run on OS X opt, Windows 7 opt nor Windows 7 debug. However, only on Linux and OS X debug, it runs.

See this treehearder:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f7b75f060b3899d51b7f1689c8ba203f7de1bf51

I used:
> ./mach try -b do -p linux,linux64,linux64-asan,linux64-stylo,linux64-st-an,linux64-valgrind,linux64-haz,macosx64,macosx64-st-an,win32,win32-clang,w
in32-st-an,win64,win64-clang,win64-st-an -u mochitest-chrome -t none

Comment 1

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

And also trychooser has mochitest-chrome in Android only section, but it never runs on Android...

Comment 2

[Phil Ringnalda (:philor)]

I'm unable to reproduce "it never runs on Android" - https://treeherder.mozilla.org/#/jobs?repo=try&revision=79822274b57cc0c058152c5ba4cf030eb0dc2b30&group_state=expanded has the chunks that should be run on Android, running the tests that do manage to run on Android.

To make sense of the odd desktop behavior, you have to know about three things: there are separate and barely-compatible trychooser syntax parsers for buildbot and taskcluster, and the taskcluster one implemented treating mochitest-chrome as meaning "all chunks of mochitest-chrome which exist on a platform" while the buildbot one did not; over the course of a year we very very slowly moved from having a suite called mochitest-other which ran both mochitest-chrome and mochitest-a11y to having separate mochitest-a11y and three chunks of mochitest-chrome, without ever updating trychooser to reflect that move; across roughly the same year, taskcluster took over first the scheduling and the running of Linux and Android and then the scheduling but not the running of OS X debug. So you get them to run on Linux with "-u mochitest-chrome" because those are scheduled and run by taskcluster, and you get them to run on OS X debug because those are scheduled by taskcluster to run on buildbot, and you do not get them on OS X opt or Windows because those are scheduled by buildbot.

I have a trychooser patch to replace mochitest-o with mochitest-a11y and mochitest-chrome, but that depends on this bug landing buildbot try_parser.py support for running all the chunks when it sees "-u mochitest-chrome".

(In the meantime, the workaround is just to use "-u mochitest-chrome-1,mochitest-chrome-2,mochitest-chrome-3,mochitest-chrome-4".)

Comment 3

[Phil Ringnalda (:philor)]

Attachment: Like they do it in taskclusterland

Comment 4

[Phil Ringnalda (:philor)]

FWIW, the trychooser PR is https://github.com/mozilla-releng/services/pull/286 since I realized it doesn't actually depend on this to make things better, only to make things best.

Comment 5

[Phil Ringnalda (:philor)]

Comment on attachment 8863298 [details] [diff] [review]
Like they do it in taskclusterland

https://hg.mozilla.org/build/buildbotcustom/rev/a9845dab0ebb1e3b112f793f62b6bf4886b319b8

Comment 6

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

-> v.

Thanks!