1563281 - Make geckoview/fennec distinction more obvious on treeherder

Status: RESOLVED FIXED

(Testing :: General, enhancement, P2)

Description

[Geoff Brown [:gbrown]]

There is some confusion around which tests run against which mobile applications.

Comment 1

[Geoff Brown [:gbrown]]

Tests on "Android 4.3 API16+" generally run against Fennec, while tests on "Android 7.0 x86-64" generally run against geckoview, but there are some complications:

• cppunit tests are of course app-agnostic
• Android 4.3 geckoview-junit runs against geckoview
• Android 7.0 x86-64 xpcshell tests run against the fennec apk (bug 1553225)

Other considerations:

• Fennec tests are labelled "-1proc" currently.
• Fennec tests on trunk should be retired in September.
• raptor tests include "geckoview" in the test name where appropriate.

I think the current platform labels (eg "Android 7.0 x86-64") are appropriate. Possibly we can add "geckoview" to test names, as raptor does...but will that be noticed/obvious?

Comment 2

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6525e9ce08c00c3dff959ec18d7d51f8e78ea793

Comment 3

[Geoff Brown [:gbrown]]

Comment 2 demonstrated adding "-gv" to group names, eg M-gv(1 2 3 gpu). That is easy to implement, but does add visual clutter to treeherder. If special group names are to be added, I think it would be better to change the fennec group names: That way the change is temporary, going away naturally once fennec tests are retired. Something like:

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=f1d8a5c3641ec4eb7cdde61405577ffb94574a84

...but wait: that's just reinforcing the existing -1proc names! I'm not convinced there is much value to this approach.

Instead, let's just add fennec/geckoview to task labels: Those will show up in 'mach try fuzzy', 'mach try chooser', and in the job name in the lower-left pane of treeherder.

Comment 4

[Geoff Brown [:gbrown]]

Attachment: Bug 1563281 - Add mobile project name to test task labels; r=ahal

This adds the mobile project name, like "geckoview" or "fennec" to the Android
task labels. This makes the firefox/geckoview/fennec distinction more obvious
in 'mach try fuzzy' and 'mach try chooser'; it does not appear to affect
try syntax. This also adds "geckoview"/"fennec" to the job names displayed in
the lower-left treeherder pane. Hopefully this will help to clarify which
Android application is under test for each task.

Comment 5

[Geoff Brown [:gbrown]]

:jmaher - This modifies the test task labels for most Android tests. For example, mochitest-gpu-e10s becomes geckoview-mochitest-gpu-e10s, and mochitest-gpu-1proc becomes fennec-mochitest-gpu-1proc. ahal thinks this will have SETA implications, with all tests running as high priority for a while. Can you confirm? Can that be reasonably avoided? (If it is very hard, I'm okay with abandoning my change.)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

this will have SETA issues for a 24 hour period of time, so lets plan on 2 days of hiccups- maybe land this on a Friday afternoon to reduce issues?

We have to be aware of the load on emulators and phones, but a 2 day slow period of time will work just fine.

Comment 7

[Geoff Brown [:gbrown]]

Thanks Joel.

:aerickson - Unless you object, I will land this patch this Friday evening. We expect to see more Android test tasks per integration branch push, for 24 hours.

Comment 8

[Andrew Erickson [:aerickson]]

Sounds good. Thanks. :)

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4977e31ada62
Add mobile project name to test task labels; r=ahal

Comment 10

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/4977e31ada62