1274610 - Taskcluster Android 4.3 tests appear beside Android 4.0 builds

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect)

Description

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Currently on treeherder.mozilla.org, we have "Android 4.0 API15+ opt" and "Android 4.3 API15+ opt" jobs. "Android 4.3 API15+ opt" tests depend on "Android 4.0 API15+ opt" builds, but tests are displayed alongside Android 4.3 -- and that's all good.

I am looking at adding taskcluster-initiated Android 4.3 tests. It seems that whatever I try, the tests appear in treeherder alongside "Android 4.0 API15+ opt".

camd kindly suggested that I use:

+  extra:
+    treeherder:
+      machine:
+        platform: android-4-3-armv7-api15

which seems brilliant, but didn't work:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5896dee001cc&selectedJob=21167837

Comment 1

[Cameron Dawson [:camd]]

Geoff: would you check my work on this one?  Because when I inspect the task for the job you referenced, it shows this in the treeherder section:

  "treeherder": {
      "groupSymbol": "tc-M",
      "collection": {
        "opt": true
      },
      "machine": {
        "platform": "android-4-0-armv7-api15"
      },
      "groupName": "Desktop mochitests",
      "build": {
        "platform": "android-4-0-armv7-api15"
      },
      "tier": 2,
      "revision_hash": "03150084c740ebb10cf94830beaa789b0518cc88",
      "symbol": "c",
      "revision": "5896dee001cc3ab543f346cd18d554df23a860e4"
    }
  }

https://tools.taskcluster.net/task-inspector/#HqA82Cm_TsWmjDRzJO3E3w/
https://queue.taskcluster.net/v1/task/HqA82Cm_TsWmjDRzJO3E3w

So it's set to the 4-0 in there.  Maybe this task definition didn't actually get updated?

Comment 2

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Oh, that's interesting!

I did set treeherder.machine.platform to android-4-3-armv7-api15:

https://hg.mozilla.org/try/rev/281e8ce8dcad168f7caea28b699044e80968fc86#l4.45

android-4-0-armv7-api15 is of course in the yml for the build which triggers that test:

https://hg.mozilla.org/try/rev/86f1f9e84713995e72b3eaa3d3949d963846f468#l1.7

I wonder if there is something in taskcluster that sets a test task's treeherder.machine.platform to that of the build that triggered the test. :garndt -- do you know?

Comment 3

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Nevermind, I found it: 

https://hg.mozilla.org/mozilla-central/annotate/8d0aadfe7da7/testing/taskcluster/taskcluster_graph/mach_util.py#l78

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Yes, we don't have great support for differentiating build_platform and test_platform right now.  You can probably hack something into mach_util.py or taskcluster/taskgraph/kind/legacy.py though.

Comment 5

[Geoff Brown [:gbrown] (pto Apr 11-18)]

I tried hacking mach_util.py to not inherit machine: 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a9ddfb3b8070

which produced

https://queue.taskcluster.net/v1/task/erbbOlhDRomas1n18NcjSg

    "treeherder": {
      "groupSymbol": "tc",
      "collection": {
        "opt": true
      },
      "machine": {
        "platform": "android-4-3-armv7-api15"
      },
      "groupName": "Desktop tests",
      "build": {
        "platform": "android-4-0-armv7-api15"
      },
      "tier": 2,
      "revision_hash": "9f9307d1e907378d7e262d2329799ec4db47ccee",
      "symbol": "Cpp",
      "revision": "a9ddfb3b8070e74d015b270f784ca941d269696a"
    }

but failed to "move" the Cpp test job in treeherder - it still showed up beside Android 4.0.


Then I tried additional hacking to set build.platform to "android-4-3-armv7-api15" also, but that seems to strangely affect scheduling (tc build does not start until buildbot build completes??). It's perhaps better for someone else to sort this out properly.

Comment 6

[Geoff Brown [:gbrown] (pto Apr 11-18)]

(In reply to Geoff Brown [:gbrown] (pto May 28-June 13) from comment #5)
> Then I tried additional hacking to set build.platform to
> "android-4-3-armv7-api15" also, but that seems to strangely affect
> scheduling (tc build does not start until buildbot build completes??). 

The taskcluster android-api-15 build was hidden on treeherder recently (while I was making these changes and observing results!) -- that may have confused me. (Likely hidden for bug 1275016.)

Comment 7

[Cameron Dawson [:camd]]

OK, now looking further, the data for the job is correct.  But our UI was doing the wrong thing with it.

your job now has these fields in the JSON:

    "build_platform": "android-4-0-armv7-api15",
    "platform": "android-4-3-armv7-api15",

However, we were using ``build_platform`` to decide where to group the jobs, rather than ``platform``.  In the past, these values were always the same (for buildbot, and usually elsewhere) so we never noticed it.  Anyway, sorry for the hassle, but I'll submit a PR shortly to fix this.  Very easy fix.  :)

Comment 8

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:platform-aggregation > mozilla:master

Comment 9

[William Lachance (:wlach)]

Comment on attachment 8756970 [details] [review]
[treeherder] mozilla:platform-aggregation > mozilla:master

Makes sense.

Comment 10

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/1f199dd249f474be1b3c1d9ea98e70b56c802e09
Bug 1274610 - Use platform field to group jobs instead of build_platform