1325602 - Taskcluster Windows builds reported as "buildbot-unknown"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[(not currently active) Ted Mielczarek]

The perfherder build metrics recording code looks for `TASKCLUSTER_INSTANCE_TYPE` in the environment, and if it can't find it it assumes buildbot:
https://hg.mozilla.org/mozilla-central/file/91c099f8ec09/testing/mozharness/mozharness/base/python.py#l504

Unfortunately this isn't set for the Taskcluster generic worker builds, so they're being reported as 'buildbot-unknown':
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,58dd798456c0c59350fb5d65dc18fee63b89dd2a,1,2%5D&series=%5Bmozilla-inbound,14c1be0797a43a19e08666617834e7d763ad2a44,1,2%5D

Comment 1

[(not currently active) Ted Mielczarek]

I suppose the simplest fix (from my perspective) would be to make the generic worker set that environment variable, since it already has that info.

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

added to occ: https://github.com/mozilla-releng/OpenCloudConfig/commit/1016020101080262534c32b7a4fd016fe097f677

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

seems to work: https://treeherder.mozilla.org/perf.html#/graphs?timerange=86400&series=%5Bmozilla-inbound,e3a8706e69d38952bbffc4fbeaea4cdcd0189d9e,1,2%5D&series=%5Bmozilla-inbound,0a97132341cbfa94907add5fcd82457e129c43d6,1,2%5D&series=%5Bmozilla-inbound,6d26d23ca01b577243eb7e11a98ba6090f38aeb4,1,2%5D

Comment 4

[Gregory Szorc [:gps]]

Thank you for fixing this, Rob! Useful metrics FTW!

Comment 5

[(not currently active) Ted Mielczarek]

Thanks!

Comment 6

[(not currently active) Ted Mielczarek]

I apparently didn't look very hard at this when it landed, but I don't think this is quite right. The docker-worker sets this to the EC2 instance type, so our build metrics there look like "taskcluster-c4.4xlarge":
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,47ecd10e6c861b9e37227ca90ec05d6f59af4233,1,2%5D&series=%5Bmozilla-inbound,4578bf1db2ce1ac9169c983dc386809d891c1658,1,2%5D

Your patch is putting the *worker type* in that field, so we're getting data like "taskcluster-gecko-3-b-win2012":
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,e3a8706e69d38952bbffc4fbeaea4cdcd0189d9e,1,2%5D

IIRC, workers can have multiple instance types configured, so we can still get mixed-up data, and this also makes comparing build_times data between try and other branches difficult, because the try worker type is gecko-1 and other branches are gecko-1.

Is it possible to fix this to use the actual EC2 instance type instead?

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

hopefully done now: https://github.com/mozilla-releng/OpenCloudConfig/commit/79c21298bfe302e82657c1fcaa2a37090c1a8807
... should kick in with ami propagation ...

Comment 8

[Rob Thijssen [:grenade (EET/UTC+0300)]]

the patch seems to have taken:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=172800&series=%5Bmozilla-central,f0923bd27bb555b7e23fcd60d6806d21dec78adf,1,2%5D&series=%5Bmozilla-central,d416d6e2ae32b6d43bb1355394d31bfe4fbeee27,1,2%5D&series=%5Bmozilla-central,b46ec686c0f95460d45cd55e6ca84404814f944f,1,2%5D

Comment 9

[(not currently active) Ted Mielczarek]

Thanks again, that looks good now!