1400012 - Make an easier developer patch to test worker type changes

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Justin Wood (:Callek)]

So, in IRC today, I saw :grenade's largish patch to do worker type testing for taskcluster. I created a patch that utilizes the task morph's instead, to avoid the need to do this all via a lot of different kind.yml's and transforms.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

You might also do this with a try template:
  https://firefox-source-docs.mozilla.org/taskcluster/taskcluster/try.html#modifying-tasks-in-a-try-push

Comment 2

[Justin Wood (:Callek)]

Attachment: morph version of patch

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8908427 [details] [diff] [review]
morph version of patch

Review of attachment 8908427 [details] [diff] [review]:
-----------------------------------------------------------------

i would propose the following slight modification:

            if task.task["workerType"] == worker:
                if task.task["workerType"].endswith('-gpu'):
                    task.task["workerType"] = "{}-b".format(worker)
                else:
                    task.task["workerType"] = "{}-beta".format(worker)

required because gpu beta workers are suffixed with "-b" rather than "-beta" due to workertype name length limitations

Comment 4

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Callek,

Maybe just an explicit mapping?

{
  "gecko-t-win7-32":      "gecko-t-win7-32-beta",
  "gecko-t-win7-32-gpu":  "gecko-t-win7-32-gpu-b",
  "gecko-t-win10-64":     "gecko-t-win10-64-beta",
  "gecko-t-win10-64-gpu": "gecko-t-win10-64-gpu-b",
  "gecko-1-b-win2012":    "gecko-1-b-win2012-beta",
}

Note, in a try push, we don't need to (probably shouldn't) convert level 2/3 worker type names.

Comment 5

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

... also makes it easy to comment out a line for each worker type you don't want to include in the transformation (often we want beta testers, but the regular win2012 builder) ...

Comment 6

[Justin Wood (:Callek)]

Pete, Rob, by all means adjust this -- I mainly wrote it as a place to store a temporary patch you could use on any try push you wanted to, in order to validate a worker, without having constant bitrot from the various places we specify workers :-).

I'm going to leave a n-i for you both here, and close it, since I consider the Proof Of Concept delivered ;-)

Comment 7

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Thanks.

Comment 8

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Cheers!

Comment 9

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Attachment: Transform to replace workerType with beta versions

This is an alternative implementation using a regular transform.

Comment 10

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Attachment: Transform to replace workerType with beta versions

Same as last version, with a flake8 (linting) fix.

Comment 11

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Attachment: Transform to replace workerType with beta versions

There was a minor bug in the last version - the transform has to be performed after `taskgraph.transforms.task:transforms` since the task transform replaces `{level}` in worker type name. Therefore the last patch updated tests, but not builds...

Fixed in this version.

Comment 12

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Attachment: Transform to replace workerType with beta versions

Whoops, I should have tested that previous patch first! xD

Comment 13

[Pete Moore [:pmoore][:pete] [PTO until 13 April 2026]]

Attachment: bug1400012_gecko_v5.patch

Added OS X staging worker type to transformation:

  "releng-hardware/gecko-t-osx-1010" => "releng-hardware/gecko-t-osx-1010-beta"

See bug 1452095 for details:
  https://bugzilla.mozilla.org/show_bug.cgi?id=1452095#c42