1955941 - Tag tasks with missing worker-implementation

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Johan Lorenzo [:jlorenzo]]

Attachment: Number of tasks on mozilla-central per day per worker-implementation

For the d2g migration, we would like to measure how many tasks are moved off of docker-worker to generic-worker. I'm working on a dashboard to visualize this. On the attached chart, we can see that a third of the tasks don't have any worker-implementation set. Data shows these task kinds don't set the tag worker-implementation:

partials-signing
partials
beetmover-checksums
balrog
beetmover-repackage
mar-signing-l10n
repackage-signing-l10n
repackage-signing-msi
shippable-l10n-signing
signing
build-signing
shippable-l10n-mac-signing
shippable-l10n-mac-notarization
signing-apk
beetmover-geckoview
geckodriver-signing
mar-signing
repackage-signing
build-mac-signing
repackage-signing-shippable-l10n-msix
geckodriver-mac-notarization
signing-bundle
beetmover-android-app
repackage-signing-msix
build-mac-notarization
release-update-product-channel-version
bouncer-locations
beetmover-apt
repo-update
merge-automation

As far as I know, the tag worker-implementation is used on the Taskcluster UI, but we don't use it anywhere else for data exploration. Tags were introduced in bug 1565378. and bug 1359976. Before then, we did have the concept of worker implementation (which was likely introduced in bug 1286075), but they were not exposed as tags.

Bug 1565378 explicitly excluded tasks that already have the aforementioned worker implementation. This logic explains the current situation where 1/3 of the tasks don't have the tag worker-implementation set.

Let's change that.

Comment 1

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1955941 - part 1: Refactor `set_implementation()` so that no task is skipped r=ahal

Comment 2

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1955941 - part 2: Tag all scriptworker implementations as scriptworker and os as linux r=ahal

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ec3fbbb9ceb5
part 1: Refactor `set_implementation()` so that no task is skipped r=ahal,taskgraph-reviewers
https://hg.mozilla.org/integration/autoland/rev/08e548c00ee6
part 2: Tag all scriptworker implementations as scriptworker and os as linux r=ahal,taskgraph-reviewers,releng-reviewers

Comment 4

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/ec3fbbb9ceb5
https://hg.mozilla.org/mozilla-central/rev/08e548c00ee6

Comment 5

[Johan Lorenzo [:jlorenzo]]

Attachment: Worker-implementation on April 9th, 2025.png

The patch reached mozilla-central 2 days ago. Therefore, almost all tasks that run yesterday were correctly tagged - see attached screenshot (source). The only ones that didn't get a worker-implementation set is the bootstrap kind. I looked into that. Actually, there was a backfill of these tasks yesterday which led to bug 1959550. The backfill went all the way up to a revision prior to mine. That explain why some bootstrap tasks had no worker-implementation (e.g.) while some do (e.g.).

Thus, this bug can remain resolved 🙂