1289779 - Make sure that different docker-image tasks use different job symbols

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Right now all the docker image tasks on TC use the job symbol `I`, which makes it hard to identify which docker image is actually build. It would be great if we can have different symbols for easier inspection on Treeherder.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

This would involve making some modifications under taskcluster/ci/docker-image and/or taskcluster/taskgraph/task/docker_image.py.  There are docs under taskcluster/docs/ and also https://docs.taskcluster.net/tutorial.  I'll be happy to fill in gaps.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Hammad, if I remember correctly, you said you were hacking on this because it was a dependency of one of the verification bugs.

Comment 3

[HAMMAD AKHTAR]

Attachment: Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

Review commit: https://reviewboard.mozilla.org/r/98336/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/98336/

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98594

You're on a roll!

::: taskcluster/ci/docker-image/kind.yml:14
(Diff revision 1)
> -  - desktop-test
> -  - desktop1604-test
> -  - desktop-build
> -  - tester
> -  - lint
> -  - android-gradle-build
> +  desktop-test: A
> +  desktop1604-test: B
> +  desktop-build: C
> +  tester: D
> +  lint: E
> +  android-gradle-build: F

Please invent some shorthands here that remind people what they mean -- something like "dt" for desktop-test, etc.

::: taskcluster/taskgraph/task/docker_image.py:73
(Diff revision 1)
> +            if 'extra' in image_task['task']:
> +                image_task['task']['extra']['treeherder']['symbol'] += image_symbol

Let's make this a taskGroup, so each task looks like

 I(dt)
 I(d16t)
 I(db)

etc..

Comment 5

[HAMMAD AKHTAR]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98614

::: taskcluster/ci/docker-image/kind.yml:14
(Diff revision 1)
> -  - desktop-test
> -  - desktop1604-test
> -  - desktop-build
> -  - tester
> -  - lint
> -  - android-gradle-build
> +  desktop-test: A
> +  desktop1604-test: B
> +  desktop-build: C
> +  tester: D
> +  lint: E
> +  android-gradle-build: F

how about following naming
desktop-test: dt
desktop1604-test: d16t
desktop-build: db
tester: tst
lint: lnt
android-gradle-build: agb

::: taskcluster/taskgraph/task/docker_image.py:73
(Diff revision 1)
> +            if 'extra' in image_task['task']:
> +                image_task['task']['extra']['treeherder']['symbol'] += image_symbol

so I should make `I` as groupSymbol
and `dt` as symbol in case of desktop-build. same for other images.

Comment 6

[HAMMAD AKHTAR]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/98336/diff/1-2/

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98614

> how about following naming
> desktop-test: dt
> desktop1604-test: d16t
> desktop-build: db
> tester: tst
> lint: lnt
> android-gradle-build: agb

I like it.

> so I should make `I` as groupSymbol
> and `dt` as symbol in case of desktop-build. same for other images.

Yes, exactly

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98636

"treeherder": {
          "build": {
            "platform": "taskcluster-images"
          },
          "groupSymbol": "I",
          "jobKind": "other",
          "symbol": "dt"
        },
        
Looks good!

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98638

You will need to clear the two mozreview issues you've opened before I can land this.

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98640

Sorry if that wasn't clear -- when you replied to my comments on the review, you left the "Open an Issue" checkbox checked.  Since you opened those issues, only you can close them by marking them as fixed.  And I can't land the commit until all of the issues are closed.

Comment 11

[HAMMAD AKHTAR]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98640

I think I have closed all the open issues :)

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8818199 [details]
Bug 1289779 - Make sure that different docker-image tasks use different job symbols;

https://reviewboard.mozilla.org/r/98336/#review98640

Looks good -- I sent it to auto-land.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e06c085d1397
Make sure that different docker-image tasks use different job symbols; r=dustin

Comment 14

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/e06c085d1397

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dustin, can we get this uplifted to aurora so it will hit the next ESR release? Would be great given that we will support it for over a year.

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

If you'd like, that's fine -- but how useful will this be during that time?  Will ESR be generating new docker images?

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Frankly, I don't have an answer to this question. But if the patch applies fine on current aurora why we shouldn't try? In case of merge conflicts we could still say no.

Comment 18

[Carsten Book [:Tomcat]]

doesn't this need relman approval ?

Comment 19

[Ryan VanderMeulen [:RyanVM]]

It's NPOTB, so no.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/55bfc9ee4482

Comment 21

[Dustin J. Mitchell [:dustin] (he/him)]

Why not uplift everything that lands in central to aurora, if it applies without conflicts?  I'm just confused why this came up.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Currently mozilla-aurora will become our next ESR release which will stay around for another year. That was the main reason why I asked for an uplift. In that case it helps a lot to see the different types of docker jobs when investigating issues as what I noticed the last months. Generally we do not want to uplift everything.