1339151 - Use different tasks for signed and unsigned bits in release promotion

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect, P1)

Description

[Rail Aliiev [:rail]]

After we switch to Tc based CI builds we have to use different tasks for signed (installers, MARs, etc) and unsigned (tests.zip, simbols, etC) bits in releasetasks.

The changes should ride the trains.

At least `task_for_revision()` https://hg.mozilla.org/build/tools/file/tip/lib/python/kickoff/__init__.py#l161 should change the behaviour to read the routes from somewhere else. As a possible solution we can use in-tree configs and checkout them run-time in releaserunner.

Comment 1

[Rail Aliiev [:rail]]

Attachment: releasetasks

Comment 2

[Rail Aliiev [:rail]]

Attachment: configs.diff

Comment 3

[Rail Aliiev [:rail]]

Attachment: tools.diff

Comment 4

[Rail Aliiev [:rail]]

This approach also requires splitting some beetmover configs.

Comment 5

[Jordan Lund (:jlund)]

Comment on attachment 8837168 [details] [diff] [review]
configs.diff

Review of attachment 8837168 [details] [diff] [review]:
-----------------------------------------------------------------

hm, so signed and unsigned routes are the same except for linux in jamun?

Comment 6

[Jordan Lund (:jlund)]

Comment on attachment 8837168 [details] [diff] [review]
configs.diff

Review of attachment 8837168 [details] [diff] [review]:
-----------------------------------------------------------------

ah so this is what we want until we actually change things. And that's why jamun is already different as this is your staging env?

Comment 7

[Jordan Lund (:jlund)]

Comment on attachment 8837169 [details] [diff] [review]
tools.diff

Review of attachment 8837169 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildfarm/release/release-runner.py
@@ +279,5 @@
>      # TODO: to be moved under kickoff soon, once new relpro sanity is in place
>      # bug 1282959
>      platforms = kwargs.get('en_US_config', {}).get('platforms', {})
>      for platform in platforms.keys():
> +        task_id = platforms.get(platform).get('signed_task_id', {})

I know this is existing logic but do we want to allow for no taskid and instead return an empty dict? Or should we fail here on key error?

::: lib/python/kickoff/build_status.py
@@ +70,5 @@
>                  # Tasks are always completed if they are referenced in the index
>                  # https://docs.taskcluster.net/reference/core/index
> +                # Assuming that the signed tasks are completed after their
> +                # unsigned counterparts
> +                task_id = signed_task_for_revision(

where is this method implemented?

Comment 8

[Rail Aliiev [:rail]]

(In reply to Jordan Lund (:jlund) from comment #5)
> hm, so signed and unsigned routes are the same except for linux in jamun?

Correct.

(In reply to Jordan Lund (:jlund) from comment #6)
> ah so this is what we want until we actually change things. And that's why
> jamun is already different as this is your staging env?

Correct.

(In reply to Jordan Lund (:jlund) from comment #7)
> >      for platform in platforms.keys():
> > +        task_id = platforms.get(platform).get('signed_task_id', {})
> 
> I know this is existing logic but do we want to allow for no taskid and
> instead return an empty dict? Or should we fail here on key error?

Good point. I'll change this.

 
> ::: lib/python/kickoff/build_status.py
> @@ +70,5 @@
> >                  # Tasks are always completed if they are referenced in the index
> >                  # https://docs.taskcluster.net/reference/core/index
> > +                # Assuming that the signed tasks are completed after their
> > +                # unsigned counterparts
> > +                task_id = signed_task_for_revision(
> 
> where is this method implemented?

Bah. That was my initial implementation. I'll fix that.

Thank you for the feedback!

Comment 9

[Rail Aliiev [:rail]]

Attachment: Bug 1339151 - Remove leadind "index." from routes

Review commit: https://reviewboard.mozilla.org/r/112758/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/112758/

Comment 10

[Rail Aliiev [:rail]]

I'd like to land the configs, so I can play with other patches on jamun/bm83

Comment 11

[Jordan Lund (:jlund)]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

https://reviewboard.mozilla.org/r/112758/#review114344

have fun!

Comment 12

[Rail Aliiev [:rail]]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

https://hg.mozilla.org/build/buildbot-configs/rev/fb545d14c97fa68d476ba334c1025f488a76f9f4

Comment 13

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/fb545d14c97f

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/112758/diff/1-2/

Comment 15

[Aki Sasaki (not active)]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

https://reviewboard.mozilla.org/r/112758/#review114982

*stamp*

Comment 16

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/aaf7f4db57ff

Comment 17

[Rail Aliiev [:rail]]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/112758/diff/2-3/

Comment 18

[Aki Sasaki (not active)]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

https://reviewboard.mozilla.org/r/112758/#review117190

Comment 19

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/de82546851ec

Comment 20

[Rail Aliiev [:rail]]

Comment on attachment 8837709 [details]
Bug 1339151 - Remove leadind "index." from routes

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/112758/diff/3-4/

Comment 21

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/931d94078878