Closed Bug 1341352 Opened 3 years ago Closed 3 years ago

[tcmigration] add balrogscript support for releases

Categories

(Release Engineering :: Release Automation: Other, defect, P2)

defect

Tracking

(firefox55 fixed)

RESOLVED FIXED
Tracking Status
firefox55 --- fixed

People

(Reporter: mtabara, Assigned: mtabara)

References

()

Details

Attachments

(2 files)

As part of the TC migration train ride towards beta/release, we need to extend balrogscript[1] functionality to include the proper bits (at least) for:

* scopes
* channels

This bug is tracking this work for [1].

[1]: https://github.com/mozilla-releng/balrogscript
Priority: -- → P2
Summary: add balrogscript support for releases → [tcmigration] add balrogscript support for releases
Comment on attachment 8842024 [details]
Bug 1341352 - add balrogworker support for releases.

https://reviewboard.mozilla.org/r/116042/#review117468

You're getting good at this!
Attachment #8842024 - Flags: review?(aki) → review+
https://github.com/mozilla-releng/scriptworker/commit/be23980c6e2ff4e1aff8aa0bc35c88dc7d8ae8e4
Bug 1341352 - cleanup/add new format for balrog scopes. r=aki 

https://github.com/mozilla-releng/balrogscript/commit/b4b4d8e7c6281f669e092fd45413ec994c495321
Bug 1341352 - add balrogscript support for releases. r=aki 

https://hg.mozilla.org/build/puppet/rev/01dda7e6ff84
Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki

Leftovers:
* wait for trees to open so that https://reviewboard.mozilla.org/r/116042/diff/1/ gets autolanded 
* munging the scopes in https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3 to make the new scopes available in the hook and delete the old ones.
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #4)
> https://github.com/mozilla-releng/scriptworker/commit/
> be23980c6e2ff4e1aff8aa0bc35c88dc7d8ae8e4
> Bug 1341352 - cleanup/add new format for balrog scopes. r=aki 
> 
> https://github.com/mozilla-releng/balrogscript/commit/
> b4b4d8e7c6281f669e092fd45413ec994c495321
> Bug 1341352 - add balrogscript support for releases. r=aki 
> 
> https://hg.mozilla.org/build/puppet/rev/01dda7e6ff84
> Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki
> 
> Leftovers:
> * wait for trees to open so that
> https://reviewboard.mozilla.org/r/116042/diff/1/ gets autolanded 
> * munging the scopes in
> https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3 to
> make the new scopes available in the hook and delete the old ones.

Backed-out puppet patch[1] until we're in a more "ready" state to rollout the balrog changes - 
KWierso was kind to offer to backout the changes from central if indeed the in-tree patch landed in autoland (not known yet due to tree being closed)

[1]: https://hg.mozilla.org/build/puppet/rev/cc3ffd29de08
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7478b9959a51
add balrogworker support for releases. r=aki
Backout by kwierso@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/3cb6398d924b
Backed out changeset 7478b9959a51 at developer's request a=backout
Okay, we've postponed this long enough; With Fennec beta in good shape and nightlies looking in shape for a good while now, I think it's safe to resurrect these patches.

Things we need to land:

0. tweak hooks/roles to take into account the new scopes

1. in-tree patch to alter the balrog tasks scopes: rebase against central + land to inbound - https://reviewboard.mozilla.org/r/116042/diff/1#index_header

2. amend scriptworker scopes under scriptworker/constants:
PR already deployed to do this - https://github.com/mozilla-releng/scriptworker/pull/85

3. amend balrogscript to take into account the new scopes
PR already deployed but not yet bumped - https://github.com/mozilla-releng/balrogscript/commit/02f8f12a6deab036ecd3a22a19c48418b342a886
PR to adapt the new release blob string "tc_release" - https://github.com/mozilla-releng/balrogscript/pull/20

4. amend puppet to bump balrogscript to newest version and also account for this change - https://github.com/mozilla/build-puppet/pull/31/files


After migration is complete:
1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes.

2. cleanup old scopes from hooks/roles

:aki: does this sound good? Am I missing something?
Flags: needinfo?(aki)
I think so. It's been a while, so I've lost context as well :)

https://github.com/mozilla-releng/balrogscript/pull/20 only removes old release support, and doesn't add it back in, correct?
Flags: needinfo?(aki)
(In reply to Aki Sasaki [:aki] from comment #9)
> I think so. It's been a while, so I've lost context as well :)

Heh, I feel you :)

> https://github.com/mozilla-releng/balrogscript/pull/20 only removes old
> release support, and doesn't add it back in, correct?

Yes. It doesn't add it back. That is because we need to rewrite that piece anyway when the time comes. However, I realized we won't be using it anyway for now, nor for Fennec (deactivated anyway), nor for Firefox (done via relpro anyway). So I just cleaned that code. I think we'll need to revisit that piece when we'll have funsize into the tree and also the tasks that are updating the channel rules. Until then, to be honest, we don't need any channel munging anyway either but since it's added, it won't hurt.
Okay, enough is enough :) I'll wait for GtB for today's beta, following which I'll start deploying gradually the leftovers.
As soon as the beta starts today, I'll land all the patches. Doing so to prevent breaking the scopes/CI builds before I go to bed and leave the mess for others to deal with.
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5a8eea2b095e
add balrogworker support for releases. r=aki a=release DONTBUILD"
(In reply to Pulsebot from comment #13)
> Pushed by mtabara@mozilla.com:
> https://hg.mozilla.org/integration/mozilla-inbound/rev/5a8eea2b095e
> add balrogworker support for releases. r=aki a=release DONTBUILD"

Okay. Both betas (desktop + mobile) are in good shape. 
I pushed the in-tree balrog changes to inbound.
As soon as that lands on central, I need to uplift the code to: aurora, beta, jamun.

After that, leftovers are:

2. amend scriptworker scopes under scriptworker/constants:
PR already deployed to do this - https://github.com/mozilla-releng/scriptworker/pull/85

3. amend balrogscript to take into account the new scopes
PR already deployed but not yet bumped - https://github.com/mozilla-releng/balrogscript/commit/02f8f12a6deab036ecd3a22a19c48418b342a886
PR to adapt the new release blob string "tc_release" - https://github.com/mozilla-releng/balrogscript/pull/20

4. amend puppet to bump balrogscript to newest version and also account for this change - https://github.com/mozilla/build-puppet/pull/31/files


After migration is complete:
1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes.

2. cleanup old scopes from hooks/roles

Also, it's a bit unclear to me as to which hooks will fail due to scopes missing but we'll find out soon.
Augment

1. https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta

to also encompass:
project:releng:balrog:server:nightly
project:releng:balrog:server:beta
project:releng:balrog:channel:nightly
project:releng:balrog:channel:beta
project:releng:balrog:channel:beta-localtest
project:releng:balrog:channel:beta-cdntest


2. https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:*

to also encompass

project:releng:balrog:server:nightly
project:releng:balrog:server:aurora
project:releng:balrog:channel:nightly
project:releng:balrog:channel:aurora

Once all this works and nightlies + beta are confirmed, we need to:

1. Remove from https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta
project:releng:balrog:dep
project:releng:balrog:nightly

2. https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:*
project:releng:balrog:dep
project:releng:balrog:nightly

Dropping a NI to myself here to keep this in mind.
P.S. Some dev roles might need some similar tweaking but we'll see where this goes.
Flags: needinfo?(mtabara)
Status update:
1. Hooks/roles amended. 
2. in-tree patch landed to inbound - pending its merge to central.
3. scriptworker - nothing to do, stuff already landed a while ago
4. landed both balrogscript changes within 0.1.0 version of it
5. prepared patch to land in puppet as soon as in-tree patch merges to central
https://hg.mozilla.org/mozilla-central/rev/5a8eea2b095e
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Comment on attachment 8854414 [details] [diff] [review]
Bug 1341352 - add balrogscript support for releases - aurora uplift patch.

I only skimmed to see if it looks substantially different than the central landing, it doesn't.
Attachment #8854414 - Flags: review?(bugspam.Callek) → review+
https://hg.mozilla.org/build/puppet/rev/0c1ba02eb64e
bump balrogscript in balrogworker r=trivial

https://hg.mozilla.org/build/puppet/rev/a7f5c000ed1d
Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki
(In reply to Mihai Tabara [:mtabara]⌚️GMT+8 from comment #21)
> https://hg.mozilla.org/build/puppet/rev/0c1ba02eb64e
> bump balrogscript in balrogworker r=trivial
> 
> https://hg.mozilla.org/build/puppet/rev/a7f5c000ed1d
> Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki

Updated the balrogworkers to encompass these changes.
Status update - all pieces are in place. Now we need the newly set of nightlies generated (with new scopes, so respinning old balrog jobs won't work) to see if all this works smoothly.

Once we confirm the transition has been made successfully, the cleanup encompasses:

1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes.

2. cleanup old scopes from hooks/roles

a) Remove from https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta
project:releng:balrog:dep
project:releng:balrog:nightly

b) https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:*
project:releng:balrog:dep
project:releng:balrog:nightly

I think that would be it. Then PROFIT! :)
Flags: needinfo?(mtabara)
See Also: → 1353642
See Also: → 1353671
See Also: → 1353949
Both aurora/central nightlies look good. I think we're good to go with the cleanup.

(In reply to Mihai Tabara [:mtabara]⌚️GMT+8 from comment #23)
> 2. cleanup old scopes from hooks/roles
> 
> a) Remove from
> https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta
> project:releng:balrog:dep
> project:releng:balrog:nightly
> 
> b) https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:*
> project:releng:balrog:dep
> project:releng:balrog:nightly
> 

This is done now.

Also:
1) In addition, I've also removed:
project:releng:balrog:dep
project:releng:balrog:nightly

and add instead:
project:releng:balrog:server:nightly
project:releng:balrog:channel:nightly

for https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fcandidates-fennec-dev

2) In addition, I've also removed:
project:releng:balrog:dep
project:releng:balrog:nightly

for: (
https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fnightly-desktop-dev 
https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fnightly-fennec-dev
https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:mozilla-beta
)
The transition is now complete and cleanup done.
https://github.com/mozilla-releng/scriptworker/pull/102 is to be merged and picked-up by a future scriptworker bump.

Will reopen if anything else pops out intermitently.
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.