Closed
Bug 1341352
Opened 7 years ago
Closed 7 years ago
[tcmigration] add balrogscript support for releases
Categories
(Release Engineering :: Release Automation: Other, defect, P2)
Release Engineering
Release Automation: Other
Tracking
(firefox55 fixed)
RESOLVED
FIXED
Tracking | Status | |
---|---|---|
firefox55 | --- | fixed |
People
(Reporter: mtabara, Assigned: mtabara)
References
()
Details
Attachments
(2 files)
As part of the TC migration train ride towards beta/release, we need to extend balrogscript[1] functionality to include the proper bits (at least) for: * scopes * channels This bug is tracking this work for [1]. [1]: https://github.com/mozilla-releng/balrogscript
Updated•7 years ago
|
Priority: -- → P2
Assignee | ||
Updated•7 years ago
|
Summary: add balrogscript support for releases → [tcmigration] add balrogscript support for releases
Comment hidden (mozreview-request) |
Assignee | ||
Comment 2•7 years ago
|
||
My initial approach to fix balrogscript to handle releases contains: * r? from above * https://github.com/mozilla-releng/scriptworker/pull/85 * https://github.com/mozilla/build-puppet/pull/31 * https://github.com/mozilla-releng/balrogscript/pull/18 + alteering scopes in https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:*
Comment 3•7 years ago
|
||
mozreview-review |
Comment on attachment 8842024 [details] Bug 1341352 - add balrogworker support for releases. https://reviewboard.mozilla.org/r/116042/#review117468 You're getting good at this!
Attachment #8842024 -
Flags: review?(aki) → review+
Assignee | ||
Comment 4•7 years ago
|
||
https://github.com/mozilla-releng/scriptworker/commit/be23980c6e2ff4e1aff8aa0bc35c88dc7d8ae8e4 Bug 1341352 - cleanup/add new format for balrog scopes. r=aki https://github.com/mozilla-releng/balrogscript/commit/b4b4d8e7c6281f669e092fd45413ec994c495321 Bug 1341352 - add balrogscript support for releases. r=aki https://hg.mozilla.org/build/puppet/rev/01dda7e6ff84 Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki Leftovers: * wait for trees to open so that https://reviewboard.mozilla.org/r/116042/diff/1/ gets autolanded * munging the scopes in https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3 to make the new scopes available in the hook and delete the old ones.
Assignee | ||
Comment 5•7 years ago
|
||
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #4) > https://github.com/mozilla-releng/scriptworker/commit/ > be23980c6e2ff4e1aff8aa0bc35c88dc7d8ae8e4 > Bug 1341352 - cleanup/add new format for balrog scopes. r=aki > > https://github.com/mozilla-releng/balrogscript/commit/ > b4b4d8e7c6281f669e092fd45413ec994c495321 > Bug 1341352 - add balrogscript support for releases. r=aki > > https://hg.mozilla.org/build/puppet/rev/01dda7e6ff84 > Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki > > Leftovers: > * wait for trees to open so that > https://reviewboard.mozilla.org/r/116042/diff/1/ gets autolanded > * munging the scopes in > https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3 to > make the new scopes available in the hook and delete the old ones. Backed-out puppet patch[1] until we're in a more "ready" state to rollout the balrog changes - KWierso was kind to offer to backout the changes from central if indeed the in-tree patch landed in autoland (not known yet due to tree being closed) [1]: https://hg.mozilla.org/build/puppet/rev/cc3ffd29de08
Pushed by mtabara@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7478b9959a51 add balrogworker support for releases. r=aki
Backout by kwierso@gmail.com: https://hg.mozilla.org/integration/autoland/rev/3cb6398d924b Backed out changeset 7478b9959a51 at developer's request a=backout
Assignee | ||
Comment 8•7 years ago
|
||
Okay, we've postponed this long enough; With Fennec beta in good shape and nightlies looking in shape for a good while now, I think it's safe to resurrect these patches. Things we need to land: 0. tweak hooks/roles to take into account the new scopes 1. in-tree patch to alter the balrog tasks scopes: rebase against central + land to inbound - https://reviewboard.mozilla.org/r/116042/diff/1#index_header 2. amend scriptworker scopes under scriptworker/constants: PR already deployed to do this - https://github.com/mozilla-releng/scriptworker/pull/85 3. amend balrogscript to take into account the new scopes PR already deployed but not yet bumped - https://github.com/mozilla-releng/balrogscript/commit/02f8f12a6deab036ecd3a22a19c48418b342a886 PR to adapt the new release blob string "tc_release" - https://github.com/mozilla-releng/balrogscript/pull/20 4. amend puppet to bump balrogscript to newest version and also account for this change - https://github.com/mozilla/build-puppet/pull/31/files After migration is complete: 1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes. 2. cleanup old scopes from hooks/roles :aki: does this sound good? Am I missing something?
Flags: needinfo?(aki)
Comment 9•7 years ago
|
||
I think so. It's been a while, so I've lost context as well :) https://github.com/mozilla-releng/balrogscript/pull/20 only removes old release support, and doesn't add it back in, correct?
Flags: needinfo?(aki)
Assignee | ||
Comment 10•7 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #9) > I think so. It's been a while, so I've lost context as well :) Heh, I feel you :) > https://github.com/mozilla-releng/balrogscript/pull/20 only removes old > release support, and doesn't add it back in, correct? Yes. It doesn't add it back. That is because we need to rewrite that piece anyway when the time comes. However, I realized we won't be using it anyway for now, nor for Fennec (deactivated anyway), nor for Firefox (done via relpro anyway). So I just cleaned that code. I think we'll need to revisit that piece when we'll have funsize into the tree and also the tasks that are updating the channel rules. Until then, to be honest, we don't need any channel munging anyway either but since it's added, it won't hurt.
Assignee | ||
Comment 11•7 years ago
|
||
Okay, enough is enough :) I'll wait for GtB for today's beta, following which I'll start deploying gradually the leftovers.
Assignee | ||
Comment 12•7 years ago
|
||
As soon as the beta starts today, I'll land all the patches. Doing so to prevent breaking the scopes/CI builds before I go to bed and leave the mess for others to deal with.
Comment 13•7 years ago
|
||
Pushed by mtabara@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5a8eea2b095e add balrogworker support for releases. r=aki a=release DONTBUILD"
Assignee | ||
Comment 14•7 years ago
|
||
(In reply to Pulsebot from comment #13) > Pushed by mtabara@mozilla.com: > https://hg.mozilla.org/integration/mozilla-inbound/rev/5a8eea2b095e > add balrogworker support for releases. r=aki a=release DONTBUILD" Okay. Both betas (desktop + mobile) are in good shape. I pushed the in-tree balrog changes to inbound. As soon as that lands on central, I need to uplift the code to: aurora, beta, jamun. After that, leftovers are: 2. amend scriptworker scopes under scriptworker/constants: PR already deployed to do this - https://github.com/mozilla-releng/scriptworker/pull/85 3. amend balrogscript to take into account the new scopes PR already deployed but not yet bumped - https://github.com/mozilla-releng/balrogscript/commit/02f8f12a6deab036ecd3a22a19c48418b342a886 PR to adapt the new release blob string "tc_release" - https://github.com/mozilla-releng/balrogscript/pull/20 4. amend puppet to bump balrogscript to newest version and also account for this change - https://github.com/mozilla/build-puppet/pull/31/files After migration is complete: 1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes. 2. cleanup old scopes from hooks/roles Also, it's a bit unclear to me as to which hooks will fail due to scopes missing but we'll find out soon.
Assignee | ||
Comment 15•7 years ago
|
||
Augment 1. https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta to also encompass: project:releng:balrog:server:nightly project:releng:balrog:server:beta project:releng:balrog:channel:nightly project:releng:balrog:channel:beta project:releng:balrog:channel:beta-localtest project:releng:balrog:channel:beta-cdntest 2. https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:* to also encompass project:releng:balrog:server:nightly project:releng:balrog:server:aurora project:releng:balrog:channel:nightly project:releng:balrog:channel:aurora Once all this works and nightlies + beta are confirmed, we need to: 1. Remove from https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta project:releng:balrog:dep project:releng:balrog:nightly 2. https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:* project:releng:balrog:dep project:releng:balrog:nightly Dropping a NI to myself here to keep this in mind. P.S. Some dev roles might need some similar tweaking but we'll see where this goes.
Flags: needinfo?(mtabara)
Assignee | ||
Comment 16•7 years ago
|
||
Status update: 1. Hooks/roles amended. 2. in-tree patch landed to inbound - pending its merge to central. 3. scriptworker - nothing to do, stuff already landed a while ago 4. landed both balrogscript changes within 0.1.0 version of it 5. prepared patch to land in puppet as soon as in-tree patch merges to central
Comment 17•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/5a8eea2b095e
Assignee | ||
Updated•7 years ago
|
Assignee | ||
Comment 18•7 years ago
|
||
Attachment #8854414 -
Flags: review?(bugspam.Callek)
Comment 19•7 years ago
|
||
Comment on attachment 8854414 [details] [diff] [review] Bug 1341352 - add balrogscript support for releases - aurora uplift patch. I only skimmed to see if it looks substantially different than the central landing, it doesn't.
Attachment #8854414 -
Flags: review?(bugspam.Callek) → review+
Assignee | ||
Comment 20•7 years ago
|
||
Uplifted patch to aurora/beta https://hg.mozilla.org/releases/mozilla-aurora/rev/d19defd55fa5ebe0a3d5f991c93a5ddf38b39f06 https://hg.mozilla.org/releases/mozilla-beta/rev/d32025f923a3d4320c3b7eb061d72b6838e9ef7a
Assignee | ||
Comment 21•7 years ago
|
||
https://hg.mozilla.org/build/puppet/rev/0c1ba02eb64e bump balrogscript in balrogworker r=trivial https://hg.mozilla.org/build/puppet/rev/a7f5c000ed1d Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki
Assignee | ||
Comment 22•7 years ago
|
||
(In reply to Mihai Tabara [:mtabara]⌚️GMT+8 from comment #21) > https://hg.mozilla.org/build/puppet/rev/0c1ba02eb64e > bump balrogscript in balrogworker r=trivial > > https://hg.mozilla.org/build/puppet/rev/a7f5c000ed1d > Bug 1341352 - tweak balrogworker configuration for scopes per project. r=aki Updated the balrogworkers to encompass these changes.
Assignee | ||
Comment 23•7 years ago
|
||
Status update - all pieces are in place. Now we need the newly set of nightlies generated (with new scopes, so respinning old balrog jobs won't work) to see if all this works smoothly. Once we confirm the transition has been made successfully, the cleanup encompasses: 1. Deploy https://github.com/mozilla-releng/scriptworker/pull/102 to cleanup the old balrog-format scopes. 2. cleanup old scopes from hooks/roles a) Remove from https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta project:releng:balrog:dep project:releng:balrog:nightly b) https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:* project:releng:balrog:dep project:releng:balrog:nightly I think that would be it. Then PROFIT! :)
Flags: needinfo?(mtabara)
Assignee | ||
Comment 24•7 years ago
|
||
Both aurora/central nightlies look good. I think we're good to go with the cleanup. (In reply to Mihai Tabara [:mtabara]⌚️GMT+8 from comment #23) > 2. cleanup old scopes from hooks/roles > > a) Remove from > https://tools.taskcluster.net/auth/roles/#project:releng:release:mozilla-beta > project:releng:balrog:dep > project:releng:balrog:nightly > > b) https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:* > project:releng:balrog:dep > project:releng:balrog:nightly > This is done now. Also: 1) In addition, I've also removed: project:releng:balrog:dep project:releng:balrog:nightly and add instead: project:releng:balrog:server:nightly project:releng:balrog:channel:nightly for https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fcandidates-fennec-dev 2) In addition, I've also removed: project:releng:balrog:dep project:releng:balrog:nightly for: ( https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fnightly-desktop-dev https://tools.taskcluster.net/auth/roles/#hook-id:project-releng%252fnightly-fennec-dev https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:mozilla-beta )
Assignee | ||
Comment 25•7 years ago
|
||
The transition is now complete and cleanup done. https://github.com/mozilla-releng/scriptworker/pull/102 is to be merged and picked-up by a future scriptworker bump. Will reopen if anything else pops out intermitently.
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → FIXED
Comment 26•6 years ago
|
||
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in
before you can comment on or make changes to this bug.
Description
•