Closed Bug 1338186 Opened 7 years ago Closed 7 years ago

[tcmigration] add beetmoverworker support for releases

Categories

(Release Engineering :: Release Automation: Other, defect, P2)

defect

Tracking

(firefox54 fixed)

RESOLVED FIXED
Tracking Status
firefox54 --- fixed

People

(Reporter: mtabara, Assigned: mtabara)

References

()

Details

Attachments

(3 files, 1 obsolete file)

As part of the TC migration train ride towards beta/release, we need to extend beetmoverscript[1] functionality to include the proper bits (at least) for:

* scopes
* mapping manifests
* S3 creds

This bug is tracking this work for [1].

[1]: https://github.com/mozilla-releng/beetmoverscript/
Status: NEW → ASSIGNED
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #1)
> Created attachment 8839188 [details]
> Bug 1338186 - add staging creds in beetmoverworker.
> 
> Review commit: https://reviewboard.mozilla.org/r/113894/diff/#index_header
> See other reviews: https://reviewboard.mozilla.org/r/113894/

Adding some missing staging creds in beetmoverworker,a leftover from a few months ago. Useful in preparation for testing Fennec betas off of jamun.
Comment on attachment 8839188 [details]
Bug 1338186 - make beetmoverworker deal with releases.

https://reviewboard.mozilla.org/r/113894/#review115460
Attachment #8839188 - Flags: review?(aki) → review+
Comment on attachment 8839188 [details]
Bug 1338186 - make beetmoverworker deal with releases.

https://hg.mozilla.org/build/puppet/rev/5f51520fb68a
Attachment #8839188 - Flags: checked-in+
Summary: add beetmoverscript support for releases → [tcmigration] add beetmoverscript support for releases
Priority: -- → P2
Changes for beetmoverscript are tracked in https://github.com/mozilla-releng/beetmoverscript/pull/51.
Currently shipping a 0.3.0-b1 python package out of it.
Attachment #8839188 - Attachment is obsolete: true
Attachment #8839188 - Flags: review?(rail)
Attachment #8840437 - Flags: review?(rail)
Comment on attachment 8839188 [details]
Bug 1338186 - make beetmoverworker deal with releases.

https://reviewboard.mozilla.org/r/113894/#review116382

syntax check passed :)
Attachment #8839188 - Flags: review+
Comment on attachment 8840437 [details] [diff] [review]
Bug 1338186 - make beetmoverworker deal with releases. r=rail

r+-ed by Rail in the other obsolete patch in the reviewboard.

Landed:
https://hg.mozilla.org/build/puppet/rev/712832941278
Bug 1338186 - make beetmoverworker deal with releases. r=rail
Attachment #8840437 - Flags: review?(rail) → review+
Comment on attachment 8840437 [details] [diff] [review]
Bug 1338186 - make beetmoverworker deal with releases. r=rail

BTW, it looked the same in reviewboard.
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #10)
> Comment on attachment 8840437 [details] [diff] [review]
> Bug 1338186 - make beetmoverworker deal with releases. r=rail
> 
> BTW, it looked the same in reviewboard.

Yeah, noticed that afterwards. However, I wanted to have two separate patches, nicely listed at the top of the bug. But I guess reviewboard only supports one bug / repository when you submit. TIL :)
Shipped 0.3.0-b1 beetmoverscript with support for releases.
Backward compatibility:
* works fine with linux l10n job - successfully rerun https://tools.taskcluster.net/task-inspector/#CNQbhLTURSSSjXARJQr83Q/1
* works fine with linux signing job - successfully rerun https://tools.taskcluster.net/task-inspector/#LMOHDRgGS8e8Ren3QD5-yQ/1
* works fine with android-api-15 l10n job - successfully rerun https://public-artifacts.taskcluster.net/cBwIoGZWSS-Jqd6zCHgAzQ/1/public/logs/live_backing.log
* works fine with android-x86 too
* works fine with checksums and test_packages.json content altering.
Summary: [tcmigration] add beetmoverscript support for releases → [tcmigration] add beetmoverworker support for releases
Comment on attachment 8840648 [details]
Bug 1338186 - assign bucket specific scopes per type of release.

https://reviewboard.mozilla.org/r/115098/#review116586

::: taskcluster/taskgraph/transforms/beetmover.py:190
(Diff revision 1)
> +            'beta': 'push-to-candidates',
> +            'release': 'push-to-candidates',
> +        }
> +        for target, action in BEETMOVER_TARGET_ACTIONS.items():
> +            if target in config.params['target_tasks_method']:
> +                action_scopes.append(base_action_scope.format(action))

Hm.

The main thing I don't like here is `if target in`, because that's imprecise and may fuzzy match.

What do you think about adding a `BEETMOVER_ACTION_SCOPES` [like this](https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/util/scriptworker.py#l59) ? and a `get_beetmover_action_scope` [like this](https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/util/scriptworker.py#l122) ?

::: taskcluster/taskgraph/transforms/beetmover_checksums.py:60
(Diff revision 1)
> +            'beta': 'push-to-candidates',
> +            'release': 'push-to-candidates',
> +        }
> +        for target, action in BEETMOVER_TARGET_ACTIONS.items():
> +            if target in config.params['target_tasks_method']:
> +                action_scopes.append(base_action_scope.format(action))

`get_beetmover_action_scope` would also allow you to reuse the logic via an import, rather than duplicating the logic in two places.
Attachment #8840648 - Flags: review?(aki) → review-
Comment on attachment 8840648 [details]
Bug 1338186 - assign bucket specific scopes per type of release.

https://reviewboard.mozilla.org/r/115098/#review116784

Awesome.  We may have to adjust these as things change, but we now have the framework to do that.  Thanks!
Attachment #8840648 - Flags: review?(aki) → review+
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7a651efeb9e8
assign bucket specific scopes per type of release. r=aki
https://hg.mozilla.org/mozilla-central/rev/7a651efeb9e8
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Merged https://github.com/mozilla-releng/beetmoverscript/pull/53
Bug 1338186 - revert the hacking to use real release/nightly scopes now.r=aki

Merged to master, didn't bump it via package. Will do so tomorrow after I graft https://hg.mozilla.org/mozilla-central/rev/7a651efeb9e8 to aurora and jamun.
Blocks: 1342952
Uplifting the change to aurora/jamun as well - that means that for a few hours we'll propagate the error there as well. But as soon as the incoming fix will land in central, we'll fix both of the nightlies (central/aurora) in the same time.

https://hg.mozilla.org/releases/mozilla-aurora/rev/3cdcc34a1693
In https://tools.taskcluster.net/auth/roles/#project:releng:nightly:level-3:* I did the following:
> removed "project:releng:beetmover:nightly" 
> added "project:releng:beetmover:bucket:nightly"
> added "project:releng:beetmover:action:push-to-nightly"
Comment on attachment 8841664 [details]
Bug 1338186 - tweak the bucket scope to new form. r=aki

r=me if you also update the all-release-branches and default versions to project:releng:beetmover:bucket:release and project:releng:beetmover:bucket:dep, respectively.
Attachment #8841664 - Flags: review?(aki) → review+
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e00f58c31a85
tweak the bucket scope to new form. r=aki a=release DONTBUILD
Pushed by kwierso@gmail.com:
https://hg.mozilla.org/mozilla-central/rev/cc0d34f0490c
tweak the bucket scope to new form. r=aki a=release DONTBUILD
https://hg.mozilla.org/build/puppet/rev/06236a525e31
Bug 1338186 - bump beeetmoverscript in beetmoverworker. r=trivial
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #26)
> https://hg.mozilla.org/build/puppet/rev/06236a525e31
> Bug 1338186 - bump beeetmoverscript in beetmoverworker. r=trivial

Landed on production - https://hg.mozilla.org/build/puppet/rev/5cc31756afd8
Updated both instances of beetmoverworkers.
No longer blocks: 1342952
I think it's safe to close this now as we've gotten ourselves a successful Fennec staging 53.0b1.
If there's more stuff needed in beetmoverscript, we'll put bugs on file hanging off bug 1343393.
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.