[try-stage] Configure tasks by release type not branch.

RESOLVED FIXED

Status

defect
RESOLVED FIXED
9 months ago
6 months ago

People

(Reporter: tomprince, Assigned: rjl)

Tracking

(Blocks 1 bug)

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr60 fixed)

Details

Attachments

(11 attachments, 3 obsolete attachments)

46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
10.56 KB, patch
tomprince
: review+
Details | Diff | Splinter Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
13.94 KB, patch
tomprince
: review+
Details | Diff | Splinter Review
5.84 KB, patch
tomprince
: feedback+
Details | Diff | Splinter Review
Reporter

Description

9 months ago
Staging releases want to behave like a release on another branch. Switch the in-tree config to key off of release type, and set the that per-branch and allow configuring it on try.
Reporter

Updated

9 months ago
Keywords: leave-open
Comment on attachment 9007876 [details]
Bug 1490094: [taskgraph] Remove some dead code refering to `release_type`; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9007876 - Flags: review+

Comment 3

9 months ago
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/93b5503a50b0
[taskgraph] Remove some dead code refering to `release_type`; r=bhearsum
Reporter

Comment 5

9 months ago
Currently, release resources such as balrog, bouncer and CDN urls are
configured by project. However, all production branches use one value and all
other branches use another. Rather than duplicate lists of projects, add a
helper to identify production release branches.

Comment 6

9 months ago
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/3748dbd2539c
[release] Configure staging release resources via dedicated parameter; r=bhearsum
Comment on attachment 9008188 [details]
Bug 1490094: [release] Configure staging release resources via dedicated parameter; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008188 - Flags: review+

Comment 9

9 months ago
Posted patch 1490094-C-C-part.patch (obsolete) — Splinter Review
Assignee: nobody → jorgk
Attachment #9008345 - Flags: review?(mozilla)

Comment 10

9 months ago
Comment on attachment 9008345 [details] [diff] [review]
1490094-C-C-part.patch

That's all I can do via pattern matching :-(
Attachment #9008345 - Flags: feedback?(rob)

Comment 11

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/21c798a31bdf
Port bug 1490094: [release] Configure staging release resources via dedicated parameter. rs=bustage-fix

Comment 12

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/4658bcf16f8d
Port bug 1490094: Follow-up: Fix indentation issues. rs=white-space-only DONTBUILD
Assignee

Comment 13

9 months ago
Comment on attachment 9008345 [details] [diff] [review]
1490094-C-C-part.patch

Review of attachment 9008345 [details] [diff] [review]:
-----------------------------------------------------------------

There's a number of places where by-project has been changed to by-release-level, yet we're still referring to comm-(beta|esr.*). I suspect those should be staging based on the M-C changes.
I don't think I flagged them all.

::: taskcluster/ci/release-bouncer-aliases/kind.yml
@@ +19,1 @@
>              comm-(beta|esr.*): scriptworker-prov-v1/tb-bouncer-v1

should this be "staging" now?

::: taskcluster/ci/release-bouncer-sub/kind.yml
@@ +15,1 @@
>              comm-(beta|esr.*): scriptworker-prov-v1/tb-bouncer-v1

staging?

::: taskcluster/ci/release-generate-checksums/kind.yml
@@ +65,1 @@
>                      comm-(beta|esr.*):

production?

@@ +65,4 @@
>                      comm-(beta|esr.*):
>                          stage_product: "thunderbird"
>                          bucket_name: "net-mozaws-prod-delivery-archive"
> +                    production:

staging?

::: taskcluster/ci/release-mark-as-shipped/kind.yml
@@ +21,2 @@
>              comm-(beta|esr.*): scriptworker-prov-v1/tb-shipit-v1
> +            production: scriptworker-prov-v1/tb-shipit-dev

production
staging

@@ +27,1 @@
>              comm-(beta|esr.*): ['project:comm:thunderbird:releng:ship-it:production']

production
staging

::: taskcluster/ci/release-update-verify-config/kind.yml
@@ +33,1 @@
>                  try-comm-central: "http://ftp.stage.mozaws.net/pub"

staging
should there be a default on these?

@@ +37,1 @@
>                  try-comm-central: "https://archive.mozilla.org/pub"

staging

@@ +41,1 @@
>                  try-comm-central: "https://aus5.stage.mozaws.net"

staging
Attachment #9008345 - Flags: feedback?(rob) → feedback-

Comment 14

9 months ago
(In reply to Rob Lemley [:rjl] from comment #13)
> There's a number of places where by-project has been changed to
> by-release-level, yet we're still referring to comm-(beta|esr.*). I suspect
> those should be staging based on the M-C changes.
I was wondering about that. But the M-C changes had nothing of the form
  try:
or
  mozilla-(beta|esr.*)

Anyway, the damage is done and the patch has landed. It got Dailies going. So backing it out is not an option. I'll leave it for you to fix ;-)

Comment 15

9 months ago
BTW, I'm happy to have this backed out completely if you want to start from scratch.
Reporter

Comment 16

9 months ago
Comment on attachment 9008345 [details] [diff] [review]
1490094-C-C-part.patch

Review of attachment 9008345 [details] [diff] [review]:
-----------------------------------------------------------------

The only valid values for `release-level` are `production` and `staging`. `production` corresponds to any of comm-{central,beta,esr60} whereas `staging` corresponds to `try-comm-central`.
Attachment #9008345 - Flags: review?(mozilla) → review-

Comment 17

9 months ago
Thanks. I hope our build/release engineer will now take care of this.
Flags: needinfo?(rob)

Comment 18

9 months ago
Comment on attachment 9008345 [details] [diff] [review]
1490094-C-C-part.patch

Thanks for the clarification, Tom. I'll land a follow-up to fix the release levels. I've considered backing the previous patches out, but that would just be unnecessary churn to correct a few lines.
Flags: needinfo?(rob)

Comment 19

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/e4373d203bcc
Follow-up: fix incorrect release levels. r=me DONTBUILD

Comment 20

9 months ago
Here's the sum of what I landed if you want to take another look.
Attachment #9008648 - Flags: review?(mozilla)
Attachment #9008803 - Attachment description: Bug 1490094: Add a transform to skip tasks by release type. → Bug 1490094: [taskgraph] Add a transform to skip tasks by release type; r?bhearsum
Reporter

Comment 23

9 months ago
Currently, release resources such as balrog, bouncer and CDN urls are
configured by project. However, all production branches use one value and all
other branches use another. Rather than duplicate lists of projects, add a
helper to identify production release branches.
Reporter

Comment 24

9 months ago
These tasks only run on rc release promotion steps, so there is no need to very
the configuration per-project.

Updated

9 months ago
Assignee: jorgk → mozilla
Reporter

Comment 28

9 months ago
Comment on attachment 9008648 [details] [diff] [review]
folded-patch-for-review.patch

Review of attachment 9008648 [details] [diff] [review]:
-----------------------------------------------------------------

These changes all look correct.
Attachment #9008648 - Flags: review?(mozilla) → review+

Comment 29

9 months ago
Comment on attachment 9008345 [details] [diff] [review]
1490094-C-C-part.patch

Problems of this patch amended in follow-up, landed in comment #19.
Attachment #9008345 - Attachment is obsolete: true
Comment on attachment 9008823 [details]
Bug 1490094: [release] Remove per-project secondary update verify config variation; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008823 - Flags: review+
Comment on attachment 9008822 [details]
Bug 1490094: [release] Configure staging release resources via dedicated parameter; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008822 - Flags: review+
Attachment #9008822 - Attachment is obsolete: true
Comment on attachment 9008803 [details]
Bug 1490094: [taskgraph] Add a transform to skip tasks by release type; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008803 - Flags: review+
Comment on attachment 9008820 [details]
Bug 1490094: [taskgraph] Change `release_type` to include more than just `rc`; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008820 - Flags: review+
Comment on attachment 9008824 [details]
Bug 1490094: [release] Use release_type to configure update-verify-config tasks; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008824 - Flags: review+
Comment on attachment 9008825 [details]
Bug 1490094: [release] Use release_type to configure update-verify tasks; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008825 - Flags: review+
Comment on attachment 9008826 [details]
Bug 1490094: [release] Use release_type to configure balrog tasks; r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9008826 - Flags: review+

Comment 37

9 months ago
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/9f2efc6828d5
[taskgraph] Add a transform to skip tasks by release type; r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/f345b131a734
[taskgraph] Change `release_type` to include more than just `rc`; r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/9cb31823dcd9
[release] Remove per-project secondary update verify config variation; r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/708cf763c173
[release] Use release_type to configure update-verify-config tasks; r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/5070b8f0149a
[release] Use release_type to configure update-verify tasks; r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/5749fce97426
[release] Use release_type to configure balrog tasks; r=bhearsum

Comment 39

9 months ago
This appears to be working, I did my best matching the patterns. Most likely this will need a follow-up. I added 'default' eight times where the FF files didn't have it, I'll point it out in the next comment.

Ideally of course we'd prepare a patch and have it reviewed and approved before the M-C patch even hits autoland/inbound.
Attachment #9009885 - Flags: review?(mozilla)

Comment 40

9 months ago
Comment on attachment 9009885 [details] [diff] [review]
C-C patch for porting changesets landed on 2018-09-18

Review of attachment 9009885 [details] [diff] [review]:
-----------------------------------------------------------------

::: taskcluster/ci/release-update-verify-config/kind.yml
@@ +49,5 @@
> +            by-release-type:
> +                beta: beta
> +                esr60: nonbeta
> +                nightly: beta
> +                default: null

(1) Added this.

@@ +62,1 @@
>                  default: null

(2) I believe this was removed in the FF changesets.

@@ +79,5 @@
> +                by-release-type:
> +                    beta: "beta-localtest"
> +                    esr60: "release-localtest"
> +                    nightly: "default"
> +                    default: null

(3-7): Added this five times for the five platforms.

::: taskcluster/ci/release-update-verify/kind.yml
@@ +43,1 @@
>                      default: "default"

(8) I believe this was removed in the FF changesets.

Comment 41

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/03d79b0cf048
Port changesets from bug 1490094 and bug 1487172. rs=bustage-fix
Reporter

Comment 42

9 months ago
Comment on attachment 9009885 [details] [diff] [review]
C-C patch for porting changesets landed on 2018-09-18

Review of attachment 9009885 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good. There are a couple of changes needed for the staging balrog instances (where the rules don't line up with production). I had these values in my try pushes, but hadn't pushed them anywhere else.

::: taskcluster/ci/release-balrog-scheduling/kind.yml
@@ +36,5 @@
> +                            default: []
> +                    staging:
> +                        by-release-type:
> +                            beta: [43]
> +                            esr60: [820]

This should be 729.

::: taskcluster/ci/release-balrog-submit-toplevel/kind.yml
@@ +47,5 @@
> +                            default: []
> +                    staging:
> +                        by-release-type:
> +                            beta: [43]
> +                            esr60: [820]

This should be 729.
Attachment #9009885 - Flags: review?(mozilla) → review+

Comment 43

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/743a00e211a5
Follow-up: Correct balrog rule numbers for Thunderbird's 'staging'. r=me
Assignee

Updated

9 months ago
Assignee: mozilla → rob
Status: NEW → ASSIGNED
Assignee

Updated

9 months ago
Attachment #9010974 - Flags: review?(mozilla)
Reporter

Comment 45

9 months ago
Comment on attachment 9010974 [details] [diff] [review]
Follow-up: Change release-type from esr60 to release

Review of attachment 9010974 [details] [diff] [review]:
-----------------------------------------------------------------

This looks sensible.

::: taskcluster/ci/release-update-verify-config/kind.yml
@@ +59,5 @@
>                          default: "56.0b3"
> +                release:
> +                    by-platform:
> +                        win64.*: null
> +                        default: "52.0"

I can't see the context for this, but I'd guess that win64 probably wants to have 60.0 as a watershed.
Attachment #9010974 - Flags: review?(mozilla)
Assignee

Comment 46

9 months ago
Yeah, win64 can be set to 60.0. I'll make the necessary change.
Assignee

Comment 47

9 months ago
Patch updated to set win64 watershed version to 60.0.

Thunderbird does not use "esr.*" as a release type.
See https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/decision.py#l95
Attachment #9011036 - Flags: review?(mozilla)
Assignee

Updated

9 months ago
Attachment #9010974 - Attachment is obsolete: true
Reporter

Updated

9 months ago
Attachment #9011036 - Flags: review?(mozilla) → feedback+

Comment 48

9 months ago
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/ec9dbe9537af
Follow-up: Change release-type from esr60 to release. f=tomprince, rs=jorgk
Reporter

Updated

7 months ago
Status: ASSIGNED → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.