Closed Bug 1343990 Opened 7 years ago Closed 7 years ago

releasetasks should trigger second graph with candidates -> releases and post-release tasks

Categories

(Release Engineering :: Release Automation: Other, enhancement, P1)

Product:
Release Engineering
Component:
Release Automation: Other
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mtabara, Assigned: mtabara)

References

Details

Attachments

(4 files)

Bug 1343990 - add Fennec Release Graph 2. r=aki
48 bytes, text/x-github-pull-request
mozilla
: review+
Details | Review
Bug 1343990 - alter release runner to encompass stage_product as well.
59 bytes, text/x-review-board-request
rail
: review+
mtabara
: checked-in+
Details
Bug 1343990 - add Fennec Release Graph 2. r=rail
48 bytes, text/x-github-pull-request
rail
: review+
mtabara
: checked-in+
Details | Review
Bug 1343990 - add fennec beta specific configs. r=rail
48 bytes, text/x-github-pull-request
rail
: review+
mtabara
: checked-in+
Details | Review 
We need to setup a manual script (similarly to have we have for RC relpro https://github.com/mozilla/releasewarrior/blob/master/how-tos/relpro.md#how-1 ) that can run various pieces of the full release graph. We organize the "subgraphs" as this:
        
Graph1:
a) source builder  templates (source, checksums + deps that kmoir has already been addded in https://github.com/mozilla/releasetasks/commit/5ea2d8793ed29bdb07b000cc22a1fd2738ff43fc )
b) the DECISION task for the fennec candidates graph
            
Graph2: <-- mtabara will be working on this
a) push to releases
b) uptake monitoring
c) (optional but it's broken for Fennec) publish to Balrog
d) post-release tasks

Bug 1343649 is tracking work for the first graph while this bug is tracking the second graph.
Depends on: 1344202
We can take care of bug 1344202 post-phase1 as it's not blocking us right now. Just to keep this in mind that at some point in the automation process we'll need to alter this as well.
No longer depends on: 1344202
See Also: → 1344202
See Also: → 1344222
See Also: → 1344225
See Also: → 1344229
See Also: → 1344248
Comment on attachment 8843418 [details] [review]
Bug 1343990 - add Fennec Release Graph 2. r=aki

As mentioned in IRC: 1) awesome, 2) I didn't catch anything bad, but 3) I'm not well versed in releasetasks (yet) so a real review might be good.
Attachment #8843418 - Flags: review?(aki) → review+
Successfully ran the second graph (just the push-to-releases builder since the other ones need some tweaks) and moved artifacts from http://bucketlister-delivery.stage.mozaws.net/pub/mobile/candidates/53.0b1-candidates/ to http://bucketlister-delivery.stage.mozaws.net/pub/mobile/releases/53.0b1/
candidates -> releases is done now, but I'll rephrase a bit the title to keep this as a tracking bug for all the changes that need to be addressed to the post-release builders.
Summary: releasetasks should trigger candidates -> releases subgraph → releasetasks should trigger second graph with candidates -> releases and post-release tasks
Comment on attachment 8843487 [details]
Bug 1343990 - alter release runner to encompass stage_product as well.

https://reviewboard.mozilla.org/r/117210/#review119022
Attachment #8843487 - Flags: review?(rail) → review+
Comment on attachment 8843487 [details]
Bug 1343990 - alter release runner to encompass stage_product as well.

https://hg.mozilla.org/build/tools/rev/55aeed6f95ac
Attachment #8843487 - Flags: checked-in+
Sum-up of the current terminology so that we all talk in the same language.

Graph 1
 Contains:
     * jamun source generationun
     * jamun source signing
     * Generate beetmover docker image
     * [beetmover] mobile jamun move source bundle checksums
     * [beetmover] mobile jamun move source bundles
     * mobile jamun checksums builder
     * [D] jamun candidates_fennec - this is a decision task!

The graph above is generated either via:
    i) hook https://tools.taskcluster.net/hooks/#project-releng/candidates-fennec-dev 
    ii) release runner script https://github.com/mozilla/releasewarrior/blob/master/how-tos/relpro.md#how-1

Example: https://tools.taskcluster.net/task-group-inspector/#/oj8z6k-AQruZ1VbwJLILBQ/

---

Graph 2:
Contains

    * the nightly graph with building, signing, checksums and beetmover stuff


The graph above is generated via:
    i) is generated and submitted by the decision task from Graph 1. That is because the decision task is part of the releasetasks graph 1, but it generates its own graph (this one, graph2) and submits it separately.  Each task from this graph have the taskGroupId set to the decision task's taskId (this is convention in mach taskgraph). 

Example: https://tools.taskcluster.net/task-group-inspector/#/_EQQGvwGQWC5FQq8fSwc4A/ (generated and chained to the decision task from graph 1 above)

---

Graph 3:
    Contains:
        * push to releases
        * publish to balrog
        * post-release tasks (mark as shipped, version bump, bouncer aliases, etc)

The graph above is generated via:
    i) release runner script https://github.com/mozilla/releasewarrior/blob/master/how-tos/relpro.md#how-1

Example: lost my example from Friday but it's basically sa simple graph containing a single task, the "push-to-releases" task. The other post-release builders need some tweaks before they can be enabled.
Priority: -- → P1
Attachment #8845007 - Flags: review?(rail) → review+
@rail: there is also https://reviewboard.mozilla.org/r/117210/diff/1-2/ 
I think you won't be flagged by reviewboard as r? because I already pushed and checked-in a commit via the reviewboard for this bug for the tools repo with your r+.

r? - https://reviewboard.mozilla.org/r/117210/diff/1-2/
Flags: needinfo?(rail)
Comment on attachment 8843487 [details]
Bug 1343990 - alter release runner to encompass stage_product as well.

https://reviewboard.mozilla.org/r/117210/#review120224
r+
Flags: needinfo?(rail)
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #14)
> r+

Thanks for reviews, rail!

https://hg.mozilla.org/build/tools/rev/988c20890711
Bah, sorry Rail to insist here with the review requests, forgot to add a mozilla-beta specific config file for Fennec. I'd rather add this file now and potentially remove it later, than temporary tweaking it on the bm85 from "mozilla-release" -> "mozilla-beta" just to run the graph.

What do you think?
Attachment #8845391 - Flags: review?(rail)
Attachment #8845391 - Flags: review?(rail) → review+
Push to releases builder worked - http://archive.mozilla.org/pub/mobile/releases/53.0b1/ via https://tools.taskcluster.net/task-group-inspector/#/vo2JO4PYTGCJxFPXSFSv5g/cOlkME2XQfWz1T9T9qY4sw?_k=t94psn

Status update on the other builders:
* uptake monitoring
* mark release as shipped 
* bouncer aliases

.. have all failed because we didn't enabled the BB builders for them:
rail> https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla/builder_master.cfg#155-160
00:20:48 <rail> maybe even need to revise https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla/builder_master.cfg#153
00:20:50 <mtabara> is it just me or it's a simple oneliner with 'fennec' near that 'firefox' string?
00:20:55 <rail> and make it per product
00:21:07 <rail> not sure
00:21:19 <rail> may need to touch release.py as well

We deliberatedly skipped the version bump since the version was bumped anyway by Firefox 53.0b1 this week. The tagging will be added manually.
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #17)
> We deliberatedly skipped the version bump since the version was bumped
> anyway by Firefox 53.0b1 this week. The tagging will be added manually.

Pushed the tag manually - https://hg.mozilla.org/releases/mozilla-beta/rev/a571068c57c8
Comment on attachment 8845391 [details] [review]
Bug 1343990 - add fennec beta specific configs. r=rail

https://github.com/mozilla/releasetasks/pull/220
Attachment #8845391 - Flags: checked-in+
See Also: → 1346806
See Also: → 1346892
We have variations of (formerly knwown "graph1" from comment 9) that are currently working via the releasetasks (including the push to releases step of Fennec 53.0b1). I think it's safe to close this bug now. The remaining related builders (uptake monitoring, bouncer aliases, version bump, mark as shipped, publish to Balrog, bouncer submission, etc) are tracked in bug 1346892 as part of a "phase 2" effort.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
See Also: → 1346899
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: