Closed Bug 1282180 Opened 4 years ago Closed 3 years ago

Create a periodic task to launch nightly builds

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: amiyaguchi, Assigned: kmoir)

References

Details

Attachments

(1 file)

In order to create a nightly build, there is a need for a mechanism to kick off an in-tree build periodically. I propose a nightly decision task similar to the decision task that is registered with taskcluster-hooks. This task should get information about what revision in tree to build, instantiate a template of task that creates the nightly graph, and put the task on the queue.
Attachment #8765079 - Flags: feedback?(jlund)
Attachment #8765079 - Attachment description: [WIP] This script instantiates a template similar to m-c/.taskcluster.yml with a parameter file → [WIP] nightlydecision.py - instantiates a template similar to m-c/.taskcluster.yml with a parameter file
Attachment #8765079 - Attachment is patch: true
Attachment #8765079 - Attachment mime type: text/x-python-script → text/plain
Comment on attachment 8765079 [details] [diff] [review]
[WIP] nightlydecision.py - instantiates a template similar to m-c/.taskcluster.yml with a parameter file

I think this is the right approach and will be needed.

I'd be curious to see what are dustin's thoughts here. I am assuming retrofitting current decision task logic so we have one script for CI and Releases, would be more troublesome than it's worth..
Attachment #8765079 - Flags: feedback?(jlund) → feedback+
So this isn't actually the decision task, it's a script that creates a decision task.

I definitely think it makes sense for the nightly and CI task graphs to be generated by the same decision code, but it looks like that's what's going on here.
That's the plan. I think my use of the terminology was a bit off. I meant to say that this would be similar to mozilla-taskcluster where instead of polling the mercurial pushlog, we trigger the task through a hook.

The goal is to have a good starting point to figure out the rest of the nightly graph. I'm hoping that we can have most of the task-graph generation for both CI and nightlies in one place with this script/worker to kick off the nightly graph generation. I think the issues that I will encounter once I flesh this script out will probably be figuring out which worker to run this on and what scopes it will need, which I haven't looked into yet.
Depends on: 1282923
Depends on: 1286989
Blocks: 1277682
No longer blocks: 1277595
Depends on: 1290969
Depends on: 1292393
Depends on: 1286979
I have created a hook for nightly-fennec builds under hooks:releng/nightly-fennec-dev. This hook has a role that allows it to create tasks that use scopes required by the dummy signing worker.

This hook definition is based on hook defined by mtabara under hooks:releng/pgo, with some differences to accommodate my nightly patches. I have also added a way to trigger hooks from the hooks manager which will make the process of debugging much easier for these builds.
Blocks: 1277595
Kim: how does this fit in with the graph work your doing? Is this part already done?
Assignee: acmiyaguchi → kmoir
Flags: needinfo?(kmoir)
Yes, I have hooks created for both nightly fennec and nightly desktop and they are running nightly builds on date.  Once we are running them on inbound, the scopes/client will need to be adjusted to run them on that branch.
Flags: needinfo?(kmoir)
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
(In reply to Kim Moir [:kmoir] from comment #6)
> Yes, I have hooks created for both nightly fennec and nightly desktop and
> they are running nightly builds on date.  Once we are running them on
> inbound, the scopes/client will need to be adjusted to run them on that
> branch.

To be clear, we won't be running them on inbound, but will be on m-c (and aurora, although possibly 6 weeks later).
Yes, sorry, that's what I meant.  The patches will land on inbound for the nightly graph, the hook will run on m-c.
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.