1282180 - Create a periodic task to launch nightly builds

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: [WIP] nightlydecision.py - instantiates a template similar to m-c/.taskcluster.yml with a parameter file

In order to create a nightly build, there is a need for a mechanism to kick off an in-tree build periodically. I propose a nightly decision task similar to the decision task that is registered with taskcluster-hooks. This task should get information about what revision in tree to build, instantiate a template of task that creates the nightly graph, and put the task on the queue.

Comment 1

[Jordan Lund (:jlund)]

Comment on attachment 8765079 [details] [diff] [review]
[WIP] nightlydecision.py - instantiates a template similar to m-c/.taskcluster.yml with a parameter file

I think this is the right approach and will be needed.

I'd be curious to see what are dustin's thoughts here. I am assuming retrofitting current decision task logic so we have one script for CI and Releases, would be more troublesome than it's worth..

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

So this isn't actually the decision task, it's a script that creates a decision task.

I definitely think it makes sense for the nightly and CI task graphs to be generated by the same decision code, but it looks like that's what's going on here.

Comment 3

[Anthony Miyaguchi [:amiyaguchi]]

That's the plan. I think my use of the terminology was a bit off. I meant to say that this would be similar to mozilla-taskcluster where instead of polling the mercurial pushlog, we trigger the task through a hook.

The goal is to have a good starting point to figure out the rest of the nightly graph. I'm hoping that we can have most of the task-graph generation for both CI and nightlies in one place with this script/worker to kick off the nightly graph generation. I think the issues that I will encounter once I flesh this script out will probably be figuring out which worker to run this on and what scopes it will need, which I haven't looked into yet.

Comment 4

[Anthony Miyaguchi [:amiyaguchi]]

I have created a hook for nightly-fennec builds under hooks:releng/nightly-fennec-dev. This hook has a role that allows it to create tasks that use scopes required by the dummy signing worker.

This hook definition is based on hook defined by mtabara under hooks:releng/pgo, with some differences to accommodate my nightly patches. I have also added a way to trigger hooks from the hooks manager which will make the process of debugging much easier for these builds.

Comment 5

[Chris Cooper [:coop] (he/him)]

Kim: how does this fit in with the graph work your doing? Is this part already done?

Comment 6

[Kim Moir [:kmoir] ET]

Yes, I have hooks created for both nightly fennec and nightly desktop and they are running nightly builds on date.  Once we are running them on inbound, the scopes/client will need to be adjusted to run them on that branch.

Comment 7

[Chris Cooper [:coop] (he/him)]

(In reply to Kim Moir [:kmoir] from comment #6)
> Yes, I have hooks created for both nightly fennec and nightly desktop and
> they are running nightly builds on date.  Once we are running them on
> inbound, the scopes/client will need to be adjusted to run them on that
> branch.

To be clear, we won't be running them on inbound, but will be on m-c (and aurora, although possibly 6 weeks later).

Comment 8

[Kim Moir [:kmoir] ET]

Yes, sorry, that's what I meant.  The patches will land on inbound for the nightly graph, the hook will run on m-c.