Use a file watcher to rebuild the cached taskgraph in the background

RESOLVED FIXED in Firefox 68

Status

enhancement
RESOLVED FIXED
3 months ago
3 months ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

(Depends on 1 bug)

unspecified
mozilla68
Dependency tree / graph

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(4 attachments)

One of the most annoying things about mach try fuzzy/chooser is the need to wait for the taskgraph to regenerate. Instead of making developers wait for this, we should use a file-watcher to do this in the background whenever something under /taskcluster changes.

I had a proposal in bug 1467147 that would make doing things like this easier. We don't need to block on that, but it might be worth solving this more generically first anyway.

Type: defect → enhancement

This allows consumers to call 'generate_tasks()' without arguments.

This adds a 'watchman.json' file to /tools/tryselect and some documentation on
how to use it. Tl;dr, install watchman and then:

$ cd path/to/gecko
$ watchman -j < tools/tryselect/watchman.json

Depends on D28770

Since we need to generate the full_task_set as a prereq to the target_task_set, we might
as well save the computed set to the cache while we have it. This means users that run:

$ ./mach try fuzzy
$ ./mach try fuzzy --full

Will only incur task generation once. It also means that the 'watchman' trigger will cache
both taskgraphs.

Attachment #9061999 - Attachment description: Bug 1546757 - [tryselect] Cache full_task_set when generating the target_task_set, r?tomprince → Bug 1546757 - [tryselect] Always cache both the full_task_set and target_task_set when generating tasks, r?tomprince
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d0b5adc9ff32
[tryselect] Provide defaults to generate_tasks arguments, r=Callek
https://hg.mozilla.org/integration/autoland/rev/82b12442ef44
[tryselect] Add a watchman trigger to rebuild the taskgraph cache in the background, r=tomprince
https://hg.mozilla.org/integration/autoland/rev/5a253216d75b
[tryselect] Always cache both the full_task_set and target_task_set when generating tasks, r=tomprince
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0d9d2bb7e7b8
Output statedir in code snippet for testing taskgraph watchman trigger. r=ahal
You need to log in before you can comment on or make changes to this bug.