1546757 - Use a file watcher to rebuild the cached taskgraph in the background

Status: RESOLVED FIXED

(Firefox Build System :: Try, enhancement)

Description

[Andrew Halberstadt [:ahal]]

One of the most annoying things about mach try fuzzy/chooser is the need to wait for the taskgraph to regenerate. Instead of making developers wait for this, we should use a file-watcher to do this in the background whenever something under /taskcluster changes.

I had a proposal in bug 1467147 that would make doing things like this easier. We don't need to block on that, but it might be worth solving this more generically first anyway.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546757 - [tryselect] Provide defaults to generate_tasks arguments, r?tomprince

This allows consumers to call 'generate_tasks()' without arguments.

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546757 - [tryselect] Add a watchman trigger to rebuild the taskgraph cache in the background, r?tomprince

This adds a 'watchman.json' file to /tools/tryselect and some documentation on
how to use it. Tl;dr, install watchman and then:

$ cd path/to/gecko
$ watchman -j < tools/tryselect/watchman.json

Depends on D28770

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546757 - [tryselect] Always cache both the full_task_set and target_task_set when generating tasks, r?tomprince

Since we need to generate the full_task_set as a prereq to the target_task_set, we might
as well save the computed set to the cache while we have it. This means users that run:

$ ./mach try fuzzy
$ ./mach try fuzzy --full

Will only incur task generation once. It also means that the 'watchman' trigger will cache
both taskgraphs.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d0b5adc9ff32
[tryselect] Provide defaults to generate_tasks arguments, r=Callek
https://hg.mozilla.org/integration/autoland/rev/82b12442ef44
[tryselect] Add a watchman trigger to rebuild the taskgraph cache in the background, r=tomprince
https://hg.mozilla.org/integration/autoland/rev/5a253216d75b
[tryselect] Always cache both the full_task_set and target_task_set when generating tasks, r=tomprince

Comment 5

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/d0b5adc9ff32
https://hg.mozilla.org/mozilla-central/rev/82b12442ef44
https://hg.mozilla.org/mozilla-central/rev/5a253216d75b

Comment 6

[Matthew N. [:MattN]]

Attachment: Bug 1546757 - Output statedir in code snippet for testing taskgraph watchman trigger. r=ahal

Otherwise statedir is always ""

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d9d2bb7e7b8
Output statedir in code snippet for testing taskgraph watchman trigger. r=ahal

Comment 8

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/0d9d2bb7e7b8