1128414 - Add conditional caching step prior to running builds.

Status: RESOLVED WONTFIX

(Taskcluster :: UI, defect)

Description

[James Lal [:lightsofapollo]]

Recently tc-vcs has gotten a lot of love and is at the point where it can not only easily clone/cache git/hg/git-repo based projects but also cache them in relatively optimized ways (for example I measured cloning [with remote caches] emulator-kk from a cold state at 1min 30s on r3 instance). To fully utilize this we should implement a "caching" phase which will optionally happen (and block the builds from running) when a cache is not available.

The caching model I intend on implementing here is like this:

 - a custom namespace will be used _per_ worker-type (for future security proofing)
 - the caching task will upload artifacts using tc-vcs's create-[repo|clone]-cache mechanisms but only be scheduled if an artifact cannot be found from the index (most tasks should be very fast but some will take time to cache)
 - implement it in a "pluggable" way so we can run other tasks based on their namespace in the pre-build caching phase.

Comment 1

[James Lal [:lightsofapollo]]

This is a little to complex in it's current form to implement easily for now I have deployed caches manually which serves well enough... We can improve this with periodic jobs.

Comment 2

[Greg Arndt [:garndt]]

Looks like it was decided to just make caching a little easier and scheduled, which is the work Selena has been working on.

Comment 3

[Selena Deckelmann :selenamarie :selena]

We've decided to move this work into the decision task long term.