Allow caches to be shared across tasks

NEW
Unassigned

Status

Taskcluster
Worker
a year ago
a year ago

People

(Reporter: gps, Unassigned)

Tracking

Details

(Reporter)

Description

a year ago
I'm not sure if this is the correct component...

Currently, if multiple tasks run on the same Docker worker, each task gets its own cache instance. I think this makes sense as a default behavior because it is safest.

However, in some cases consumers of a cache know how to lock files and allow simultaneous access across concurrently running tasks. Mercurial store caches fall into this bucket: Mercurial obtains a lock that only allows single writers to repo stores.

When triggering multiple decision tasks on try pushes, I've seen at least 4 decision tasks running on the same worker. Each worker is doing its own `hg clone` into its own independent cache. This makes the tasks much slower because 4x the amount of I/O is being performed on the worker.

I'd love to be able to mark a cache as "shared" between tasks so we could make things like version control operations more efficient.

Since we currently don't run multiple tasks per worker very often, this is likely a low priority.
Component: Docker-Worker → Worker
You need to log in before you can comment on or make changes to this bug.