1491771 - Smarter Purge Cache

Status: RESOLVED WONTFIX

(Taskcluster :: Services, enhancement, P5)

Description

[Pete Moore [:pmoore][:pete]]

Would be neat if workers registered caches they create, so at any time you could see which caches exist for a given worker or the list of all caches in use by a worker of a given workerType/provisionerId.

This would then allow us to throw a warning if a purge cache request is made for a cache which would have no effect (typical use case would be misspelling of workerType/provisionerId/cacheName) - note, sometimes users submit purge cache requests for "dockerworker:cache:<cache>" instead of "<cache>", for example.

Another possibility it introduces is to allow users to select provisionerId, workerType, cacheName from a list, rather than needing to type in strings.

Another problem is once a user has submitted a purge cache request, there is no feedback if any caches *actually* do get purged as a result.

Another nice feature would be if workers registered when they deleted actual caches, so that the user interface could show which workers deleted caches as a result of a purge cache request. This would make the whole experience of purging caches much more pleasant, intuitive and transparent for users.

In order to implement the above the following changes are needed:

1) The purge cache service should supply an additional endpoint for registering the creation of a cache on a worker (workerId, groupId, workerType, provisionerId). The purge cache service should persist this information in some storage backend.

2) The purge cache service should supply an additional endpoint for registering the deletion of a cache on a worker (workerId, groupId, workerType, provisionerId). The purge cache service should persist this information in some storage backend.

3) The purge cache service should supply three additional (unauthenticated) endpoints for listing existing caches on workers:

  i) per provisionerId,
  ii) per provisionerId/workerType,
  iii) per provisionerId/workerType/workerGroup/workerId

3) workers should call the endpoints described in 1) and 2) above when they create/delete caches.

4) The tools site purge cache interface should call the API endpoints in 3) above to provide this information to the user for existing purge cache requests and new ones that are made (together with all the other changes suggested in this bug). We could even have something like a cache explorer to present this information, that allowed for purging worker caches. We could even consider allowing purging worker caches on individual workers, rather than for just all workers of a given worker type.

Comment 1

[Pete Moore [:pmoore][:pete]]

(excuse the numbering)