We currently create artifacts on the form: public/build/firefox....<version>.....tar.bz2 This is hard to use in automation. And for people doing bisection by finding artifacts using the index, it's not all that fun to list artifacts and use regexps to find the right artifact. As opposed to just knowing that the build is always: public/build/firefox.tar.bz2 We can and **should** implement this using "reference" artifacts. Which allows you to create an artifact that just redirects to a URL. So we can make the artifact public/build/firefox.tar.bz2 as a "reference" that redirects to the public/build/firefox....<version>.....tar.bz2 artifact. But this pattern won't work nicely when the artifact referenced is private. Because you'll then need to sign the URL after it's been redirected. To resolve this I propose an alias artifact, which can reference any other artifact within the same task. The alias artifact has the same expiration as the artifact it aliases. But it changes the scope required to access the artifact. Example: Consider artifact: private-1/build.tar.gz requires scope queue:get-artifact:private-1/build.tar.gz Now if we create an alias artifact, for private-1/build.tar.gz private-2/alias.tar.gz It requires scope queue:get-artifact:private-2/alias.tar.gz I propose that we do not require any additional scopes access or create the artifact "private-2/alias.tar.gz". Creating an alias artifact can move a private artifact to "public/" or to another scope. But a task is only allowed to create aliases for it's own artifacts, not artifacts from other tasks. So it's safe to assume that the task that creates an alias public/build.tar.gz for the artifact private/build.tar.gz has access to the file build.tar.gz before it was uploaded. Thoughts? Is there a use-case where this isn't the case and we risk elevating scopes (or making artifacts public) that we don't want to?
(In reply to Jonas Finnemann Jensen (:jonasfj) from comment #0) > This is hard to use in automation. And for people doing bisection by finding > artifacts using the index, it's not all that fun to list artifacts and use > regexps to find the right artifact. As opposed to just knowing that the > build is always: > public/build/firefox.tar.bz2 What about non-linux? This would at least be firefox.zip on Windows and firefox.dmg on OSX. Or what if we want to change the file extension from .tar.bz2 to .tar.sz or something - would automation need to know that linux packages have different extensions before & after the switchover date?
Err, I was thinking .xz, not .sz. Maybe someone will make a .sz zipping algorithm to make my comment valid :)
Component: TaskCluster → Queue
Product: Testing → Taskcluster
Do you still want this?
I think this might be neat for private logs, and things like that. No immediate demand.
Hi, I wanted to work on this bug. I went through artifacts.js (https://github.com/taskcluster/taskcluster-queue/blob/master/src/artifacts.js#L10) and have a rough idea of how an artifact is created. I would like to start with public artifacts, since it is easier to implement. If I get it right, we have to create/update the artifact public/build/firefox.tar.bz2 everytime we receive a request for public/build/firefox..<ver>...tar.bz2 1) Do we have to maintain a separate table for storing the reference ? (firefox..<ver.. is referenced by firefox) 2) Any resources other than the docs that I can read and would help ?
There is still a few design issues to be resolved around alias artifacts and private artifacts. Maybe this is not a good bug right now, (don't worry I'd be happy to help you find one).
You need to log in before you can comment on or make changes to this bug.