Give buildbot client enough powers to attach artifacts to BBB tasks

RESOLVED WONTFIX

Status

Taskcluster
Authentication
RESOLVED WONTFIX
2 years ago
2 years ago

People

(Reporter: rail, Unassigned)

Tracking

Details

(Reporter)

Description

2 years ago
To upload TC artifact from buildbot jobs we use taskcluster-client.py to create, clam, attach artifacts and resolve tasks. L10N jobs create multiple tasks to make sure that artifacts are accessible in the index using routes containing locale name.

https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/desktop_l10n.py#1007
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/building/buildbase.py#1473


Release promotion creates a separate task for every l10n chunk just to keep the artifacts in one place and have predictable artifact URLs (we use "reference" artifacts). This is a bit annoying because it requires extra 50 tasks in the graph.

One of the options would be using the original buildbot-bridge task (which schedules l10n jobs in buildbot). With the current setup, if you try to attach an artifact from a buildbot slave to a task created for BBB, it fails with authorization error.

I guess this is because the BBB task uses different worker type/id that allowed for the buildbot client-id.

My wild guess we would need to add either/both of the follwong to the buildbot client id:

  assume:worker-id:buildbot-bridge/buildbot-bridge
  assume:worker-type:buildbot-bridge/buildbot-bridge
createArtifact requires

queue:create-artifact:<name>, assume:worker-id:<workerGroup>/<workerId>
  or
queue:create-artifact:<taskId>/<runId>

Ideally, the tasks would be using the temporary credentials returned when the BBB task was claimed by BBB -- among other things, those credentials have `queue:create-artifact:<taskId>/<runId>`.

Barring that, happily for BBB the workerGroup and workerId are always the same ("buildbot-bridge"), so we could statically assign `assume:worker-id:buildbot-bridge/buildbot-bridge` to whatever credentials the l10n tasks are using, along with `queue:create-artifact:<name>` for whatever artifact name(s) are required.  However, this would allow those l10n tasks to add artifacts to any and all BBB tasks, which is probably a bit more broad than you want.
(Reporter)

Comment 2

2 years ago
(In reply to Dustin J. Mitchell [:dustin] from comment #1)
> createArtifact requires
> 
> queue:create-artifact:<name>, assume:worker-id:<workerGroup>/<workerId>
>   or
> queue:create-artifact:<taskId>/<runId>
> 
> Ideally, the tasks would be using the temporary credentials returned when
> the BBB task was claimed by BBB -- among other things, those credentials
> have `queue:create-artifact:<taskId>/<runId>`.

This sounds like a major thing, not sure if I want to make the process so complex :/

BTW, does `queue:create-artifact:<taskId>/<runId>` mean that I can't rerun a task or using `queue:create-artifact:<taskId>/*` is possible?

> Barring that, happily for BBB the workerGroup and workerId are always the
> same ("buildbot-bridge"), so we could statically assign
> `assume:worker-id:buildbot-bridge/buildbot-bridge` to whatever credentials
> the l10n tasks are using, along with `queue:create-artifact:<name>` for
> whatever artifact name(s) are required.  However, this would allow those
> l10n tasks to add artifacts to any and all BBB tasks, which is probably a
> bit more broad than you want.

*any in progress* I guess. Even this is quite broad, I agree. I'm not sure, what else can be done here and how bad would be giving these powerz.
Using the temp creds from the claim is the "right" solution, and whether it's the best fix really depends on the lifetime of BBB.  As you said before the break, that's probably several years, so I think it's reasonable to consider taking this approach.

If you re-run a task, the claim operation gets a new set of temporary credentials.

And yes, you're right -- any task that's not resolved can have artifacts added.
(Reporter)

Comment 4

2 years ago
We talked about this today and decided to not spend time on this because in the future we won't be using BBB.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.