1546619 - "Create Interactive Task" from treeherder doesn't provide sufficient scopes for private artifacts

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Nathan Froyd [:froydnj]]

STR:

1. Find an OSX build task on try (I used https://treeherder.mozilla.org/#/jobs?repo=try&revision=358fcda789a5489807fea8032e6e612d0b206f6b&selectedJob=242157547 )
2. Click menu, then "Create interactive task"
3. When the interactive task comes up, replicate build steps.

ER:

Things build, or at least build to the point of interest.

AR:

...
10:44:33     INFO -   0:09.33 403 Client Error: Forbidden for url: http://relengapi/tooltool/sha512/8811050fe375bcc566c8b85173d86b8a87aa214
8edfed93023735c2de44b66a5a28cbaa1cfd396032447fd803e03f308ed941a200c0e2a1ad9fbe16b5606ee7c
10:44:33     INFO -   0:09.33 Failed to download MacOSX10.11.sdk.tar.xz
10:44:33    ERROR - Return code: 1
10:44:33    ERROR - 1 not in success codes: [0]
10:44:33  WARNING - setting return code to 2
10:44:33    FATAL - Halting on failure while running ['/builds/worker/workspace/build/src/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/pyt
hon', '-u', '/builds/worker/workspace/build/src/mach', 'artifact', 'toolchain', '-v', '--retry', '4', '--artifact-manifest', '/builds/worke
r/workspace/build/src/toolchains.json', '--tooltool-manifest', '/builds/worker/workspace/build/src/browser/config/tooltool-manifests/macosx
64/cross-releng.manifest', '--tooltool-url', 'http://relengapi/tooltool/', '--cache-dir', '/builds/worker/tooltool-cache', 'public/build/bi
nutils.tar.xz@K5NcyYbZRF25fJF8XzJLoA', 'public/build/cctools.tar.xz@QzKOkVYZSfqylnp62khASQ', 'public/build/clang.tar.xz@N51-2jx3RnOOd86FJO6
85A', 'public/build/hfsplus-tools.tar.xz@SJLbp6cRTimPgEoCi454pA', 'public/build/dmg.tar.xz@dEUhWzyRQoCXRTpWbD51WA', 'public/build/llvm-dsym
util.tar.xz@dkZ-fwaTQ7WbKWZ3pgGu-w', 'public/build/rustc.tar.xz@AgjE4WMiTsaL01s83M9dbg', 'public/build/rust-size.tar.xz@IlyF0MjpRDOhehriA_Z
OnQ', 'public/build/cbindgen.tar.xz@bPEue9VVQ_eRjPuRyJhmOA', 'public/build/nasm.tar.bz2@N94LYZSCRO-GGhFunBYWAw', 'public/build/node.tar.xz@
EXixwTpEQ9CLAA2Auou8wg']
10:44:33    FATAL - Running post_fatal callback...
10:44:33    FATAL - Exiting 2

The Mac SDK is marked as a private artifact. I assume the same thing happens for at least the Android builds (the NDK is a private artifact), and maybe a couple other build kinds.

This error does not happen if you go into taskcluster directly and edit the task to be interactive. But the latter is...less straightforward and ought to have the same result as the former.

Something magically fixed the same sort of problem in bug 1492339, but the fix does not seem to have stuck (bug 1487500?). IIUC, docker-worker:relengapi-proxy:tooltool.download.internal needs to be added hereish:

https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/actions/create_interactive.py#55

which would at least fix the problem, even if it's not the right fix.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

I think that is the right fix. The "private" downloads are just "things we can't redistribute" -- and allowing someone on an interactive task to download the file into that task does not constitute redistribution.

Comment 2

[Nathan Froyd [:froydnj]]

Attachment: Bug 1546619 - whitelist internal tooltool downloads for interactive task creation; r=dustin

These artifacts are "internal" because we can't redistribute them, but
downloading them into an interactive task is not redistribution.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d3d207fbbcd2
whitelist internal tooltool downloads for interactive task creation; r=dustin

Comment 4

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/d3d207fbbcd2