1328172 - Support downloading internal resources from tooltool in toolchain-script

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[(no longer active)]

In order to get this to work, we need to add the correct scope to the worker to give it access to the secrets corresponding to the access level of the respective repository.  I have a patch that makes that part work.  For example, for a push to the try server, the task scope now has "secrets:get:project/releng/gecko/build/level-1/*".  I think this is the correct scope, given that builds on inbound have "secrets:get:project/releng/gecko/build/level-3/*".

I also add the "docker-worker:relengapi-proxy:tooltool.download.public" and "docker-worker:relengapi-proxy:tooltool.download.internal" scopes set up, which are needed for access to tooltool through relengapi.

The last step is somehow downloading relengapi.tok through the secrets API, but the level-1 jobs on try have access to no secrets, and the level-3 jobs on inbound have only access to the following:

# curl http://taskcluster/secrets/v1/secrets
{
  "secrets": [
    "project/releng/gecko/build/level-3/adjust-sdk-beta.token",
    "project/releng/gecko/build/level-3/adjust-sdk.token",
    "project/releng/gecko/build/level-3/gapi.data",
    "project/releng/gecko/build/level-3/google-oauth-api.key",
    "project/releng/gecko/build/level-3/mozilla-desktop-geoloc-api.key",
    "project/releng/gecko/build/level-3/mozilla-fennec-geoloc-api.key",
    "project/taskcluster/gecko/hgfingerprint"
  ]
}

I'm not sure how else we should obtain the tooltool authentication file...  Dustin, do you have any suggestions?  Thanks!

Comment 1

[(no longer active)]

Attachment: Patch so far

Comment 2

[(no longer active)]

(In reply to :Ehsan Akhgari from comment #0)
> In order to get this to work

This being cross-compile builds for OSX clang-tidy on Linux.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

For talking to relengapi via the relengapi-proxy, you don't need a tooltool authentication file.

What secrets do you need access to?  I suspect you could get away with *just* the `tooltool-downloads: internal` stuff.

+    # tooltool downloads.  By default we download using the API endpoint, but
+    # the job can optionally request relengapi-proxy (for example when downloading
+    # internal tooltool resources.  So we define the tooltool cache unconditionally.
It would be nice to convert all toolchain builds to use the relengapi proxy, regardless of internal/external access, just for consistency.

Comment 4

[(no longer active)]

(In reply to Dustin J. Mitchell [:dustin] from comment #3)
> For talking to relengapi via the relengapi-proxy, you don't need a tooltool
> authentication file.

Hmm...  Can you explain what happened here?

https://tools.taskcluster.net/task-inspector/#fZF8rtUkTbO8hXIgti7F7A/

The task has |relengAPIProxy: true| and also the "docker-worker:relengapi-proxy:tooltool.download.public" and "docker-worker:relengapi-proxy:tooltool.download.internal" scopes, but downloading from tooltool failed:

https://public-artifacts.taskcluster.net/fZF8rtUkTbO8hXIgti7F7A/0/public/logs/live_backing.log
+ ./taskcluster/docker/recipes/tooltool.py -m browser/config/tooltool-manifests/macosx64/cross-clang.manifest fetch
INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
INFO - File clang.tar.xz fetched from https://api.pub.build.mozilla.org/tooltool/ as /home/worker/workspace/build/src/tmpCuvEXm
INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
INFO - File gcc.tar.xz fetched from https://api.pub.build.mozilla.org/tooltool/ as /home/worker/workspace/build/src/tmpbB2ewn
INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
INFO - File cctools.tar.gz fetched from https://api.pub.build.mozilla.org/tooltool/ as /home/worker/workspace/build/src/tmpIO93ZD
INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
INFO - ...failed to fetch 'MacOSX10.10.sdk.tar.xz' from https://api.pub.build.mozilla.org/tooltool/
INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
INFO - File sccache2.tar.xz fetched from https://api.pub.build.mozilla.org/tooltool/ as /home/worker/workspace/build/src/tmpYnCunk
INFO - File integrity verified, renaming tmpCuvEXm to clang.tar.xz
INFO - File integrity verified, renaming tmpbB2ewn to gcc.tar.xz
INFO - File integrity verified, renaming tmpIO93ZD to cctools.tar.gz
INFO - File integrity verified, renaming tmpYnCunk to sccache2.tar.xz
INFO - untarring "clang.tar.xz"
INFO - untarring "gcc.tar.xz"
INFO - untarring "cctools.tar.gz"
INFO - untarring "sccache2.tar.xz"
ERROR - The following files failed: 'MacOSX10.10.sdk.tar.xz'

(MacOSX10.10.sdk.tar.xz is the internal tool being downloaded here.)

> What secrets do you need access to?  I suspect you could get away with
> *just* the `tooltool-downloads: internal` stuff.

Yeah, I just need something that allows me to download internal tooltool resources.

> +    # tooltool downloads.  By default we download using the API endpoint,
> but
> +    # the job can optionally request relengapi-proxy (for example when
> downloading
> +    # internal tooltool resources.  So we define the tooltool cache
> unconditionally.
> It would be nice to convert all toolchain builds to use the relengapi proxy,
> regardless of internal/external access, just for consistency.

Yep, I'd be happy to do that once this setup works...

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Add `--url=http://relengapi/tooltool/` to the tooltool invocation and it should just work.  The short name `relengapi` points to the relengapi proxy, which adds the authentication header.

Comment 6

[(no longer active)]

(In reply to Dustin J. Mitchell [:dustin] from comment #5)
> Add `--url=http://relengapi/tooltool/` to the tooltool invocation and it
> should just work.  The short name `relengapi` points to the relengapi proxy,
> which adds the authentication header.

Nice, that works great!  Thank you.

Comment 7

[(no longer active)]

Attachment: Support downloading internal resources from tooltool in toolchain-script

Comment 8

[(no longer active)]

Attachment: Support downloading internal resources from tooltool in toolchain-script

There was a leftover in the previous patch...

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/061f9628b172
Support downloading internal resources from tooltool in toolchain-script; r=dustin

Comment 10

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/061f9628b172