Right now it is a bit unclear what the finer points of how this operates will be. Does it know that when a request starts with `api/` it replaces rootUrl with one from the environment? How does it work?
We currently have http://localhost/secrets/v1/foo/bar and http://localhost/secrets.taskcluster.net/v1/foo/bar Neither of those looks like the kind of URLs we have defined in tc-lib-urls. I would guess that we want * workers define TASKCLUSTER_ROOT_URL=http://localhost or whatever is suitable for the platform * given an /api/<name>/<version> path, the proxy substitutes its own rootUrl host and port, then adds credentials * given a legacy /<name>/<version> path, the proxy translates it to /api/<name> and does the same * given anything else, the proxy treats the first path element as a hostname, and adds credentials
Summary: Figure out how taskcluster-proxy works with TASKCLUSTER_ROOT_URL → Set TASKCLUSTER_ROOT_URL for tasks and configure proxies to handle resulting requests
It's worth noting that all of our CI is going to fail until this is fixed :)
^^ that's not true -- I have a hack in tc-lib-testing to use superagent instead of tc-client. But I think we still need to fix this, for crater at least. The intent is to have a rootUrl with a path of /, something like http://localhost/, and for the proxies in various engines to translate /api/xyz properly, for the rootUrl of the cluster the worker is running in.
For tc-worker, I think this means building a proxy named 'api' -- clever! That will require a go version of tc-lib-urls. For docker-worker, it means * upgrade taskcluster-proxy to understand paths beginning with /api and apply the current rootUrl * upgrade docker-worker to pass a rootUrl to taskcluster-proxy
Pete, is this something you can help address? I don't know if we can manage to avoid doing this in docker-worker.. This is the hack, by the way: https://github.com/taskcluster/taskcluster-lib-testing/blob/689f8015b986664d1984c4e244c7a9aa1627c718/src/secrets.js#L80
In bug 1492664 I'm setting up the Firefox CI decision task to set TASKCLUSTER_ROOT_URL to the "real" rootUrl where the workers are running, and TASKCLUSTER_PROXY_URL to a rootUrl that will use taskclusterProxy (only if the proxy is enabled on the task). Hopefully, this will match exactly the env vars that the workers pass to tasks -- at which point we can remove it from payload['env'].
Until the changes here have landed, I'm including a bunch of comments in the mozilla-central code matching /bug 1460015/. Those should be addressed once these changes land.
So I think the part to do here is: > Update tc-proxy to accept `/api/..` paths once that's done, we can do the work to make this happen in the two worker implementations in separate bugs. My understanding per irc is that pete is working on this.
I was going to get this started to support bug 1508383 but now I'm halfway done so I'll just finish it up.
Assignee: pmoore → dustin
Pete, could you review this when you get a chance?
Looking at this now...
Commits pushed to master at https://github.com/taskcluster/taskcluster-proxy https://github.com/taskcluster/taskcluster-proxy/commit/d905b77a0c88acb194aaaabc50eaf692b1307261 Bug 1460015 - accept --root-url on the command line https://github.com/taskcluster/taskcluster-proxy/commit/9142d6f4e7cb6ab357e7de5636e10c5a0ff07a25 Bug 1460015 - support rootUrl for /<service> https://github.com/taskcluster/taskcluster-proxy/commit/b22c0b075a405598bffe13333e09be6251032cd3 Bug 1460015 - support /api-format requests https://github.com/taskcluster/taskcluster-proxy/commit/df4cb32610dd17ebd8dbc929b40c1882c2062bcd Merge pull request #38 from djmitche/bug1460015 Bug 1460015 - add rootUrl support
This bug had been narrowed to just adding support in tc-proxy. The work on generic-worker and docker-worker is handled in bug 1508383 and bug 1516458.
Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.