Closed Bug 1574663 Opened 5 years ago Closed 5 years ago

Migrate git-cinnabar to community taskcluster deployment

Categories

(Taskcluster :: Operations and Service Requests, task)

Product:
Taskcluster
Component:
Operations and Service Requests
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

References

Details

Make a plan to move this project to the new community deployment.

Notes:
relatively simple changes to CI/variables.py and download.py

Not related to the git-cinnabar CI, but presently, I'm (ab)using taskcluster to generate git-cinnabar bundles as artifacts that are exposed by the mercurial servers. This was put in place in bug 1550183, and currently uses index.taskcluster.net urls. Depending on the time frames, we should either deal with that somehow (assuming index urls will change), or move the process to somewhere more permanent (publishing the bundles on the hg cdn, like the hg bundles)

The index URLs will, indeed, change. Either option works for me -- which would you like?

What's the time frame for the transition?

At this point, we're looking at having everyone migrated by November (but that may change).

I should add, we'll keep index.taskcluster.net running for a while (on the scale of months) so that existing links will work. However, that copy of the index DB will be static at that point so it won't include new versions. So if you just switch the mercurial servers to point to the new index URLs once they have artifacts, then that's probably sufficient. Let's plan on that, and if you're feeling ambitious and want to move the process somewhere more permanent, then that's great too.

Since I'm currently looking into issues I have with my codecov reports, I was reminded that there's also a url for access to secrets via http://taskcluster/secrets/v1 in CI/decision.py, if that needs adjustment too.

That one will keep working, since it's via taskcluster-proxy, but I might update it a little while I'm poking around (generating a URL from $TASKCLUSTER_PROXY_URL, with a different path).

(checked that this is a v1 .taskcluster.yml).

I'm working on this now, but getting some failures I don't recognize:
https://community-tc.services.mozilla.com/tasks/groups/LZsbP9pTQ_C6xWs1s-XC2Q

EBUG:urllib3.connectionpool:http://localhost:None "GET /images/git-cinnabar/base:e575eaf7068f536fd85453736a28a67608900a9e/json HTTP/1.1" 404 74
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): community-tc.services.mozilla.com:443
DEBUG:urllib3.connectionpool:https://community-tc.services.mozilla.com:443 "GET /queue/v1/task/j5De4JLcRMq6zLJ2tJW7Ig/artifacts/public/image.tar.zst HTTP/1.1" 200 579
Traceback (most recent call last):
  File "repo/CI/docker.py", line 398, in <module>
    sys.exit(getattr(CommandHandler, args.command)(image))
  File "repo/CI/docker.py", line 311, in build
    ret = cls.load(image.base)
  File "repo/CI/docker.py", line 357, in load
    headers={'Content-Type': 'application/x-tar'},
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 469, in send
    for i in request.body:
zstd.ZstdError: zstd decompress error: Unknown frame descriptor

any hints on that basis? i will keep looking but I'm off for the day so I figured I'd fish for advice :)

That doesn't make much sense. The same version of zstandard is used in this task and the task it's downloading the parent image from... why would it fail to decompress?

I know right? :)

Ok, the problem is that the artifact url doesn't work: https://community-tc.services.mozilla.com/queue/v1/task/j5De4JLcRMq6zLJ2tJW7Ig/artifacts/public/image.tar.zst returns a code HTTP 200, with HTML contents that display an error 404. Obviously, that's not a docker image compressed with zstd.

Got it, thanks!

https://github.com/glandium/git-cinnabar/compare/master...djmitche:bug1574663?expand=0

This has been pretty easy so far, if a bit fiddly -- nicely structured decision task! It's just a lot of bits and pieces to put together..

The remaining bits for me here:

  • set up a win2012r2 worker pool
  • manage clients with community-tc-config and create client credentials
  • pull-request, tag, and release scopes

And for you (@glandium) once this is ready to land:

Depends on: 1590557

By the way, we would like to eventually transition to using "static" workers:
https://community-tc.services.mozilla.com/docs/reference/core/worker-manager/static
for this sort of thing. The config is a bit easier: just provide the worker with a static secret value, and it will get fresh credentials and config from worker-manager. It can also be "managed" from worker-manager: errors are reported there, etc.

Anyway, that's not ready yet because generic-worker doesn't support worker-runner yet, but once that's ready it might be a nicer experience for you.

The plan for the windows pool is to set that up in the just-created AWS account (bug 1584379) using the same ami as we use in the current deployment.

Depends on: 1584379

I'm not sure how that would work with my abuse of travis as taskcluster workers...

It should continue to work -- I think you'll just need to reset the access tokens on
https://community-tc.services.mozilla.com/auth/clients/project%2Fgit-cinnabar%2Fworker-osx-10-10
https://community-tc.services.mozilla.com/auth/clients/project%2Fgit-cinnabar%2Fworker-osx-10-11
and encrypt those for travis. The other changes are in CI/start-worker.sh.

Depends on: 1590910
Depends on: 1591243

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #19)

https://github.com/mozilla/community-tc-config/pull/37
https://github.com/glandium/git-cinnabar/pull/231

To be clear, I guess the latter can't be merged until the former is deployed, right?

Yes, but I have already applied it :)

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #21)

Yes, but I have already applied it :)

Huh? But the PR is open...

I applied it in the process of testing. We should still merge it, of course -- just pointing out that you can begin using it now.

PSA: The existing (https://taskcluster.net) deployment will be shut down a week from today, on November 9. After that point, any CI not migrated to the new community cluster will stop functioning. The TC team is ready and eager to help get everything migrated by that time, but the deadline is firm.

Apologies for failing to communicate this as broadly and loudly as necessary, and for the bugspam now.

PR is merged and I see commits on master green in the new deployment!

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.