Closed Bug 1277838 Opened 6 years ago Closed 6 years ago

[taskcluster-vcs] Move caches from git.m.o to other repositories

Categories

(Taskcluster :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: wcosta, Assigned: wcosta)

References

Details

Attachments

(2 files)

git.m.o will shutdown. The following repositories are cached by taskcluster-vcs, we need to either remove them or redirect to appropriate alternative repos:

https://git.mozilla.org/b2g/B2G.git
https://git.mozilla.org/b2g/B2G (yes, it is duplicated, marked here to remember to fix that)
https://git.mozilla.org/build/tooltool.git
https://git.mozilla.org/external/google/gerrit/git-repo.git
(In reply to Wander Lairson Costa [:wcosta] from comment #0)
> git.m.o will shutdown. The following repositories are cached by
> taskcluster-vcs, we need to either remove them or redirect to appropriate
> alternative repos:

If you do want to redirect, here are the upstream URLs used to mirror
these repositories that were being mirrored.

> https://git.mozilla.org/b2g/B2G.git
> https://git.mozilla.org/b2g/B2G 
git://github.com/mozilla-b2g/B2G.git -> git+ssh://git.m.o/b2g/B2G.git

> https://git.mozilla.org/build/tooltool.git
Er, that version has been out of date for a long time (May 2015). The
"live" version is at https://github.com/mozilla/build-tooltool

Per that README, only the one file is needed - ni :dustin to see if he has a suggestion on 'proper usage'
Flags: needinfo?(dustin)
I don't know if anything is still using that cache?  I think we bake tooltool into the image now:

testing/docker/desktop-build/Dockerfile
# install tooltool directly from github where tooltool_wrapper.sh et al. expect
# to find it
RUN wget -O /builds/tooltool.py https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py
RUN chmod +x /builds/tooltool.py

However, I don't know if there are other things outside of Gecko using the tc-vcs repo.
Flags: needinfo?(dustin)
(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> I don't know if anything is still using that cache?  I think we bake
> tooltool into the image now:
> 
> testing/docker/desktop-build/Dockerfile
> # install tooltool directly from github where tooltool_wrapper.sh et al.
> expect
> # to find it
> RUN wget -O /builds/tooltool.py
> https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py
> RUN chmod +x /builds/tooltool.py
> 
> However, I don't know if there are other things outside of Gecko using the
> tc-vcs repo.

We do have a few things specifying git.m.o in tree though for tooltool:

https://dxr.mozilla.org/mozilla-central/search?q=git.mozilla.org%2Fbuild%2Ftooltool&redirect=true

Such as: https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/scripts/builder/install-packages.sh#10
(In reply to Wander Lairson Costa [:wcosta] from comment #0)
> git.m.o will shutdown. The following repositories are cached by
> taskcluster-vcs, we need to either remove them or redirect to appropriate
> alternative repos:
> https://git.mozilla.org/external/google/gerrit/git-repo.git

Okay, this one has some lore. We stopped mirroring when upstream broke our builds:
  https://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-configs/rev/68c767f8bebb
I can't find a bug that was filed with more details - I do recall we did a hard reset on the server repo to pin the version (otherwise the 'repo' script tried to update itself). iirc, trees were closed, but I find no bug for that, either. It was hectic!

The current heads on the server are:
[root@git1.dmz.scl3 git-repo.git]# git branch -av
  maint  34acdd2 Fix ManifestParseError when first child node is comment
* master 503d66d Merge "project.RemoteFetch: Handle depth cases more robustly"
  stable 4ccad75 Fix substitution err for schemeless manifest urls

Original upstream URL:
  https://gerrit.googlesource.com/git-repo
  
ni: Catlee for more on the lore side of this. He was in the thick of it.
Flags: needinfo?(catlee)
(In reply to Justin Wood (:Callek) from comment #3)
> 
> We do have a few things specifying git.m.o in tree though for tooltool:
> 
> https://dxr.mozilla.org/mozilla-central/search?q=git.mozilla.
> org%2Fbuild%2Ftooltool&redirect=true
> 
> Such as:
> https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/scripts/
> builder/install-packages.sh#10

This seems not-a-taskcluster issue, so made it's own bug 1277957
This leaves only gerrit repo. It feels like the official repository is this: https://gerrit.googlesource.com/gerrit/

I am not sure why do need the gerrit repository, however. I guess this is a Connected Devices dependency.
Bug 1007259 has some of the context, and links to other bugs where we were having problems due to 'repo' updating.

I don't think we need it now.
Flags: needinfo?(catlee)
So can this bug be closed now?
Flags: needinfo?(wcosta)
(In reply to Jonathan Griffin (:jgriffin) from comment #8)
> So can this bug be closed now?

Hrm, good question, I will assign to myself and check it.
Assignee: nobody → wcosta
Status: NEW → ASSIGNED
Flags: needinfo?(wcosta)
git.m.o will be gone, so we move repo caches to other repositories.
Additionally, we remove the deprecated cache.js module.
Attachment #8765967 - Flags: review?(garndt)
Comment on attachment 8765967 [details] [review]
Remove git.m.o references from cache update

r+, deployed 2.3.38-rc.1 to test this out.
Attachment #8765967 - Flags: review?(garndt) → review+
leaving the bug open because we need to release a new version of taskcluster-vcs and ship in the relevant docker images.
Keywords: leave-open
Attached file Update docker image
Attachment #8766343 - Flags: review?(garndt)
Comment on attachment 8766343 [details] [review]
Update docker image

Looks good to me, just needs to be built and pushed before the tc-hook is updated to use that new image.  Let me know if you need me to assist with any of that.
Attachment #8766343 - Flags: review?(garndt) → review+
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Keywords: leave-open
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.