1186676 - conversion of partner-repacks to github

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Shane Caraveo (:mixedpuppy)]

Attachment: repackfix

Move partner-repacks to a private github repo.  This repo will contain git submodules for each parter repo so a single clone/pull will contain/update everything needed to do a full run of repacks.

- partner-repacks has been imported (with history) to https://github.com/mozilla-partners/partner-repacks
- each existing partner has also been imported individually (without history) to individual private repos in mozilla-partners
- submodules added to git partner-repacks repo, old directory structure largely removed

The directory structure has changed slightly to support this.   Attached patch fixes partner-repacks.py to work with the new directory structure.

Comment 1

[[e-mail] (:bhearsum)]

Can we get release folks added to the access lists of these repositories? That's the following people:
bhearsum
nthomas-mozilla
rail
hwine

Comment 2

[Chris Cooper [:coop] (he/him)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #0)
> Move partner-repacks to a private github repo.  This repo will contain git
> submodules for each parter repo so a single clone/pull will contain/update
> everything needed to do a full run of repacks.

I spoke to catlee today about submodules because he has more experience there than I do. submodules are apparently pretty fragile. Have we considered alternatives like git subtree or google's repo as is used for b2g?

Comment 3

[Shane Caraveo (:mixedpuppy)]

(In reply to Ben Hearsum [:bhearsum] from comment #1)
> Can we get release folks added to the access lists of these repositories?
> That's the following people:
> bhearsum
> nthomas-mozilla
> rail
> hwine

@bhearsum, you should have admin privi on the mozilla-partners account, but I've added nthomas and hwine who were missing from the list

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #3)
> (In reply to Ben Hearsum [:bhearsum] from comment #1)
> > Can we get release folks added to the access lists of these repositories?
> > That's the following people:
> > bhearsum
> > nthomas-mozilla
> > rail
> > hwine
> 
> @bhearsum, you should have admin privi on the mozilla-partners account, but
> I've added nthomas and hwine who were missing from the list

Hmm, this might be PEBKAC, but I only see two repos on https://github.com/mozilla-partners, and https://github.com/mozilla-partners/partner-repacks gives me a 404.

Comment 5

[Chris Cooper [:coop] (he/him)]

(In reply to Ben Hearsum [:bhearsum] from comment #4)
> Hmm, this might be PEBKAC, but I only see two repos on
> https://github.com/mozilla-partners, and
> https://github.com/mozilla-partners/partner-repacks gives me a 404.

Shane's still in the process of moving things. I suspect he has it as local changes right now.

Comment 6

[Shane Caraveo (:mixedpuppy)]

Look now.  I just added all the repos to the "mozilla" team, which is probably what the problem was

Comment 7

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8637511 [details] [diff] [review]
repackfix

Review of attachment 8637511 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with nits fixed.

::: scripts/partner-repacks.py
@@ +875,5 @@
>                  continue
> +
> +        for f in files:
> +            if f == 'repack.cfg':
> +                print root, _, f

Don't need the print in production.

@@ +879,5 @@
> +                print root, _, f
> +                partner_dirs[os.path.split(root)[-1]] = (root, os.path.join(root, f))
> +                # partner_dirs.append(root)
> +    # print partner_dirs
> +    # sys.exit(0)

These 3 commented lines can disappear.

Comment 8

[Hal Wine [:hwine] use NI!]

Shane - when you're ready for the old hg repos to go read only, open a bug against Developer Services :: Mercurial: hg.mozilla.org

Comment 9

[[e-mail] (:bhearsum)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #6)
> Look now.  I just added all the repos to the "mozilla" team, which is
> probably what the problem was

Got it, thanks!

Comment 10

[Shane Caraveo (:mixedpuppy)]

ok, I have this working using google repo (a modified version to fix private repo's).  Just go to https://github.com/mozilla-partners/repack-manifests and follow the readme (copy/paste) to pull everything down.  You'll end up with a structure similar to partner-repacks repo.  Just run scripts/partner-repacks.py as usual and everything "should" work.

@hwine I'm happy to move to this after the 40.0b8 builds are done, given someone else gives this a test spin.

Comment 11

[Hal Wine [:hwine] use NI!]

(In reply to Shane Caraveo (:mixedpuppy) from comment #10)
> @hwine I'm happy to move to this after the 40.0b8 builds are done, given
> someone else gives this a test spin.

Bug 1188219 is opened to let you give the okay when developer services can make that change.

Comment 12

[Shane Caraveo (:mixedpuppy)]

see comment #10

Comment 13

[Chris Cooper [:coop] (he/him)]

I've already tested the setup steps in https://github.com/mozilla-partners/repack-manifests#readme and verified that they work.

From the releng-side, I need to replace the existing hg checkout of the partner-repacks ScriptFactory with the git setup steps, This may require a migration to mozharness.

Comment 14

[Shane Caraveo (:mixedpuppy)]

IMO we can make the mercurial repo read-only at this point.

Comment 15

[Chris Cooper [:coop] (he/him)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #14)
> IMO we can make the mercurial repo read-only at this point.

I agree, especially if we need to further mirror github back into git.m.o.

Comment 16

[Shane Caraveo (:mixedpuppy)]

@hwine ping

Comment 17

[Hal Wine [:hwine] use NI!]

As noted in comment 11, bug 1188219 is where the actual setting to read only will occur. But only after this bug is r/f (it's the only reasonable way to handle cross product/team handoffs)

As I understand the situation, there is no functional problem with leaving the repos r/w, it's just cleanup we need to do after this bug is closed.

Is there any other reason to change the repos to r/o before this bug is resolved? If so, removing the dependency will free things up (best done from bug 1188219 so it's clear it's a "go")

Comment 18

[Shane Caraveo (:mixedpuppy)]

@hwine, sorry, had dependencies backwards.

I am r/f with this bug, @coop is building from the git repos and there are other bugs for various pieces left for getting automation done.