Closed Bug 929338 Opened 6 years ago Closed 6 years ago

rebase script to help developers move from github/mozilla/mozilla-central to github/mozilla/integration-gecko-dev

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aki, Assigned: johns)

References

()

Details

Attachments

(1 file)

Developers have code based off the github/mozilla/mozilla-central SHAs, and they need to be rebased to live on the github/mozilla/integration-gecko-dev SHAs, hopefully in an automated fashion.
Note: the script here also needs to update the branch upstreams as well.
Blocks: 847643
Let me know if you need help figuring out how this should work.

Ideally the script should check all your branches for old shas, and give you the exact rebase commands you should run to move them over. It should also prompt to remove the old remote, and check that you don't have any remaining tags pointing to the old shas.

Another issue is going to be pruning the old shas - once they become unreachable in the local repo, the next git-gc will try to unpack *all* of the old commit objects out of pack files for eventual pruning, which was not a process designed with 3.6million dead objects in mind - and will probably result in a 70GiB .git folder (I haven't tried it)

The workaround for this is to have the script make sure the old shas are unreachable (no branches, tags, or reflog entries), and then run a |git gc --prune=now| to ensure the unreachables are deleted straight away.

Losing your reflog entries is non-ideal -- working around this could be iffy. One idea would be sticking a gc-hook in .git/config when you run the migration script that checks if the old shas have become unreachable and manually git-prunes them. Ew.
That's definitely far beyond what I was going to do... I'm mainly working on a short number of commands that will get any branch rebased onto master (or a different branch of your choice), and that's it.

So yes, if you want to put some time into this so the script matches your expectations, that would be great.
In fact, from an email thread, it's looking like you're willing to take this bug?
I'm trying to figure it out at the moment, but it's slow going.

It looks like we're getting pressure to have this live very soon, as in possibly this week.  a) are you willing to take this bug, and b) how long would it take you?
Flags: needinfo?(jschoenick)
Sure, I can take this and try to get something put together this week -- do we have any specific requirements not mentioned here?
Flags: needinfo?(jschoenick)
Awesome.
I would guess your proposal would cover most if not all of it.
If there are requirements not stated here: https://groups.google.com/forum/#!msg/mozilla.dev.planning/hfZfm7vAHKA/fJNSQFlyUtIJ then asking for people to comment here might help flush out more requirements.
Assignee: aki → jschoenick
FYI, we're planning to rename integration-gecko-{dev,projects} to gecko-{dev,projects} in bug 935403.  The SHAs should stay the same, so hopefully this has minimal impact here.
I should have a beta-ish version of this script in the next day or two
Status: NEW → ASSIGNED
Sorry for the delay, I ran into an issue with automatically mapping old commits to new commits for rebasing without pulling huge map files, but I believe I have that solved now. I'll have a beta version of the script up shortly, we should find some people who know their way around git to try it out before firing it off to the mailing lists, however.
This is still pretty WIP-y, but I'm heading out for the day:

https://github.com/Nephyrin/moz-git-migrator/blob/master/moz-git-migrator.sh

Run it with ./moz-git-migrator ./my-repo -- It should *not* touch your repo at all, rather, just give you the commands you need to rebase your branches and continue.

I still need to finish up the help text and add the config setting to prevent GC issues (important), so don't use this just yet unless you know what you're doing in git!

I'll finish this up and test it on several repos I have tomorrow morning, if all goes well I can make a newsgroup post inviting users to try it out.
I polished this up a lot more today and tried to catch any edge-cases I could think of that might cause people to shoot themselves in the foot. I'm going to fish around for a few people to guinea pig this, and will make a mailing list post if no issues turn up.
Attached file First try
Here's my terminal output from the first try of this script.
This is working much better now, and i've had a few people test run it on their complex repos successfully. I made a post to the thread on the lists:
https://groups.google.com/forum/#!topic/mozilla.dev.planning/rkyKnCMQmYg

I'm going to call this fixed, but I'll keep an eye out for any issues and update as necessary.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Hi guys,

I had the previous repo, made a few commits and then rebased to create a new patch. However, this last rebase apparently removed everything, as the repo was retired. What can I do to recover the changes I made?
(In reply to comment #14)
> Hi guys,
> 
> I had the previous repo, made a few commits and then rebased to create a new
> patch. However, this last rebase apparently removed everything, as the repo was
> retired. What can I do to recover the changes I made?

I have a backup repo hosted here with the exact same content: https://github.com/ehsan/mozilla-central
(And sorry for the pain)
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.