1288981 - Problems pushing to repo after hg convert

Status: RESOLVED WORKSFORME

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Ian Neal]

Created a file map to copy help viewer files from m-c to c-c, and ran an hg convert:
hg convert ~/mozilla-central/ ~/tmp-repo/ --filemap help.map

That completed successfully.
Cloned a fresh c-c using:
hg clone http://hg.mozilla.org/comm-central/ help-cc
and updated .hg/hgrc

Pulled files from temp repo into help-cc:
hg pull -f ~/tmp-repo/
pulling from ~/tmp-repo/
searching for changes
warning: repository is unrelated
requesting all changes
adding changesets
adding manifests
adding file changes
added 47 changesets with 83 changes to 23 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)

Ran hg merge
21 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)

Did an hg commit then an hg push and got the following:
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 48 changesets with 83 changes to 23 files
remote: *** pushing unrelated repository ***
remote: 
remote: Changeset 303534336131 introduces a new root changeset into this repository. This
remote: almost certainly means you accidentally force pushed to the wrong
remote: repository and/or URL.
remote: 
remote: Your push is being rejected because this is almost certainly not what you
remote: intended.
remote: transaction abort!
remote: rollback completed
remote: pretxnchangegroup.single_root hook failed
abort: push failed on remote

Is there something wrong with the process I am using or do I now need special permissions to do this sort of push?

Comment 1

[Ed Morley [:emorley]]

You added another head:
 "(+1 heads)".

You need to use the --splicemap feature to rebase the new changesets onto the old, rather than merging in a completely separate root.

See:
https://www.mercurial-scm.org/wiki/ConvertExtension#A--splicemap

Comment 2

[Ed Morley [:emorley]]

Sorry the +1 head was of course resolved by the merge. I meant added another root (ie changeset with no parents):

remote: Changeset 303534336131 introduces a new root changeset into this repository. This
remote: almost certainly means you accidentally force pushed to the wrong
remote: repository and/or URL.

Comment 3

[Ian Neal]

From the target repo the latest changeset is:
19617:398d89cd600f

From the temporary repo the first changeset is:
0:0543a1643d72

I got the 40 character hashes from running "hg id -r{rev} --debug" on each repo.

At what point would I then make use of the splicemap? Is it something have to do as part of the initial hg convert or can it be done as part of the hg pull, merge and commit?

I have tried running the following:
hg convert --splicemap help.splicemap ~/tmp-repo/ ~/help-cc/
but I ended up with a duplicate set of changesets

Comment 4

[Ed Morley [:emorley]]

As an example, here is a script I've used to import all of one repo into another:
https://emorley.pastebin.mozilla.org/8886706

You could adjust the filemap [1] to only include certain files from the target repository. Note: the script pulls the revisions into the *current* directory repo.

Other than that, I'd ask :gps, since he knows more about Mercurial (I only happened to see this bug in a daily email summary, I only really use Git any more).

[1] https://www.mercurial-scm.org/wiki/ConvertExtension#A--filemap

Comment 5

[Gregory Szorc [:gps]]

What likely happened when you ran `hg convert` is that the history from the source repo got converted to an independent DAG in the new repo. i.e. your repo history looks like:

o
|  o
o  |
|  o
o  |
   o

You then did a `hg merge` to make your history like:

o
|\
o \
|  o
o  |
|  o
o  |
   o

But there are still 2 root changesets here (changesets with no parents). You can find these changesets by running `hg log -r 'roots(all())'`.

We have a global hook on hg.mozilla.org that prevents pushing new roots to repos because this is almost always not what was intended.

As emorley says, you should use a splicemap so the first commit from the source repo you are converting (rev 0 in that repo) is parented on the last commit in the destination repo. That way, when you do the convert your history will look like:

o <- last commit from imported repo
.
.
o <- first commit from imported repo
|
o <- last commit from mozilla-central
.
.
o <- first commit from mozilla-central

Alternatively, if this is a one-time conversion, you can achieve the same result as a splice map by using `hg rebase`. But I recommend using a splice map.

Comment 6

[Ian Neal]

Attachment: repo_copy.sh

This is the amended script I am using.
Once it is completed do I need to do an hg merge before an hg push?

Comment 7

[Gregory Szorc [:gps]]

The script looks mostly OK.

If the repo being imported has multiple heads, the "single head per branch" hook won't let you push because `hg convert` will have converted all heads. If you don't care about those extra heads, you can specify convert.hg.revs (see `hg help convert`) to a revset of source revisions to convert. You probably want something like '::abcdef123' or '::default' or '::tip' to select the ancestors of a specific commit.

Comment 8

[Ian Neal]

I still ending up having to do an hg rebase even after using a splicemap

I'm now attempting a different copy from mozilla-aurora to comm-central. I have checked the source repo and there is only the one head (tip), but once the hg convert is completed it has created 5 heads in the target repo as well as not having a linear history.

I used --config convert.skiptags=true convert.hg.revs='::tip' on the command line but that did not seem to make any difference.

Any suggestions?

Comment 9

[Ian Neal]

Ah my target repo has 5 heads so it is not creating any more, just not splicing.

Comment 10

[Ian Neal]

Attachment: console.map

This is the filemap

Comment 11

[Ian Neal]

Source repo is clone of mozilla-aurora
Target repo is clone of comm-central
Splicemap contains
495261ce314cb0ea17b38be27efc20cd357f23e0 8ba995b74e18334ab3707f27e9eb8f4e37ba3d29

Comment 12

[Gregory Szorc [:gps]]

Your splice map might be backwards.

The first entry in splicemap is a revision in the *source* repo. Subsequent entries are revisions in the *target* repo.

Second, since you are only including some files, I think what's happening is revision 0 from mozilla-aurora (8ba995b74e18334ab3707f27e9eb8f4e37ba3d29) is getting skipped since it doesn't include any converted files. Therefore the splicemap entry doesn't do anything.

The first commit that gets converted from mozilla-aurora is 9b2a99adc05e53cd4010de512f50118594756650.

If I make the splicemap:

9b2a99adc05e53cd4010de512f50118594756650 495261ce314cb0ea17b38be27efc20cd357f23e0

Then the splice works and the first new commit produced is 67b4c24cb24b and its parent is the last commit from comm-central. e.g.

o  changeset:   19670:879c095b1272
|  user:        [e-mail]
|  date:        Thu Mar 22 16:28:22 2007 -0700
|  summary:     Bug 366479 - [mac] Copy command is broken in the JS console. r=gavin,neil. sr=neil.
|
o  changeset:   19669:67b4c24cb24b
|  user:        [e-mail]
|  date:        Thu Mar 22 10:30:00 2007 -0700
|  summary:     Free the (distributed) Lizard! Automatic merge from CVS: Module mozilla: tag HG_REPO_INITIAL_IMPORT at 22 Mar 2007 10:30 PDT,
|
o  changeset:   19668:495261ce314c
|  fxtree:      comm
|  user:        rsx11m <[e-mail]>
|  date:        Tue Jul 12 18:02:36 2016 -0500
|  summary:     Bug 494424 - Update help for geolocation. r=IanN, a=Help change

Comment 13

[Ian Neal]

ok, so switching to revision 1 has helped.
The hg convert works and correctly splices in the history, so hg out now looks correct
Running "hg update" gave:
21 files updated, 0 files merged, 0 files removed, 0 files unresolved
Trying to push gives:
pushing to ssh://[e-mail]/comm-central/
searching for changes
abort: push creates new remote head 11926776c325!
(merge or see "hg help push" for details about pushing new heads)

hg merge gives:
abort: branch 'default' has one head - please merge with an explicit rev
(run 'hg heads' to see all heads)

Comment 14

[Ian Neal]

Attachment: hg_out.txt

Comment 15

[Ian Neal]

Attachment: hg_heads.txt

Comment 16

[Gregory Szorc [:gps]]

My guess is someone else pushed to comm-central?

What does `hg in` say? If it says anything, you'll need to `hg pull` then rebase against that revision. Keep in mind this will screw up your SHA-1 map. If you want to preserve the SHA-1 map, you'll need to obtain a fresh version of comm-central and do the `hg convert` again.

Comment 17

[Gregory Szorc [:gps]]

You may also want to do `hg out -G` to view a graphical display of the DAG. This will make heads very obvious (although from the `hg heads` output it looks like you only have a single outgoing head).

Comment 18

[Ian Neal]

`hg in` has no incoming changes and `hg out -G` confirms only a single outgoing head
Solution would be to `hg push -f`?

Comment 19

[Ian Neal]

Attachment: hg_out_g.txt

Comment 20

[Ian Neal]

clokep suggested `hg push -r .` and that worked

Comment 21

[Gregory Szorc [:gps]]

Are `hg in` and `hg out` operating on the same repo as your `hg push` (which is targeting your user repo)?

Comment 22

[Ian Neal]

(In reply to Gregory Szorc [:gps] from comment #21)
> Are `hg in` and `hg out` operating on the same repo as your `hg push` (which
> is targeting your user repo)?

They were all operating on comm-central.

Anyway it landed successfully with `hg push -r .`