481835 - client.py should use hg pull without -r tip argument

Status: RESOLVED FIXED

(MailNews Core :: Build Config, defect)

Description

[Mark Banner (:standard8)]

Attachment: The fix

Using hg 1.1.2 and trying to pull today's hg via python client.py checkout. We've got:

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/d40f92c60170 ... GECKO191b3_20090304_RELBRANCH tip

on the tip of the repository (currently ), and several changesets back:

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/b5401b3b7b92 ... default

When I do python client.py checkout this gives me:

Executing command: ['hg', 'pull', '-R', './mozilla', '-r', 'tip']
pulling from http://hg.mozilla.org/releases/mozilla-1.9.1/
searching for changes
adding changesets
adding manifests
adding file changes
added 8 changesets with 12 changes to 9 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
Executing command: ['hg', 'update', '-r', 'default', '-R', './mozilla']
abort: crosses branches (use 'hg merge' or 'hg update -C')
Traceback (most recent call last):
  File "client.py", line 254, in <module>
    do_hg_pull('mozilla', options.mozilla_repo, options.hg, options.mozilla_rev)
  File "client.py", line 134, in do_hg_pull
    check_call_noisy(cmd)
  File "client.py", line 52, in check_call_noisy
    check_call(cmd, *args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['hg', 'update', '-r', 'default', '-R', './mozilla']' returned non-zero exit status 255

I used rollback in the mozilla/ directory, then played around a bit.

"hg pull -R . -r tip" gave me:

pulling from http://hg.mozilla.org/releases/mozilla-1.9.1/
searching for changes
adding changesets
adding manifests
adding file changes
added 8 changesets with 12 changes to 9 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)

a subsequent "hg pull" gave me:

pulling from http://hg.mozilla.org/releases/mozilla-1.9.1/
searching for changes
adding changesets
adding manifests
adding file changes
added 7 changesets with 13 changes to 12 files
(run 'hg update' to get a working copy)

"hg update -r default" then worked fine.

Doing an update of a local clone, python client.py checkout update it to:

b5401b3b7b92 tip

which as noted above, isn't the tip, its the default. Using just "hg pull" and "hg update" this became just:

b5401b3b7b92

the same as on my direct mozilla-1.9.1 clone.

I've seen similar problems to these before when we've hit branch points.

I think its pretty clear we should drop the -r argument from hg pull to avoid these problems. Going for dual review for some sanity checking ;-)

Comment 1

[Robert Kaiser]

We run into a major problem with this if we are on a named branch, IIRC, as then we don't pull anything newer from default and we are not able to reasonably switch away from the named branch without a pull -r tip, from what I saw.

Comment 2

[Robert Kaiser]

Comment on attachment 365855 [details] [diff] [review]
The fix

IIRC, Callek had very specific reasons to do it this way, I'd like him to OK this.

Comment 3

[Vlado Valastiak [:wladow]]

Please fix L10n checkouts as well - the c-c L10n tinderboxen are now failing due to GECKO191b3_20090304_RELBRANCH in L10n repos

Comment 4

[Philippe M. Chiasson (:gozer)]

IIRC, the correct way to do this, normally would be
$> hg pull
$> hg update -r default

Unfortunately, there was a HG bug with hg pull (or hg pull -r default) that forced us to go the hg pull -r tip

The background for this hapenned in bug 459396

Comment 5

[Philippe M. Chiasson (:gozer)]

(In reply to comment #3)
> Please fix L10n checkouts as well - the c-c L10n tinderboxen are now failing
> due to GECKO191b3_20090304_RELBRANCH in L10n repos

Can you point to a log of the bustage in question? Looking at things now, things look okay on the l10n front, so once the tip moved back on default, the builders possibly fixed themselves.

Comment 6

[Vlado Valastiak [:wladow]]

(In reply to comment #5)
http://tinderbox.mozilla.org/Mozilla-l10n-sk/ all the c-c builds are burning because they are looking for the L10n files on the tip and that's relbranch now. Yes, once the tip is back on default they'll go green again, but I'm finding the current state at least unlucky.

Comment 7

[Mark Banner (:standard8)]

(In reply to comment #4)
> Unfortunately, there was a HG bug with hg pull (or hg pull -r default) that
> forced us to go the hg pull -r tip
> 
> The background for this hapenned in bug 459396

Hmm you're right, and the hg issue is still open :-(

My only other thought is if hg pull -r default might work.

Comment 8

[Justin Wood (:Callek)]

(In reply to comment #7)
> (In reply to comment #4)
> > Unfortunately, there was a HG bug with hg pull (or hg pull -r default) that
> > forced us to go the hg pull -r tip
> > 
> > The background for this hapenned in bug 459396
> 
> Hmm you're right, and the hg issue is still open :-(
> 
> My only other thought is if hg pull -r default might work.

my thought was that we discussed (me and you) -r default at one point, and I had thought it had already changed to that.

rs+ with s/tip/default/ from me on that line....

If the HG bug is fixed in 1.1.2 and official MozBuild is updated to use at least that version then I'm ok with changing this to drop the -r argument altogether. (but I'd rather avoid dropping the workaround if possible)

Comment 9

[Robert Kaiser]

The locale build issue isn't client.py actually, but buildbot config, right?

Comment 10

[Philippe M. Chiasson (:gozer)]

(In reply to comment #9)
> The locale build issue isn't client.py actually, but buildbot config, right?

Correct.

Comment 11

[Philippe M. Chiasson (:gozer)]

Of course, looking at this, it's in the guts of buildbot's Mercurial code.

Looks like it might be possible to trick buildbot with:

Mercurial(..., revision='default')

I am trying that right now, let's see if it helps.

Comment 12

[Robert Kaiser]

Comment on attachment 365855 [details] [diff] [review]
The fix

I'll leave the judgement to Callek when it comes to this, he knows client.py better than me at this point.
Note that the locale stuff is a different issue that doesn't affect client.py but only buildbot configs.

Comment 13

[Justin Wood (:Callek)]

Comment on attachment 365855 [details] [diff] [review]
The fix

r+ on the condition we no longer even need the workaround (as in, fixed in current mercurial) _AND_ if fixed in whatever version MozillaBuild on windows ships.

Can you please provide a link to upstream bug all over again so (we) can verify that?

Comment 14

[Mark Banner (:standard8)]

Upstream issue(s):

http://www.selenic.com/mercurial/bts/issue1313
http://www.selenic.com/mercurial/bts/issue1320

Looks like it got fixed for 1.1, but its mainly a server side fix. From what I can tell hg.mozilla.org is on 1.0 therefore this depends on waiting for the server to be upgraded - bug 483782.

(or we try s/tip/default/ in the pull command).

Comment 15

[Justin Wood (:Callek)]

(In reply to comment #14)
> (or we try s/tip/default/ in the pull command).
rs+ if you want to do that.

Comment 16

[Mark Banner (:standard8)]

(In reply to comment #15)
> (In reply to comment #14)
> > (or we try s/tip/default/ in the pull command).
> rs+ if you want to do that.

I think comment 4 indicates this won't work. So we'll just have to stick with it for now.

Comment 17

[Mark Banner (:standard8)]

Comment on attachment 365855 [details] [diff] [review]
The fix

Dropping review as this is a future patch.

Comment 18

[Mark Banner (:standard8)]

Bug 483782 has now been fixed, so I think I we should give this a try.

Whilst I'd like to do it before TB3b3/SM2b1 so that we've got sane handling around branches, I think it is best to do it after, before we re-open the tree, maybe giving it half a day to check stability of the trees.

Any objections? If not I'll queue it up for after the freeze.

Comment 19

[Dan Mosedale (:dmosedale, :dmose)]

Sounds fine to me.

Comment 20

[Mark Banner (:standard8)]

This has now been checked in and is running fine so far, leaving open whilst it is baking a bit:

http://hg.mozilla.org/comm-central/rev/e9c696610b61

Comment 21

[Mark Banner (:standard8)]

Hmm, I've just found this again. I think we've generally been ok since this landed, maybe one or two failures but nothing recent or frequent. So closing this as fixed.