764284 - hgtool.py isn't working correctly for releases (fails to pull release branch)

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Mark Banner (:standard8)]

hgtool.py is currently used on the source step. However it isn't working correctly, and will always clone the repository rather than using the share.

I've see two versions of logs.

1) If the hg share for the repository doesn't exist, hgtool will clone the whole repository to the share location and do a share off of that.

This works fine as cloning the whole repository gets everything including all branches.

2) If the hg share for the repository does exist, the following happens:

command: START
command: hg pull -b default http://hg.mozilla.org/releases/mozilla-beta
command: cwd: /builds/hg-shared/releases/mozilla-beta
command: output:
pulling from http://hg.mozilla.org/releases/mozilla-beta
searching for changes
no changes found
command: END (1.86s elapsed)

Trying to share /builds/hg-shared/releases/mozilla-beta to /builds/slave/rel-m-beta-firefox-source/mozilla-beta
command: START
command: hg share -U /builds/hg-shared/releases/mozilla-beta /builds/slave/rel-m-beta-firefox-source/mozilla-beta
command: cwd: /builds/slave/rel-m-beta-firefox-source
command: output:
command: END (0.11s elapsed)

command: START
command: hg update -C -r FIREFOX_14_0b7_RELEASE
command: cwd: /builds/slave/rel-m-beta-firefox-source/mozilla-beta
command: output:
abort: unknown revision 'FIREFOX_14_0b7_RELEASE'!
command: END (27.90s elapsed)

Error updating /builds/slave/rel-m-beta-firefox-source/mozilla-beta from sharedRepo (/builds/hg-shared/releases/mozilla-beta): 
command: START
command: hg clone http://hg.mozilla.org/releases/mozilla-beta /builds/slave/rel-m-beta-firefox-source/mozilla-beta
command: cwd: /builds/slave/rel-m-beta-firefox-source
command: output:
requesting all changes
adding changesets
adding manifests
adding file changes
added 96053 changesets with 453822 changes to 93742 files (+207 heads)
updating to branch default
58052 files updated, 0 files merged, 0 files removed, 0 files unresolved
command: END (389.90s elapsed)


Hence the share has "hg pull -b default" performed on it - because of the branch specification, this doesn't necessarily pull in the revisions from the branches. So the update to the revision that is on the branch then fails, so it pulls the whole repository from scratch.

Comment 1

[Mark Banner (:standard8)]

I think the solution here is if no branch argument is specified to hgtool, then don't put in "default", just leave the branch passed to the mercurial function as None, xref:

http://hg.mozilla.org/build/tools/file/cf15fda270bf/buildfarm/utils/hgtool.py#l22
http://hg.mozilla.org/build/tools/file/cf15fda270bf/buildfarm/utils/hgtool.py#l85

If the branch isn't specified, then hg pull is free to get all branches and hence hg update can then do the right thing.

If we're concerned about that pulling all branches, another option might be to enable clone_by_revision, but I don't think that's quite necessary, and I'm not sure it would work with tags.

Comment 2

[[e-mail] (:bhearsum)]

That seems reasonable to me. Any thoughts, Rail or Chris?

Comment 3

[Rail Aliiev [:rail]]

I think that hgtool.py shouldn't decide instead of you what branch to pull from. If you want to pull from a branch, just use -b! :)

Setting branch=None as a default value sounds OK to me.

Comment 4

[[e-mail] (:bhearsum)]

Attachment: don't set a default branch in hgtool

Comment 5

[[e-mail] (:bhearsum)]

I tried to verify whether or not we always pull by revision or branch, but it's almost impossible to because they're read from a properties file in most cases. Based on the build logs I've looked at it seems like we are in all of the most common cases....