1060195 - comm-esr31 does wasteful things while cloning mozilla-esr31

Status: RESOLVED WORKSFORME

(Release Engineering :: Release Automation: Other, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Saw some strangeness in the en-US 31.1.0 build2. make.py is called like this:

'python' 'c:/builds/moz2_slave/tb-rel-c-esr31-w32_bld-0000000/build/build/pymake/make.py' '-f' 'client.mk' 'build' u'MOZ_BUILD_DATE=20140828165811'

It clones mozilla-esr31 and ldap-sdks, and updates both to the appropriate tag. Then this runs

c:\builds\moz2_slave\tb-rel-c-esr31-w32_bld-0000000\build\client.mk:455:0$ python c:/builds/moz2_slave/tb-rel-c-esr31-w32_bld-0000000/build/client.py checkout --hg-options='--time' --hgtool=../tools/buildfarm/utils/hgtool.py --hgtool1=../scripts/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print

It wants to try to use hg share and says

hg path isn't correct (https://hg.mozilla.org/releases/mozilla-esr31/ should be https://hg.mozilla.org/releases/mozilla-esr31); clobbering
Using _rmtree_windows ...
Updating shared repo

Then it goes on get a working copy, update to tag, and to compile. We're probably wasting 30 mins or more on this.

Comment 1

[Mark Banner (:standard8)]

Ok, this is a side-effect of two things. There's a changeset I'm generally landing each cycle that stops l10n using hgtool (because of previous issues with hgtool & l10n):

http://hg.mozilla.org/releases/comm-beta/rev/98d598127699

Due to the way our clone and pull system works, we currently end up pulling twice.

Additionally, there's code in client.py to strip the trailing slash for hgtool builds:

https://hg.mozilla.org/releases/comm-esr31/file/ab4cdce19931/client.py#l334

There's probably a few potential solutions (or combinations) here:

- Find a way to not do the pull cycle twice
- Always drop the trailing slash, not just for hgtool
- Find out what's going on with L10n builds, and why hgtool doesn't work.

Tentatively tagging myself with looking at this in the next release cycle.

Comment 2

[Mark Banner (:standard8)]

Unfortunately, I'm unlikely to find time to work on this in the short term.

cc'ing some folks who might be interested - this would be a big win for build time and would probably reduce the time for release builds a bit.

Most of the details are in comment 1.

Comment 3

[Mark Banner (:standard8)]

Note: this also causes us to need to transplant:

http://hg.mozilla.org/releases/comm-beta/rev/3ae4f13858fd

for every beta cycle.

Comment 4

[Mark Banner (:standard8)]

Bug 757798 is the L10n hgtool.py issue.

Comment 5

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

We seem to be using hgtool for l10n builds on both beta and central now and the "hg path isn't correct" message does not show during repacks. Therefore I think we can close this issue.