764323 - client.py needs to pass the revision argument to hgtool.py

Status: RESOLVED FIXED

(MailNews Core :: Build Config, defect)

Description

[Mark Banner (:standard8)]

hgtool.py can take a revision argument, and will also do a reasonable job of ensuring the end result has the right revision, including cloning from scratch if necessary.

I think if we've got a revision specified, we should just let hgtool.py do the work rather than try and do it ourselves in client.py.

Comment 1

[Mark Banner (:standard8)]

I believe this doesn't need to block bug 757798 as bug 764284 is fixing the fundamental issue of hgtool.py being wrong. This bug is further bullet-proofing on top of fixing those bugs.

Comment 2

[Philipp Kewisch [:Fallen][☁️📆]]

Attachment: Fix - v1

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on c-c, etc.): 
Risk to taking this patch (and alternatives if risky):

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on c-c, etc.): 
Risk to taking this patch (and alternatives if risky):

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on c-c, etc.): 
Risk to taking this patch (and alternatives if risky):

I am using this bug to fix the remains of bug 757798, and the current failures we have been getting for a while:

command: START
command: hg pull --traceback -b releases/comm-beta https://hg.mozilla.org/releases/mozilla-beta
command: cwd: /builds/hg-shared/releases/mozilla-beta
command: env: {'HGPLAIN': '1'}
Hit exception running hg:
Traceback (most recent call last):
  File "../scripts/buildfarm/utils/../../lib/python/util/hg.py", line 99, in get_hg_output
    return get_output(['hg'] + cmd, timeout=timeout, env=env, **kwargs)
  File "../scripts/buildfarm/utils/../../lib/python/util/commands.py", line 207, in get_output
    raise error
CalledProcessError: Command '['hg', 'pull', '--traceback', '-b', u'releases/comm-beta', 'https://hg.mozilla.org/releases/mozilla-beta']' returned non-zero exit status 255
Error pulling changes into /builds/hg-shared/releases/mozilla-beta from https://hg.mozilla.org/releases/mozilla-beta; clobbering
command: START
command: hg clone --traceback -U https://hg.mozilla.org/releases/mozilla-beta /builds/hg-shared/releases/mozilla-beta
command: cwd: /builds/slave/tb-rel-c-beta-m64_rpk_2-000000/comm-beta
command: env: {'HGPLAIN': '1'}
command: END (171.73s elapsed)

This problem occurs since the environment has PROPERTIES_FILE set, which hgtool uses if no revision/branch is specified. The sourcestamp section of the properties file has revision = null, so the branch is used instead, which in the properties file is releases/comm-beta.

The patch fixes this by making sure PROPERTIES_FILE is unset and also passing the revision to hgtool explicitly.

Comment 3

[Justin Wood (:Callek)]

Comment on attachment 8658652 [details] [diff] [review]
Fix - v1

Review of attachment 8658652 [details] [diff] [review]:
-----------------------------------------------------------------

Looks sane enough, sadly though I don't have the bmo magic to approve.

Comment 4

[Kent James (:rkent)]

Why after 3 years is this problem now causing failures? Did something change in the build infrastructure? If so, does that change affect all of our branches?

Comment 5

[Philipp Kewisch [:Fallen][☁️📆]]

It has been broken all along, and has also been causing failures all along. It just seems the failures didn't cause any issues. This is "just cleanup", but it would make sure that the correct revisions are pulled during repacks. If you are not comfortable with pushing to comm-esr38, I'd at least ask for beta approval since it will only really be clear if it works in beta l10n repacks. If it works there next cycle, we could backport it to esr38 (if we haven't forgotten).

Comment 6

[Kent James (:rkent)]

Comment on attachment 8658652 [details] [diff] [review]
Fix - v1

That sounds like a good idea. We look at beta before deciding what to do with esr38

Is this involved at all with the issue of the try server not reliably switching back to the default mozilla repo revision after a test that changed it?

Comment 7

[Philipp Kewisch [:Fallen][☁️📆]]

(In reply to Kent James (:rkent) from comment #6)
> Is this involved at all with the issue of the try server not reliably
> switching back to the default mozilla repo revision after a test that
> changed it?

Nope, that is bug 1154222. I can spin up a patch there that uses MOZ_AUTOMATION to automatically remove hgtool, add apply-patches, and clean out the patches.

Comment 8

[Philipp Kewisch [:Fallen][☁️📆]]

Pushed to comm-central changeset 917c6e36e640

Comment 9

[Philipp Kewisch [:Fallen][☁️📆]]

Backported to releases/comm-aurora changeset d274c6770247

Comment 10

[Philipp Kewisch [:Fallen][☁️📆]]

Backported to releases/comm-beta changeset 0ac48cae54bf

Comment 11

[Philipp Kewisch [:Fallen][☁️📆]]

We've successfully build TB 41.0b2 with this patch, can you complete the approval for esr38?

Comment 12

[Philipp Kewisch [:Fallen][☁️📆]]

Backported to releases/comm-esr38 changeset d201ce27282f