1330837 - Update tools tests to use mercurial 4.1

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Nick Thomas [:nthomas] (UTC+12)]

We upgraded from Mercurial 3.7.3 to 3.9.1 in Sep/Oct 2016 (bug 1298976) but the tools tests still use 3.7.3. See tox.ini in the base of tools for where we specify 3.7.3.

We'll have to fix this:

23) ERROR: testApplyAndPushRebaseFails (mozilla_buildtools.test.test_util_hg.TestHg)

   Traceback (most recent call last):
    lib/python/mozilla_buildtools/test/test_util_hg.py line 660 in testApplyAndPushRebaseFails
      lambda r, a: c(r, a, self.repodir), max_attempts=3)
    lib/python/util/hg.py line 564 in apply_and_push
      raise HgUtilError("Failed to push")
   HgUtilError: Failed to push
   -------------------- >> begin captured logging << --------------------
   util.commands: INFO: command: START
   util.commands: INFO: command: /Users/nthomas/source/hgmo/build/tools/lib/python/mozilla_buildtools/test/init_hgrepo.sh /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo
   util.commands: INFO: command: cwd: /private/var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5
   util.commands: INFO: command: output:
   util.commands: INFO: command: END (1.00s elapsed)

   util.commands: INFO: command: START
   util.commands: INFO: command: hg log -R /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo --template {node|short}

   util.commands: INFO: command: cwd: /private/var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.25s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: d9647854da5c
   708108b19859
   f9129973d36c
   f99e43af0df3

   redo: DEBUG: attempt 1/2
   util.commands: INFO: command: START
   util.commands: INFO: command: hg clone --traceback /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: cwd: /private/var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5
   util.commands: INFO: command: env: {'HGPLAIN': '1'}
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.12s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: updating to branch default
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved

   util.commands: INFO: command: START
   util.commands: INFO: command: hg branch
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: env: {'HGPLAIN': '1'}
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.09s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: default

   util.commands: INFO: command: START
   util.commands: INFO: command: hg update -C
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: output:
   util.commands: INFO: command: END (0.12s elapsed)

   util.commands: INFO: command: START
   util.commands: INFO: command: hg parent --template {node|short}
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: env: {'HGPLAIN': '1'}
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.10s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: d9647854da5c
   util.commands: INFO: command: START
   util.commands: INFO: command: hg branch
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: env: {'HGPLAIN': '1'}
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.09s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: default

   util.commands: INFO: command: START
   util.commands: INFO: command: hg tag -f TEST
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: output:
   util.commands: INFO: command: END (0.14s elapsed)

   util.commands: INFO: command: START
   util.commands: INFO: command: hg tag -f CONFLICTING_TAG
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo
   util.commands: INFO: command: output:
   util.commands: INFO: command: END (0.14s elapsed)

   util.commands: INFO: command: START
   util.commands: INFO: command: hg -q out --template "{node} {branches}
   " /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: env: {'HGPLAIN': '1'}
   util.commands: DEBUG: Process returned 0
   util.commands: INFO: command: END (0.10s elapsed)

   util.commands: INFO: command: output:
   util.commands: INFO: e3d64a40f316ba041c7ed41e09d0b5da4813704e

   util.commands: INFO: command: START
   util.commands: INFO: command: hg push --traceback --new-branch /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo
   util.commands: INFO: command: cwd: /var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/wc
   util.commands: INFO: command: output:
   util.commands: INFO: command: ERROR
   Traceback (most recent call last):
     File "/Users/nthomas/source/hgmo/build/tools/lib/python/util/commands.py", line 52, in run_cmd
       return subprocess.check_call(cmd, **kwargs)
     File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
       raise CalledProcessError(retcode, cmd)
   CalledProcessError: Command '['hg', 'push', '--traceback', '--new-branch', '/var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo']' returned non-zero exit status 255
   util.commands: INFO: command: END (0.13s elapsed)

   util.hg: DEBUG: Hit error when trying to push: Command '['hg', 'push', '--traceback', '--new-branch', '/var/folders/gr/f_63bw8j6jx1kbs37gfsvhf00000gn/T/tmplsLVK5/repo']' returned non-zero exit status 255

Comment 1

[Kim Moir [:kmoir] ET]

Alin you asked about this in our stand up today but I don't see any messages about the errors your ran into.  Could you add them to help debug?

Comment 2

[Alin Selagea [:aselagea]]

What I'd really like to know here is how to run these tests and how are they used in production.

Here are the steps that I followed:
    1. Cloned /tools repo
    2. Created a new py27 environment inside that repo and activated that env
    3. Ran “python setup.py develop”
    4. Ran "pip install tox"
    5. Ran “tox”

I tried 3 scenarios to run these tests:
    - on my local machine (using Cygwin): I managed to follow all the steps above, but I received something like: “360 tests run in 649.7 seconds. 3 FAILED, 6 errors (351 tests passed)” (full log attached). Nick's test failure is not among the ones that failed in my case
    - on a CentOS local VM: it doesn't manage to install one of the dependencies.

    gevent/libevent.h:9:19: fatal error: event.h: No such file or directory
     #include "event.h"
                       ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    - on a CentOS loaner ("dev-linux64-ec2-aselagea"): I had to manually install/update some packages before being able to install tox and run it. And even so, it will stuck installing the first dependency:

Collecting nose==1.3.0
  Could not find a version that satisfies the requirement nose==1.3.0 (from versions: 1.3.7)
No matching distribution found for nose==1.3.0
 
I feel like I'm missing something here though.

Comment 3

[Alin Selagea [:aselagea]]

Attachment: cygwin_log.txt

Comment 4

[Kim Moir [:kmoir] ET]

I'm on a mac.  I ran this to fix the error above

brew install libevent

but they ran into another error with another lib that apparently isn't supported on mac.  So I would suggest checking out a loaner, running the steps above, and then adding the new libraries as required by the new python version

Comment 5

[Gregory Szorc [:gps]]

4.1 is now deployed in many parts of automation. So when we bump 3.7.3 to something, it should be 4.1.

Comment 6

[Gregory Szorc [:gps]]

This bug has been idle for 3 months and I got bit by this today. So I'm stealing it.

Comment 7

[Gregory Szorc [:gps]]

Attachment: Bug 1330837 - Remove functionality to apply and push;

Tests based on this code fail in Mercurial 3.8+ due to `hg rebase`
changing its behavior for selecting default revisions. I was going
to update the code to work with 3.8+. However, I could find no actual
consumers of this method! Annotate says this was added back in
mozharness 0.4. However, I can't find a reason for it being added.
Nor can I find any consumers of this method in the mozharness repo
outside the tests. This smells like dead code to me.

Review commit: https://reviewboard.mozilla.org/r/129124/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/129124/

Comment 8

[Gregory Szorc [:gps]]

Attachment: Bug 1330837 - Use Mercurial 4.1 in mozharness tests;

Automation is now largely using Mercurial 4.1 with some lagging
components still on 3.9 and a very small sliver of random parts
still on 3.7.3. Let's update the mozharness tests to match what
automation is using.

FWIW, the Mercurial tests still pass on 3.9.

Review commit: https://reviewboard.mozilla.org/r/129126/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/129126/

Comment 9

[Aki Sasaki (not active)]

Comment on attachment 8857198 [details]
Bug 1330837 - Remove functionality to apply and push;

https://reviewboard.mozilla.org/r/129124/#review131634

It looks like we don't use the mozharness apply_and_push anywhere.

Comment 10

[Aki Sasaki (not active)]

Comment on attachment 8857199 [details]
Bug 1330837 - Use Mercurial 4.1 in mozharness tests;

https://reviewboard.mozilla.org/r/129126/#review131640

I was wondering if we needed to upload 4.1.2 to internal pypi, but it looks like 3.7.3 isn't there, so this should be status quo.
It looks like we can install 3.7.3 and 3.9.1 via puppet RPMs on releng infra.

Comment 11

[Gregory Szorc [:gps]]

I need to file bugs to get buildbot infra on 4.1. So far, I haven't cared enough. And I'm still not convinced it is worth caring.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a6a2df0ee343
Remove functionality to apply and push; r=aki
https://hg.mozilla.org/integration/autoland/rev/f7b367022241
Use Mercurial 4.1 in mozharness tests; r=aki

Comment 13

[Aki Sasaki (not active)]

We probably still need something for https://hg.mozilla.org/build/tools/file/tip/lib/python/util/hg.py#l536 and  https://hg.mozilla.org/build/tools/file/tip/lib/python/mozilla_buildtools/test/test_util_hg.py#l623 .

Comment 14

[Gregory Szorc [:gps]]

scripts/release/tag-release.py from build/tools references apply_and_push(). So I'm hesitant to remove it.

Comment 15

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/a6a2df0ee343
https://hg.mozilla.org/mozilla-central/rev/f7b367022241

Comment 16

[Aki Sasaki (not active)]

(In reply to Gregory Szorc [:gps] from comment #14)
> scripts/release/tag-release.py from build/tools references apply_and_push().
> So I'm hesitant to remove it.

Understood.  However, the summary of this bug is to fix the tools tests, and we've fixed the mozharness tests; should we reopen or file new to continue to track the tools tests?

Comment 17

[Gregory Szorc [:gps]]

(In reply to Aki Sasaki [:aki] from comment #16)
> Understood.  However, the summary of this bug is to fix the tools tests, and
> we've fixed the mozharness tests; should we reopen or file new to continue
> to track the tools tests?

Bleh. I didn't notice. I forgot we had 2 things that were essentially the same.

Yes, we should open a new bug to track getting tools upgraded. But this will be more difficult since I believe some things running with tools are still clinging to an old Mercurial version :(

Comment 18

[Aki Sasaki (not active)]

Cloned to bug 1356058.