1245008 - Something is wrong with hg on try on windows

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Mike Hommey [:glandium]]

I have 3 builds on Windows on try that took close to an hour(!) to fail during configure (voluntary failure, at that). Turns out they spent an awful lot of time doing mercurial stuff:

First chunk of 20 minutes spent clobbering because mozilla-central != try:

21:42:33     INFO -  command: START
21:42:33     INFO -  command: hg path default
21:42:33     INFO -  command: cwd: C:/builds/hg-shared\try
21:42:33     INFO -  command: env: {'HGPLAIN': '1'}
21:42:34     INFO -  command: END (0.25s elapsed)
21:42:34     INFO -  command: output:
21:42:34     INFO -  https://hg.mozilla.org/mozilla-central
21:42:34     INFO -  hg path isn't correct (https://hg.mozilla.org/mozilla-central should be https://hg.mozilla.org/try); clobbering
21:42:34     INFO -  Using _rmtree_windows ...
22:02:15     INFO -  Using _rmtree_windows ...
22:02:15     INFO -  Updating shared repo
22:02:15     INFO -  Reporting hg version in use

16 minutes spent on try.hg unbundling:

22:02:16     INFO -  Trying to use bundle https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/try.hg
22:02:16     INFO -  command: START
22:02:16     INFO -  command: hg unbundle --traceback https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/try.hg
22:02:16     INFO -  command: cwd: C:\builds\hg-shared\try
22:02:16     INFO -  command: env: {'HGPLAIN': '1'}
22:18:00     INFO -  command: END (944.25s elapsed)
22:18:00     INFO -  command: output:
22:18:00     INFO -  adding changesets
22:18:00     INFO -  adding manifests
22:18:00     INFO -  adding file changes
22:18:00     INFO -  added 276500 changesets with 1572684 changes to 229589 files
22:18:00     INFO -  (run 'hg update' to get a working copy)

2 minutes spent pulling:

22:18:00     INFO -  command: START
22:18:00     INFO -  command: hg pull --traceback -r 53d3bd0b7df88bce264610453e312c3b396978b5 https://hg.mozilla.org/try
22:18:00     INFO -  command: cwd: C:\builds\hg-shared\try
22:18:00     INFO -  command: env: {'HGPLAIN': '1'}
22:20:10     INFO -  command: END (130.00s elapsed)
22:20:10     INFO -  command: output:
22:20:10     INFO -  pulling from https://hg.mozilla.org/try
22:20:10     INFO -  searching for changes
22:20:10     INFO -  adding changesets
22:20:10     INFO -  adding manifests
22:20:10     INFO -  adding file changes
22:20:10     INFO -  added 6116 changesets with 41815 changes to 19850 files
22:20:10     INFO -  (run 'hg update' to get a working copy)

6 minutes spent checking out the revision:

22:20:10     INFO -  command: START
22:20:10     INFO -  command: hg update -C -r 53d3bd0b7df88bce264610453e312c3b396978b5
22:20:10     INFO -  command: cwd: c:\builds\moz2_slave\try-w32-0000000000000000000000\build\src
22:20:10     INFO -  command: output:
22:26:21     INFO -  135840 files updated, 0 files merged, 0 files removed, 0 files unresolved
22:26:21     INFO -  command: END (371.46s elapsed)


Presumably, all of the above is linked to that mozilla-central != try thing.

http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-53d3bd0b7df88bce264610453e312c3b396978b5/try-win32/try-win32-bm78-try1-build2527.txt.gz
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-53d3bd0b7df88bce264610453e312c3b396978b5/try-win32-debug/try-win32-debug-bm78-try1-build2447.txt.gz
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-53d3bd0b7df88bce264610453e312c3b396978b5/try-win64/try-win64-bm79-try1-build2498.txt.gz

Comment 1

[Mike Hommey [:glandium]]

This looks like to be the last build before mine on y-2008-spot-144:
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-57d7278a3691efbc3eaf8c589628d28dba257d93/try-win32-debug/try-win32-debug-bm78-try1-build2437.txt.gz

and it says:

17:24:42     INFO -  command: START
17:24:42     INFO -  command: hg path default
17:24:42     INFO -  command: cwd: c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src
17:24:42     INFO -  command: env: {'HGPLAIN': '1'}
17:24:42     INFO -  command: END (0.25s elapsed)
17:24:42     INFO -  command: output:
17:24:42     INFO -  https://hg.mozilla.org/try
17:24:42     INFO -  command: START
17:24:42     INFO -  command: hg path default
17:24:42     INFO -  command: cwd: C:/builds/hg-shared\try
17:24:42     INFO -  command: env: {'HGPLAIN': '1'}
17:24:42     INFO -  command: END (0.25s elapsed)
17:24:42     INFO -  command: output:
17:24:42     INFO -  https://hg.mozilla.org/try

So presumably, at the end of that build, `hg path default` should still be https://hg.mozilla.org/try, but it's not when my build started:

21:42:33     INFO -  command: START
21:42:33     INFO -  command: hg path default
21:42:33     INFO -  command: cwd: C:/builds/hg-shared\try
21:42:33     INFO -  command: env: {'HGPLAIN': '1'}
21:42:34     INFO -  command: END (0.25s elapsed)
21:42:34     INFO -  command: output:
21:42:34     INFO -  https://hg.mozilla.org/mozilla-central

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: https://github.com/mozilla/build-cloud-tools/pull/173

This PR to cloud tools updates the hg cache creation mechanism to correct the default path (in hgrc) for the try repository.

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

This is now patched in userdata (https://github.com/mozilla/build-cloud-tools/commit/e606df73088abcd7de3cd8b187fbdad08de5c8e9). Change should be quick to propagate as it's checked/applied on each reboot.
Please reopen, if issue persists.

Comment 4

[Mike Hommey [:glandium]]

Should it still be trying to unbundle try.hg? I have a couple win64 builds from 2 days ago that did that:
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-edb8d849e07880f37b9e56456106036ed6e1e46b/try-win64/try-win64-bm79-try1-build2785.txt.gz

Comment 5

[Mike Hommey [:glandium]]

The log doesn't have the "path isn't correct" thing, so I guess that's a separate bug, actually related to bug 1247169.