1501519 - hg robustcheckout failed assertion util.versiontuple(n=2) in ((4, 3), (4, 4), (4, 5))

Status: RESOLVED FIXED

(Developer Services :: Mercurial: robustcheckout, defect)

Description

[Jordan Lund (:jlund)]

https://taskcluster-artifacts.net/UpSVfWizQsCldgTfs_w-jQ/0/public/logs/live_backing.log

[taskcluster 2018-10-18T21:09:35.164Z] Executing command 0: "c:\Program Files\Mercurial\hg.exe" robustcheckout --sharebase y:\hg-shared --purge --upstream https://hg.mozilla.org/mozilla-unified --revision 84219fbf133cacfc6e31c9471ad20ee7162a02af https://hg.mozilla.org/releases/mozilla-release .\build\src

Z:\task_1539894279>"c:\Program Files\Mercurial\hg.exe" robustcheckout --sharebase y:\hg-shared --purge --upstream https://hg.mozilla.org/mozilla-unified --revision 84219fbf133cacfc6e31c9471ad20ee7162a02af https://hg.mozilla.org/releases/mozilla-release .\build\src 
(using Mercurial 4.7.1)
ensuring https://hg.mozilla.org/releases/mozilla-release@84219fbf133cacfc6e31c9471ad20ee7162a02af is available at .\build\src
(existing repository shared store: y:\hg-shared\8ba995b74e18334ab3707f27e9eb8f4e37ba3d29\.hg)
(purging working directory)
PERFHERDER_DATA: {"framework": {"name": "vcs"}, "suites": [{"extraOptions": ["c5.4xlarge"], "lowerIsBetter": true, "name": "overall", "shouldAlert": false, "subtests": [], "value": 1.2590000629425049}]}
Traceback (most recent call last):
  File "mercurial\scmutil.pyc", line 161, in callcatch
  File "mercurial\dispatch.pyc", line 344, in _runcatchfunc
  File "mercurial\dispatch.pyc", line 984, in _dispatch
  File "mercurial\dispatch.pyc", line 730, in runcommand
  File "mercurial\dispatch.pyc", line 992, in _runcommand
  File "mercurial\dispatch.pyc", line 981, in <lambda>
  File "mercurial\util.pyc", line 1528, in check
  File "C:/mozilla-build/robustcheckout.py", line 265, in robustcheckout
  File "C:/mozilla-build/robustcheckout.py", line 638, in _docheckout
AssertionError
** Unknown exception encountered with possibly-broken third-party extension robustcheckout
** which supports versions 4.5 of Mercurial.
** Please disable robustcheckout and try your action again.
** If that fixes the bug please report it to the extension author.
** Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:53:40) [MSC v.1500 64 bit (AMD64)]
** Mercurial Distributed SCM (version 4.7.1)
** Extensions loaded: strip, mq, purge, rebase, share, robustcheckout
Traceback (most recent call last):
  File "hg", line 41, in <module>
  File "mercurial\dispatch.pyc", line 90, in run
  File "mercurial\dispatch.pyc", line 213, in dispatch
  File "mercurial\dispatch.pyc", line 354, in _runcatch
  File "mercurial\dispatch.pyc", line 362, in _callcatch
  File "mercurial\scmutil.pyc", line 161, in callcatch
  File "mercurial\dispatch.pyc", line 344, in _runcatchfunc
  File "mercurial\dispatch.pyc", line 984, in _dispatch
  File "mercurial\dispatch.pyc", line 730, in runcommand
  File "mercurial\dispatch.pyc", line 992, in _runcommand
  File "mercurial\dispatch.pyc", line 981, in <lambda>
  File "mercurial\util.pyc", line 1528, in check
  File "C:/mozilla-build/robustcheckout.py", line 265, in robustcheckout
    sparse_profile=sparseprofile)
  File "C:/mozilla-build/robustcheckout.py", line 638, in _docheckout
    assert util.versiontuple(n=2) in ((4, 3), (4, 4), (4, 5))
AssertionError
[taskcluster 2018-10-18T21:09:36.560Z]    Exit Code: 255
[taskcluster 2018-10-18T21:09:36.560Z]    User Time: 0s
[taskcluster 2018-10-18T21:09:36.560Z]  Kernel Time: 0s
[taskcluster 2018-10-18T21:09:36.560Z]    Wall Time: 1.3964153s
[taskcluster 2018-10-18T21:09:36.560Z]       Result: FAILED
[taskcluster 2018-10-18T21:09:36.560Z] === Task Finished ===
[taskcluster 2018-10-18T21:09:36.560Z] Task Duration: 1.3964153s
[taskcluster:error] Uploading error artifact public/build/hr/target.complete.mar from file build/outputs/hr/target.complete.mar with message "Could not read file 'Z:\\task_1539894279\\build\\outputs\\hr\\target.complete.mar'", reason "file-missing-on-worker" and expiry 2019-10-18T19:55:07.798Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read file 'Z:\task_1539894279\build\outputs\hr\target.complete.mar'
[taskcluster:error] Uploading error artifact public/build/hr/target.installer.exe from file build/outputs/hr/target.installer.exe with message "Could not read file 'Z:\\task_1539894279\\build\\outputs\\hr\\target.installer.exe'", reason "file-missing-on-worker" and expiry 2019-10-18T19:55:07.798Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read file 'Z:\task_1539894279\build\outputs\hr\target.installer.exe'
[taskcluster:error] Uploading error artifact public/build/hr/target.stub-installer.exe from file build/outputs/hr/target.stub-installer.exe with message "Could not read file 'Z:\\task_1539894279\\build\\outputs\\hr\\target.stub-installer.exe'", reason "file-missing-on-worker" and expiry 2019-10-18T19:55:07.798Z


rerun worked.

Comment 1

[Jordan Lund (:jlund)]

@gps - seen this before?

Comment 2

[Connor Sheehan [:sheehan]]

That failure indicates that the copy of robustcheckout used is only compatible with Mercurial up to version 4.5. Since the re-run worked, I'd say the worker that executed the first run has an old cached version of robustcheckout on it, and the worker from the second run has an up to date version. Probably need to invalidate the cache on that worker.

Comment 3

[Jordan Lund (:jlund)]

(In reply to Connor Sheehan [:sheehan] from comment #2)
> That failure indicates that the copy of robustcheckout used is only
> compatible with Mercurial up to version 4.5. Since the re-run worked, I'd
> say the worker that executed the first run has an old cached version of
> robustcheckout on it, and the worker from the second run has an up to date
> version. Probably need to invalidate the cache on that worker.

@markco - any thoughts?

Comment 4

[Mark Cornmesser [:markco] OOO 2024/04/15]

We pull robustcheckout.py from: https://github.com/mozilla-releng/OpenCloudConfig/blob/master/userdata/Configuration/FirefoxBuildResources/robustcheckout.py

We could easily update that, or pull directly from https://github.com/mozilla/version-control-tools/blob/master/hgext/robustcheckout/__init__.py. However, where is the second version coming from? In tree? If that is the case we may want to pull out of the configuration setup and rely on what is in tree. 

NI'ing Rob for his input as well.

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

we should update the occ version of robustcheckout. we used to pull it directly from hg, but it was suggested in bug 1305485, comment 5, that it would be better to vendor the specific version we want.

Comment 6

[Connor Sheehan [:sheehan]]

Attachment: GitHub Pull Request

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 9020013 [details] [review]
GitHub Pull Request

Review of attachment 9020013 [details] [review]:
-----------------------------------------------------------------

merged: https://github.com/mozilla-releng/OpenCloudConfig/commit/c7808b58f5549df8002ec978041f2cdabb8bda94

Comment 8

[Connor Sheehan [:sheehan]]

This was deployed yesterday.

Comment 9

[Jordan Lund (:jlund)]

\o/ thanks all