Open Bug 1604486 Opened 5 years ago Updated 3 years ago

Intermittent [taskcluster:error] Uploading error artifact public/build from file artifacts/ with message "Could not read directory '/Users/task_1576583400/artifacts'", after mercurial.error.SignalInterrupt

Categories

(Developer Services :: Mercurial: hg.mozilla.org, defect, P5)

Product:
Developer Services
Component:
Mercurial: hg.mozilla.org
Type:
defect

Tracking

(Not tracked)

People

(Reporter: intermittent-bug-filer, Unassigned, NeedInfo)

References

Details

(Keywords: intermittent-failure, leave-open, perf-alert, Whiteboard: [stockwell infra])

Attachments

(2 files)

Bug 1604486 - Use sparse checkout for the macOS profile run. r?sheehan
47 bytes, text/x-phabricator-request
Details | Review
Bug 1604486 - Update profile generate sparse profile to include talos tests.
47 bytes, text/x-phabricator-request
Details | Review

Filed by: malexandru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=281534568&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YPjhDkXGSJ2Q5UORpYXlZg/runs/0/artifacts/public/logs/live_backing.log

[vcs 2019-12-17T12:06:13.416Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 243, in _posixworker
[vcs 2019-12-17T12:06:13.416Z] res = util.pickle.load(key.fileobj)
[vcs 2019-12-17T12:06:13.416Z] _pickle.UnpicklingError: pickle data was truncated
[vcs 2019-12-17T12:06:13.417Z] ** unknown exception encountered, please report by visiting
[vcs 2019-12-17T12:06:13.417Z] ** https://mercurial-scm.org/wiki/BugTracker
[vcs 2019-12-17T12:06:13.417Z] ** Python 3.7.5 (default, Nov 1 2019, 02:16:32) [Clang 11.0.0 (clang-1100.0.33.8)]
[vcs 2019-12-17T12:06:13.417Z] ** Mercurial Distributed SCM (version 5.2)
[vcs 2019-12-17T12:06:13.417Z] ** Extensions loaded: share, rebase, strip, mq, purge, robustcheckout, sparse
[vcs 2019-12-17T12:06:13.417Z] Traceback (most recent call last):
[vcs 2019-12-17T12:06:13.417Z] File "/usr/local/bin/hg", line 36, in <module>
[vcs 2019-12-17T12:06:13.419Z] dispatch.run()
[vcs 2019-12-17T12:06:13.419Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 111, in run
[vcs 2019-12-17T12:06:13.419Z] status = dispatch(req)
[vcs 2019-12-17T12:06:13.419Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 250, in dispatch
[vcs 2019-12-17T12:06:13.419Z] ret = _runcatch(req) or 0
[vcs 2019-12-17T12:06:13.419Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 424, in _runcatch
[vcs 2019-12-17T12:06:13.420Z] return _callcatch(ui, _runcatchfunc)
[vcs 2019-12-17T12:06:13.420Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 433, in _callcatch
[vcs 2019-12-17T12:06:13.420Z] return scmutil.callcatch(ui, func)
[vcs 2019-12-17T12:06:13.420Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/scmutil.py", line 177, in callcatch
[vcs 2019-12-17T12:06:13.420Z] return func()
[vcs 2019-12-17T12:06:13.420Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 414, in _runcatchfunc
[vcs 2019-12-17T12:06:13.420Z] return _dispatch(req)
[vcs 2019-12-17T12:06:13.420Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1174, in _dispatch
[vcs 2019-12-17T12:06:13.421Z] lui, repo, cmd, fullargs, ui, options, d, cmdpats, cmdoptions
[vcs 2019-12-17T12:06:13.421Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 862, in runcommand
[vcs 2019-12-17T12:06:13.421Z] ret = _runcommand(ui, options, cmd, d)
[vcs 2019-12-17T12:06:13.421Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1185, in _runcommand
[vcs 2019-12-17T12:06:13.421Z] return cmdfunc()
[vcs 2019-12-17T12:06:13.421Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1171, in <lambda>
[vcs 2019-12-17T12:06:13.422Z] d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
[vcs 2019-12-17T12:06:13.422Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/util.py", line 1843, in check
[vcs 2019-12-17T12:06:13.422Z] return func(*args, **kwargs)
[vcs 2019-12-17T12:06:13.422Z] File "/usr/local/lib/hgext/robustcheckout.py", line 261, in robustcheckout
[vcs 2019-12-17T12:06:13.422Z] sparse_profile=sparseprofile)
[vcs 2019-12-17T12:06:13.422Z] File "/usr/local/lib/hgext/robustcheckout.py", line 777, in _docheckout
[vcs 2019-12-17T12:06:13.423Z] if commands.update(ui, repo, rev=checkoutrevision, clean=True):
[vcs 2019-12-17T12:06:13.423Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/commands.py", line 7694, in update
[vcs 2019-12-17T12:06:13.425Z] ui, repo, rev, brev, clean=clean, updatecheck=updatecheck
[vcs 2019-12-17T12:06:13.425Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/hg.py", line 1106, in updatetotally
[vcs 2019-12-17T12:06:13.425Z] ret = _clean(repo, checkout)
[vcs 2019-12-17T12:06:13.425Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/hg.py", line 1043, in clean
[vcs 2019-12-17T12:06:13.426Z] stats = updaterepo(repo, node, True)
[vcs 2019-12-17T12:06:13.426Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/hg.py", line 1024, in updaterepo
[vcs 2019-12-17T12:06:13.426Z] updatecheck=updatecheck,
[vcs 2019-12-17T12:06:13.426Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/merge.py", line 2558, in update
[vcs 2019-12-17T12:06:13.427Z] repo, actions, wc, p2, overwrite, wantfiledata, labels=labels
[vcs 2019-12-17T12:06:13.427Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/merge.py", line 1927, in applyupdates
[vcs 2019-12-17T12:06:13.427Z] for final, res in prog:
[vcs 2019-12-17T12:06:13.427Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 243, in _posixworker
[vcs 2019-12-17T12:06:13.427Z] res = util.pickle.load(key.fileobj)
[vcs 2019-12-17T12:06:13.427Z] _pickle.UnpicklingError: pickle data was truncated
[taskcluster 2019-12-17T12:06:13.613Z] Exit Code: 1
[taskcluster 2019-12-17T12:06:13.613Z] User Time: 4m25.074255s
[taskcluster 2019-12-17T12:06:13.613Z] Kernel Time: 2m43.841782s
[taskcluster 2019-12-17T12:06:13.613Z] Wall Time: 2m31.335821108s
[taskcluster 2019-12-17T12:06:13.613Z] Result: FAILED
[taskcluster 2019-12-17T12:06:13.613Z] === Task Finished ===
[taskcluster 2019-12-17T12:06:13.613Z] Task Duration: 2m31.340631289s
[taskcluster:error] Uploading error artifact public/build from file artifacts/ with message "Could not read directory '/Users/task_1576583400/artifacts'", reason "file-missing-on-worker" and expiry 2020-12-16T11:17:38.610Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read directory '/Users/task_1576583400/artifacts'
[taskcluster 2019-12-17T12:06:13.841Z] [mounts] Preserving cache: Moving "/Users/task_1576583400/checkouts" to "/var/local/generic-worker/caches/H8eFo-Y3RFCwfLo1o1h9qg"
[taskcluster 2019-12-17T12:06:13.843Z] [mounts] Denying task_1576583400 access to '/var/local/generic-worker/caches/H8eFo-Y3RFCwfLo1o1h9qg'
[taskcluster:error] exit status 1
[taskcluster:error] file-missing-on-worker: Could not read directory '/Users/task_1576583400/artifacts'

[vcs 2019-12-17T12:06:11.942Z] updating [================================================> ] 281700/285287 02s
[vcs 2019-12-17T12:06:13.317Z] updating [================================================> ] 284400/285287 01sTraceback (most recent call last):
[vcs 2019-12-17T12:06:13.317Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/scmutil.py", line 177, in callcatch
[vcs 2019-12-17T12:06:13.317Z] return func()
[vcs 2019-12-17T12:06:13.317Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 207, in workerfunc
[vcs 2019-12-17T12:06:13.317Z] os.write(wfd, util.pickle.dumps(result))
[vcs 2019-12-17T12:06:13.317Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 303, in catchterm
[vcs 2019-12-17T12:06:13.317Z] raise error.SignalInterrupt
[vcs 2019-12-17T12:06:13.317Z] mercurial.error.SignalInterrupt

Summary: Intermittent [taskcluster:error] Uploading error artifact public/build from file artifacts/ with message "Could not read directory '/Users/task_1576583400/artifacts'", reason "file-missing-on-worker" and expiry 2020-12-16T11:17:38.610Z → Intermittent [taskcluster:error] Uploading error artifact public/build from file artifacts/ with message "Could not read directory '/Users/task_1576583400/artifacts'", after mercurial.error.SignalInterrupt

Hmm, this only seems to be happening on pgo run tasks. Connor, does this stack look familiar? Did we do something wrong configuring these workers?

Flags: needinfo?(sheehan)

This doesn't look familiar to me. From the source code it looks like Mercurial uses signal.signal to turn SIGTERM and SIGHUP (possibly more, didn't dig much deeper) into that exception. Hope that helps.

Flags: needinfo?(sheehan)

There are 37 total failures in the last 5 days, all on macosx64-shippable opt and macosx64 pgo

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=282008592&repo=autoland&lineNumber=334

Connor can you assign someone to take a look?

Flags: needinfo?(sheehan)
Whiteboard: [stockwell needswork:owner]

There are 28 total failures in the last 7 days on macosx64-shippable opt.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=283371878&repo=autoland&lineNumber=292

Connor have you had a chance to look?

Whiteboard: [stockwell unknown] → [stockwell needswork:owner]

This has reached the disable-recommended queue. Connor is this something to worry about?

Flags: needinfo?(sheehan)
Flags: needinfo?(sheehan)

(In reply to Andreea Pavel [:apavel] from comment #12)

This has reached the disable-recommended queue. Connor is this something to worry about?

I'm not sure to be honest. This is only happening on macos64-shippable/opt/generate-profile-macosx64-shippable, which leads me to believe it's a task configuration issue.

However these tasks are also some of the first to use Python 3 Mercurial for checkouts in automation. It's possible there's a Python 3 bug in robustcheckout or core Mercurial that we need to address. Until we start seeing failures on other platforms I don't think it's worth investigating as a Py3 problem. I don't have any bright ideas for possible causes at the moment.

Flags: needinfo?(sheehan)

Thank you!

Joel does this qualify for stockwell infra?

Flags: needinfo?(jmaher)

this is an infra issue, not related to product or tests cases specifically. Given the frequency of this we should be making this a higher priority.

Flags: needinfo?(jmaher)

I'm going to attempt to enable sparse checkouts on these workers, which will either make this less frequent/not happen or at least give us more information about where the issue is.

Currently this needs the pr at https://github.com/mozilla-platform-ops/ronin_puppet/pull/169 to be merged.

Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]

Chris, based on Joel's comment, should the priority of this bug be changed from P5? Is that something you can do?

Flags: needinfo?(cmanchester)
Whiteboard: [stockwell needswork:owner] → [stockwell infra]

I'm not sure of an appropriate priority for this bug, but I intend to follow through on the idea in comment 17, which Connor and I agree should help the situation.

Assignee: nobody → cmanchester
Flags: needinfo?(cmanchester)
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/17e35065ec03
Use sparse checkout for the macOS profile run. r=sheehan

This seems to have morphed into something that looks more like bug 1304942. Connor, does the stack from this failure look familiar: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=285117960&repo=autoland&lineNumber=155 ?

Flags: needinfo?(sheehan)

That seems to have been a temporary outage of some sort. Clearing ni? for now.

Flags: needinfo?(sheehan)

https://hg.mozilla.org/mozilla-central/rev/17e35065ec03

(In reply to Pulsebot from comment #26)

Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/17e35065ec03
Use sparse checkout for the macOS profile run. r=sheehan

Chris, makes sense that this patch caused the below regression? There were 2 suspect patches, while the other one was backed out - without fixing the regression.
== Change summary for alert #24703 (as of Thu, 16 Jan 2020 15:48:53 GMT) ==

Regressions:

14% perf_reftest_singletons external-string-pass.html macosx1014-64-shippable opt e10s stylo 887.66 -> 1,012.21
6% perf_reftest_singletons bidi-resolution-1.html macosx1014-64-shippable opt e10s stylo 160.76 -> 170.36
6% perf_reftest_singletons parent-basic-singleton.html macosx1014-64-shippable opt e10s stylo 168.08 -> 177.47

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24703

Flags: needinfo?(cmanchester)

Unfortunately this does seem to be the fault of this patch, I will post a fix.

Flags: needinfo?(cmanchester)
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/feea4bfad848
Update profile generate sparse profile to include talos tests. r=sheehan

https://hg.mozilla.org/mozilla-central/rev/feea4bfad848

Hey Chris,

There is a single regression from the changeset above, is that expected ?
Thanks!

== Change summary for alert #24738 (as of Wed, 22 Jan 2020 07:16:20 GMT) ==

Regressions:

9% perf_reftest_singletons bloom-basic.html windows7-32-shippable opt e10s stylo 44.77 -> 48.74

Improvements:

18% perf_reftest_singletons external-string-pass.html windows7-32-shippable opt e10s stylo 1,105.06 -> 903.19
18% perf_reftest_singletons external-string-pass.html windows10-64-shippable opt e10s stylo 1,008.88 -> 830.73
17% perf_reftest_singletons external-string-pass.html windows10-64-shippable-qr opt e10s stylo 1,009.56 -> 836.34
7% perf_reftest_singletons tiny-traversal-singleton.html windows7-32-shippable opt e10s stylo 786.12 -> 733.19
7% perf_reftest_singletons scrollbar-styles-1.html windows7-32-shippable opt e10s stylo 595.62 -> 556.04
6% perf_reftest_singletons tiny-traversal-singleton.html windows10-64-shippable-qr opt e10s stylo 832.34 -> 778.62
6% perf_reftest_singletons id-getter-2.html windows10-64-shippable-qr opt e10s stylo 918.48 -> 864.88
6% perf_reftest_singletons scrollbar-styles-1.html windows10-64-shippable opt e10s stylo 603.84 -> 570.65
5% perf_reftest_singletons scrollbar-styles-1.html windows10-64-shippable-qr opt e10s stylo 609.87 -> 578.58
5% perf_reftest_singletons bidi-resolution-1.html windows7-32-shippable opt e10s stylo 151.84 -> 144.10

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24738

Flags: needinfo?(cmanchester)

This does seem to be related to this change. I don't know these benchmarks well enough to know whether to be concerned about this, but it seems fine given the other improvements here. :dmajor, do you have any input here?

Flags: needinfo?(cmanchester) → needinfo?(dmajor)

The perf singleton tests really seem to have a mind of their own. I had hoped that getting them into PGO would stabilize the numbers but they still jump around a lot.

Flags: needinfo?(dmajor)

I've also been running into this exception building Nightly for Arch Linux, when our build system does a hg clone across filesystems. It only started happening after Mercurial was moved to Python 3. So this seems very much like a Mercurial core bug.

This should fix the problem: https://phab.mercurial-scm.org/D8051

Thanks for debugging this, :heftig!!

:glob and I were just looking at this yesterday, but we were having a hard time reproducing it.

== Change summary for alert #24832 (as of Sun, 02 Feb 2020 22:06:35 GMT) ==

Improvements:

5% perf_reftest_singletons bidi-resolution-1.html macosx1014-64-shippable opt e10s stylo 169.55 -> 160.95

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24832

The mitigation landed and was partly effective, but with the actual fix upstream I'm un-assigning myself.

Assignee: cmanchester → nobody
Keywords: perf-alert

There is a fix for this bug that has landed in upstream Mercurial 5.3. Upgrading the docker image to use the latest hg should fix this problem.

There are 23 failures associated to this bug in the last 7 days. These are occurring on macosx1014-64-shippable opt.

recent log: https://treeherder.mozilla.org/logviewer.html#?job_id=292692665&repo=autoland
[vcs 2020-03-11T15:31:28.548Z] Traceback (most recent call last):
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 210, in _posixworker
[vcs 2020-03-11T15:31:28.548Z] ret = scmutil.callcatch(ui, workerfunc)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/scmutil.py", line 177, in callcatch
[vcs 2020-03-11T15:31:28.548Z] return func()
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 206, in workerfunc
[vcs 2020-03-11T15:31:28.548Z] for result in func(*(staticargs + (pargs,))):
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/merge.py", line 1724, in batchget
[vcs 2020-03-11T15:31:28.548Z] fctx(f).data(),
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/context.py", line 1248, in data
[vcs 2020-03-11T15:31:28.548Z] return self._filelog.read(self._filenode)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/filelog.py", line 169, in read
[vcs 2020-03-11T15:31:28.548Z] return storageutil.filtermetadata(self.revision(node))
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/filelog.py", line 93, in revision
[vcs 2020-03-11T15:31:28.548Z] return self._revlog.revision(node, _df=_df, raw=raw)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/revlog.py", line 1715, in revision
[vcs 2020-03-11T15:31:28.548Z] return self._revisiondata(nodeorrev, _df, raw=raw)[0]
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/revlog.py", line 1743, in _revisiondata
[vcs 2020-03-11T15:31:28.548Z] rev, rawtext, validated = self._rawtext(node, rev, _df=_df)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/revlog.py", line 1812, in _rawtext
[vcs 2020-03-11T15:31:28.548Z] bins = self._chunks(chain, df=_df, targetsize=targetsize)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/revlog.py", line 1631, in _chunks
[vcs 2020-03-11T15:31:28.548Z] ladd(decomp(buffer(data, chunkstart - offset, chunklength)))
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/revlog.py", line 2095, in decompress
[vcs 2020-03-11T15:31:28.548Z] return _zlibdecompress(data)
[vcs 2020-03-11T15:31:28.548Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 303, in catchterm
[vcs 2020-03-11T15:31:28.548Z] raise error.SignalInterrupt
[vcs 2020-03-11T15:31:28.548Z] mercurial.error.SignalInterrupt
...............
[vcs 2020-03-11T15:31:28.569Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/merge.py", line 1927, in applyupdates
[vcs 2020-03-11T15:31:28.570Z] for final, res in prog:
[vcs 2020-03-11T15:31:28.570Z] File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/worker.py", line 243, in _posixworker
[vcs 2020-03-11T15:31:28.570Z] res = util.pickle.load(key.fileobj)
[vcs 2020-03-11T15:31:28.570Z] _pickle.UnpicklingError: pickle data was truncated
[taskcluster 2020-03-11T15:31:28.649Z] Exit Code: 1
[taskcluster 2020-03-11T15:31:28.649Z] User Time: 13.267878s
[taskcluster 2020-03-11T15:31:28.649Z] Kernel Time: 5.528787s
[taskcluster 2020-03-11T15:31:28.649Z] Wall Time: 14.12946886s
[taskcluster 2020-03-11T15:31:28.649Z] Result: FAILED
[taskcluster 2020-03-11T15:31:28.649Z] === Task Finished ===
[taskcluster 2020-03-11T15:31:28.649Z] Task Duration: 14.134413269s
[taskcluster:error] Uploading error artifact public/build from file artifacts/ with message "Could not read directory '/Users/task_1583938311/artifacts'", reason "file-missing-on-worker" and expiry 2021-03-11T14:45:45.761Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read directory '/Users/task_1583938311/artifacts'
[taskcluster 2020-03-11T15:31:28.871Z] [mounts] Preserving cache: Moving "/Users/task_1583938311/checkouts" to "/var/local/generic-worker/caches/T0rSF8cZR8WKXVOU7pqBQg"
[taskcluster 2020-03-11T15:31:28.871Z] [mounts] Denying task_1583938311 access to '/var/local/generic-worker/caches/T0rSF8cZR8WKXVOU7pqBQg'
[taskcluster:error] exit status 1
[taskcluster:error] file-missing-on-worker: Could not read directory '/Users/task_1583938311/artifacts'

The UnpicklingError error has been fixed in Mercurial 5.3 however the log in comment 53 indicates it's still using 5.2.1.
Who is responsible for updating the docker image to use the latest hg version?

Dave, this bug can be fixed at any time by upgrading the Mercurial version from 5.2 to 5.3+ (5.3.1 is the latest version ATM). Are you able to upgrade this? Is a patch to ronin_puppet the correct place to do such an upgrade?

Flags: needinfo?(dhouse)

Connor, thanks! That is the right repo. I need to do some other updates on the pgo branch in the coming month. So I'll do this also.

Flags: needinfo?(dhouse)
Depends on: 1626357
Flags: needinfo?(dhouse)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: