Intermittent [taskcluster:error] [mounts] reading file in zip archive: file already exists: Z:\task_1549919043\mozharness\LICENSE
Categories
(Release Engineering :: Applications: MozharnessCore, defect, P5)
Tracking
(Not tracked)
People
(Reporter: intermittent-bug-filer, Unassigned)
References
Details
(Keywords: intermittent-failure, regression)
#[markdown(off)]
Filed by: btara [at] mozilla.com
https://treeherder.mozilla.org/logviewer.html#?job_id=227756867&repo=mozilla-esr60
[taskcluster 2019-02-11T22:05:08.947Z] "machine-setup": {
[taskcluster 2019-02-11T22:05:08.947Z] "ami-created": "2019-02-11 11:12:51.812Z",
[taskcluster 2019-02-11T22:05:08.947Z] "manifest": "https://github.com/mozilla-releng/OpenCloudConfig/blob/56d247d42d33af2d3fc9b69dfcb13a30e980e6b5/userdata/Manifest/gecko-t-win10-64.json"
[taskcluster 2019-02-11T22:05:08.947Z] }
[taskcluster 2019-02-11T22:05:08.947Z] }
[taskcluster 2019-02-11T22:05:08.947Z] Task ID: fEEm1O-1RQqVflcznTadxw
[taskcluster 2019-02-11T22:05:08.947Z] === Task Starting ===
[taskcluster 2019-02-11T22:05:09.939Z] Uploading redirect artifact public/logs/live.log to URL https://gq5gfhqaaaawrxxx2e5ataxcqh3ngetqfwm2nykekwn344bt.taskcluster-worker.net:60023/log/P5TB1GcNRYmZidBB6RxACQ with mime type "text/plain; charset=utf-8" and expiry 2019-02-11T23:50:09.466Z
[taskcluster 2019-02-11T22:05:10.299Z] [mounts] Downloading task UFo0pfl9RNmMdxJY2iFUPw artifact public/build/mozharness.zip to Y:\downloads\MiqMmexmR62MBTTcZRr5Sw
[taskcluster 2019-02-11T22:05:28.091Z] [mounts] Downloaded 2323166 bytes with SHA256 b41be6c27d3b186e3fa2c5a98f8d5adb0467eefb6eaecba2427c73de6ad0e435 from task UFo0pfl9RNmMdxJY2iFUPw artifact public/build/mozharness.zip to Y:\downloads\MiqMmexmR62MBTTcZRr5Sw
[taskcluster:warn 2019-02-11T22:05:28.091Z] [mounts] Download Y:\downloads\MiqMmexmR62MBTTcZRr5Sw of task UFo0pfl9RNmMdxJY2iFUPw artifact public/build/mozharness.zip has SHA256 b41be6c27d3b186e3fa2c5a98f8d5adb0467eefb6eaecba2427c73de6ad0e435 but task payload does not declare a required value, so content authenticity cannot be verified
[taskcluster 2019-02-11T22:05:28.091Z] [mounts] Creating directory Z:\task_1549919043 with permissions 0700
[taskcluster 2019-02-11T22:05:28.092Z] [mounts] Extracting zip file Y:\downloads\MiqMmexmR62MBTTcZRr5Sw to 'Z:\task_1549919043'
[taskcluster 2019-02-11T22:05:28.656Z] Uploading redirect artifact public/logs/live.log to URL https://queue.taskcluster.net/v1/task/fEEm1O-1RQqVflcznTadxw/runs/0/artifacts/public/logs/live_backing.log with mime type "text/plain; charset=utf-8" and expiry 2020-02-11T20:55:03.410Z
[taskcluster:error] [mounts] reading file in zip archive: file already exists: Z:\task_1549919043\mozharness\LICENSE
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 2•6 years ago
|
||
:grenade, not sure if you know much about the taskcluster work or how it works on the windows aarch64 laptops, but we get these failures quite often. Do you have ideas on why would would get this?
Also what are the Y and Z drives? Are these drives on AWS?
|
||
Comment 3•6 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #2)
:grenade, not sure if you know much about the taskcluster work or how it works on the windows aarch64 laptops, but we get these failures quite often. Do you have ideas on why would would get this?
i'm not sure.
Also what are the Y and Z drives? Are these drives on AWS?
yes. the log in question appears to be from an ec2 worker type (gecko-t-win10-64). it's worth noting that the Y:\downloads location changed recently to Z:\downloads (see bug 1528198). maybe the problem above is linked to the cache being on Y?
|
|
||
Comment 4•6 years ago
|
||
we see this problem very often on the windows/aarch64 laptops at bitbar
|
|
||
Comment 5•6 years ago
|
||
for reference the aarch64 is mentioned in bug 1531878 with a long outstanding needinfo.
is there a way via taskcluster or some other machine management to see what tasks were sent to a specific machine? Possibly then we could see if we are scheduling 2 tasks at once?
Also, do we know if these machines are rebooting between jobs? my understanding is a task has a unique directory, so we would only have this file exists if that .zip file was already unzipped in the same task directory? This is why I wonder if we are running 2 tasks at once on a given machine.
:coop can you help out here as well?
|
||
Comment 6•6 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #5)
:coop can you help out here as well?
You've pinged Pete. I'll do the same, but generally that should be enough.
Also, it would be helpful to know how frequently this is occurring. Once a week is not going to bubble up to the top of our priority list any time soon. If it's happening on every push, we'll jump in immediately for tier 1 jobs.
This bug is also marked as P5 normal, and bug 1531878 doesn't have priority or status set at all. Please leverage those states if you are truly blocked by this, otherwise it's impossible for me to know your priorities and reliably get people to help you while my team is short-staffed.
|
||
Updated•6 years ago
|
| Comment hidden (Intermittent Failures Robot) |
Description
•