Stop script execution if bad zip file for download_unpack()

RESOLVED INVALID

Status

RESOLVED INVALID
2 years ago
2 years ago

People

(Reporter: armenzg, Assigned: armenzg)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

2 years ago
On bug 1300812 I made download_unpack() reliable against network issues.

Unfortunately, there's a case when the fetch data is correct, however, unzipping turns out to be a bad zip file.

I logged this in the code, however, it should be fatal() logging (the script stops) rather than exception() (We get a traceback but continue).

When discussing this with gps we decided not to retry fetching the file again since we might have a bad upload.

This will hopefully fix bug 1300345 forever.
(Assignee)

Comment 1

2 years ago
Looking at:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=36399206#L604

I was surprised to see the job continues with a bad zip file.

[1]
 [task 2016-09-23T19:44:42.643603Z] 19:44:42    INFO - retry: Calling unzip with args: (), kwargs: {'file_object': <addinfourl at 44969136 whose fp = <socket._fileobject object at 0x2ae37d0>>, 'verbose': False, 'extract_dirs': ['bin/*', 'certs/*', 'config/*', 'mach', 'marionette/*', 'modules/*', 'mozbase/*', 'tools/*', 'mochitest/*'], 'extract_to': '/home/worker/workspace/build/tests'}, attempt #1 
 [task 2016-09-23T19:46:24.106847Z] 19:46:24    ERROR - File is not a zip file
 [task 2016-09-23T19:46:24.108076Z] 19:46:24    ERROR - Traceback (most recent call last):
 [task 2016-09-23T19:46:24.108317Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.108703Z] 19:46:24    ERROR -   File "/home/worker/workspace/mozharness/mozharness/base/script.py", line 484, in unzip
 [task 2016-09-23T19:46:24.109072Z] 19:46:24    ERROR -     with zipfile.ZipFile(compressed_file) as bundle:
 [task 2016-09-23T19:46:24.109418Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.109783Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/zipfile.py", line 714, in __init__
 [task 2016-09-23T19:46:24.110154Z] 19:46:24    ERROR -     self._GetContents()
 [task 2016-09-23T19:46:24.110513Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.110903Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/zipfile.py", line 748, in _GetContents
 [task 2016-09-23T19:46:24.111276Z] 19:46:24    ERROR -     self._RealGetContents()
 [task 2016-09-23T19:46:24.111628Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.111991Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/zipfile.py", line 763, in _RealGetContents
 [task 2016-09-23T19:46:24.112334Z] 19:46:24    ERROR -     raise BadZipfile, "File is not a zip file"
 [task 2016-09-23T19:46:24.112648Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.112978Z] 19:46:24    ERROR - BadZipfile: File is not a zip file
 [task 2016-09-23T19:46:24.432277Z] 19:46:24 INFO - Downloading and extracting to /home/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/* from https://queue.taskcluster.net/v1/task/ZKTjfv1LSGeClK3WZUP9YQ/artifacts/public/build/target.mochitest.tests.zip
(Assignee)

Comment 2

2 years ago
Somebody pushed old code to inboud, thus, the spike of intermittents.

<armenzg> Armen Zambrano Gasparnian can someone confirm this with me?
3:45 PM did someone push to try code almost two weeks old?
3:45 PM https://hg.mozilla.org/integration/mozilla-inbound/file/72c8ea085d7227204666b528d001f34c1cf4b113/testing/mozharness/mozharness/base/script.py#L483-501
3:46 PM I'm so confused
3:46 PM 
<philor> people push old revs to try constantly
3:46 PM 
<armenzg> Armen Zambrano Gasparnian philor: but this is inbound? https://hg.mozilla.org/integration/mozilla-inbound/rev/72c8ea085d7227204666b528d001f34c1cf4b113
3:46 PM did they use -f?
3:46 PM https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=72c8ea085d7227204666b528d001f34c1cf4b113&selectedJob=36399206
3:47 PM oh wow what a waste of my time
3:47 PM I've been trying to remove all these intermittent badzipfile issues
3:47 PM and I thought I got it all nailed down and it suddently spikes
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → INVALID
Blocks: 1305752
You need to log in before you can comment on or make changes to this bug.