Stop script execution if bad zip file for download_unpack()



3 years ago
3 years ago


(Reporter: armenzg, Assigned: armenzg)


Firefox Tracking Flags

(Not tracked)


On bug 1300812 I made download_unpack() reliable against network issues.

Unfortunately, there's a case when the fetch data is correct, however, unzipping turns out to be a bad zip file.

I logged this in the code, however, it should be fatal() logging (the script stops) rather than exception() (We get a traceback but continue).

When discussing this with gps we decided not to retry fetching the file again since we might have a bad upload.

This will hopefully fix bug 1300345 forever.
Looking at:

I was surprised to see the job continues with a bad zip file.

 [task 2016-09-23T19:44:42.643603Z] 19:44:42    INFO - retry: Calling unzip with args: (), kwargs: {'file_object': <addinfourl at 44969136 whose fp = <socket._fileobject object at 0x2ae37d0>>, 'verbose': False, 'extract_dirs': ['bin/*', 'certs/*', 'config/*', 'mach', 'marionette/*', 'modules/*', 'mozbase/*', 'tools/*', 'mochitest/*'], 'extract_to': '/home/worker/workspace/build/tests'}, attempt #1 
 [task 2016-09-23T19:46:24.106847Z] 19:46:24    ERROR - File is not a zip file
 [task 2016-09-23T19:46:24.108076Z] 19:46:24    ERROR - Traceback (most recent call last):
 [task 2016-09-23T19:46:24.108317Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.108703Z] 19:46:24    ERROR -   File "/home/worker/workspace/mozharness/mozharness/base/", line 484, in unzip
 [task 2016-09-23T19:46:24.109072Z] 19:46:24    ERROR -     with zipfile.ZipFile(compressed_file) as bundle:
 [task 2016-09-23T19:46:24.109418Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.109783Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/", line 714, in __init__
 [task 2016-09-23T19:46:24.110154Z] 19:46:24    ERROR -     self._GetContents()
 [task 2016-09-23T19:46:24.110513Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.110903Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/", line 748, in _GetContents
 [task 2016-09-23T19:46:24.111276Z] 19:46:24    ERROR -     self._RealGetContents()
 [task 2016-09-23T19:46:24.111628Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.111991Z] 19:46:24    ERROR -   File "/usr/lib/python2.7/", line 763, in _RealGetContents
 [task 2016-09-23T19:46:24.112334Z] 19:46:24    ERROR -     raise BadZipfile, "File is not a zip file"
 [task 2016-09-23T19:46:24.112648Z] 19:46:24    ERROR - 
 [task 2016-09-23T19:46:24.112978Z] 19:46:24    ERROR - BadZipfile: File is not a zip file
 [task 2016-09-23T19:46:24.432277Z] 19:46:24 INFO - Downloading and extracting to /home/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/* from
Somebody pushed old code to inboud, thus, the spike of intermittents.

<armenzg> Armen Zambrano Gasparnian can someone confirm this with me?
3:45 PM did someone push to try code almost two weeks old?
3:45 PM
3:46 PM I'm so confused
3:46 PM 
<philor> people push old revs to try constantly
3:46 PM 
<armenzg> Armen Zambrano Gasparnian philor: but this is inbound?
3:46 PM did they use -f?
3:46 PM
3:47 PM oh wow what a waste of my time
3:47 PM I've been trying to remove all these intermittent badzipfile issues
3:47 PM and I thought I got it all nailed down and it suddently spikes
Closed: 3 years ago
Resolution: --- → INVALID
Blocks: 1305752
You need to log in before you can comment on or make changes to this bug.