626133 - [traceback] Sometimes new files are present when we try to validate

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Code Quality, defect, P3)

Description

[Wil Clouser [:clouserw]]

This is the yin to bug 625928's yang.  I've seen this in the logs too:

Jan 15 13:55:29 pm-app-amo15.mozilla.org: [190.10.107.153] z.files:INFO UPLOAD: 'firefm.xpi' (400282 bytes) to path('/mnt/netapp_amo/addons.mozilla.org-remora/files/temp/a6e3ed1a6fa842c395397395841713ff.xpi') :/data/amo_python/www/prod/zamboni/apps/files/models.py:262
Jan 15 13:55:32 pm-app-amo13.mozilla.org: [None] files.utils:ERROR XPI parse error :/data/amo_python/www/prod/zamboni/apps/files/utils.py:150 Traceback (most recent call last):   File "/data/amo_python/www/prod/zamboni/apps/files/utils.py", line 145, in parse_xpi     zip.extractall(path)   File "/usr/lib/python2.6/zipfile.py", line 935, in extractall     self.extract(zipinfo, path, pwd)   File "/usr/lib/python2.6/zipfile.py", line 923, in extract     return self._extract_member(member, path, pwd)   File "/usr/lib/python2.6/zipfile.py", line 960, in _extract_member     os.mkdir(targetpath) OSError: [Errno 17] File exists: '/mnt/netapp_amo/addons.mozilla.org-remora/shared_storage/tmp/1295128532.89/defaults'

We should fix the problem and also fail more gracefully than a traceback in the logs.

Comment 1

[Wil Clouser [:clouserw]]

From parse_xpi():

>     path = os.path.join(settings.TMP_PATH, str(time.time()))

We should be using tempfile's logic for unique dirs.  Doesn't seem like that is the problem here though, since Jorge tried it 3 more times.

Comment 2

[Wil Clouser [:clouserw]]

I've only seen this in the logs with firefm and it appears to happen every time jorge tries to upload.  The hashes are different as well as the tmp directory timestamp.  This is pretty much a dupe of bug 626132 at this point, but leaving open in case it ever affects others.  See that bug for a copy of the file to reproduce

Comment 3

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

I think my other similar bug may have fixed this, I did adjust the error to look for IOErrors, and not vomit in our logs.

I'm adding a test specific to firefm

Comment 4

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

http://github.com/jbalogh/zamboni/commit/bd765cb