642701 - post_upload.py race condition

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect, P3)

Description

[Chris AtLee [:catlee]]

Hit this on 4.0rc2:
Running post-upload command: post_upload.py -p firefox -n 2 -v 4.0rc2 --release-to-candidates-dir
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0rc2-candidates/build2/linux-i686/et/firefox-4.0rc2.tar.bz2
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0rc2-candidates/build2/update/linux-i686/et/firefox-4.0rc2.complete.mar
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0rc2-candidates/build2/linux-i686/xpi/et.xpi
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0rc2-candidates/build2/linux-i686/et/firefox-4.0rc2.checksums
Traceback (most recent call last):
  File "/usr/local/bin/post_upload.py", line 425, in ?
    func(options, upload_dir, files)
  File "/usr/local/bin/post_upload.py", line 216, in ReleaseToCandidatesDir
    os.chmod(os.path.join(root, d), 0755)
OSError: [Errno 2] No such file or directory: '/home/ftp/pub/firefox/nightly/4.0rc2-candidates/build2/linux-i686/xpi/.~NHLg4J'
Command ['ssh', '-o', 'IdentityFile=~/.ssh/ffxbld_dsa', '[e-mail]', 'post_upload.py -p firefox -n 2 -v 4.0rc2 --release-to-candidates-dir "/tmp/tmp.cojWy29446/" "/tmp/tmp.cojWy29446/linux-i686/et/firefox-4.0rc2.tar.bz2" "/tmp/tmp.cojWy29446/update/linux-i686/et/firefox-4.0rc2.complete.mar" "/tmp/tmp.cojWy29446/linux-i686/xpi/et.xpi" "/tmp/tmp.cojWy29446/linux-i686/et/firefox-4.0rc2.checksums"'] returned non-zero exit code: 1
make: *** [upload] Error 2

I believe this is caused by another process uploading files into this directory at the same time. Files get copied to a temporary name first, and then get renamed to their final name.

The code in post_upload.py gets a directory listing which includes this temporary file, and by the time it gets around to running os.chmod, the file has been renamed.

Comment 1

[Chris Cooper [:coop] (he/him)]

Mass re-assign of post_upload.py bugs to Tomcat.

Comment 2

[Carsten Book [:Tomcat]]

taking a look into this

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found in triage.

Comment 4

[[e-mail] (:bhearsum)]

this script is dead