1011518 - Blobber upload fails for compressed files on Windows.

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[James Graham [:jgraham]]

Failure messages look like 

17:51:00     INFO -  IOError: [Errno 13] Permission denied: 'c:\\docume~1\\cltbld~1.t-x\\locals~1\\temp\\tmp6yk8ik'

This appears to be due to trying to get two handles to a single temporary file. Patch on GitHub.

Comment 1

[James Graham [:jgraham]]

Attachment: Patch to just use a single file handle for compressed files

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 8423884 [details] [review]
Patch to just use a single file handle for compressed files

I tested this with blobberc on Windows and it worked fine for a file that got compressed (.txt), a file that didn't get compressed (.png), and a file that got served with content-disposition attachment (.zip).

Can't believe I wrote the code like that, I've definitely encountered this footgun before.

Comment 3

[(not currently active) Ted Mielczarek]

I think some of the confusion of that code was from before I fiddled things to pass the filename down to post_file, so it may have been unnecessarily complex.

Comment 4

[James Graham [:jgraham]]

What has to happen to land this / get it into production?

Comment 5

[(not currently active) Ted Mielczarek]

mtabara needs to merge that PR, and release a new version to pypi, then someone in releng-land needs to put a new tarball in tooltool and bump the version in mozharness (I know this from getting bug 962356 deployed).

mtabara, rail: can you two make that happen?

Comment 6

[Mihai Tabara [:mtabara]⌚️GMT]

Merged the PR. Released a new version: 1.1.4 (tar ball here - https://pypi.python.org/pypi/blobuploader/ )
Patched mozharness below for after new tar all gets into Moz repo.

Comment 7

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bump blobuploader version to 1.1.4

Comment 8

[Mihai Tabara [:mtabara]⌚️GMT]

Released a new version of blobuploader (1.1.5) with changes for bug 998796.
Tar ball, as usual - here: https://pypi.python.org/pypi/blobuploader/
Re-patched mozharness below.

Comment 9

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bump blobuploader version to 1.1.5

Just a brief reminder, this patch is to be deployed *after*:
1. Blobber server EB2 restart with new changes (bug 998796)
2. Blobuploader tar ball (1.1.5) puppetized in toolttool

Comment 10

[James Graham [:jgraham]]

Thanks!

Is there anything I need to do to push this along? Otherwise do we have an ETA on getting the updates deployed? I have some worked blocked on this so it's helpful to know if it will be days or weeks.

Comment 11

[Mihai Tabara [:mtabara]⌚️GMT]

It's a matter of 1-2 days usually if there is norhing more urgent in the queue.
Lemme ping Rail about this.

Comment 12

[Rail Aliiev [:rail]]

I'll deploy all the things tomorrow. Canada is off today ;)

Comment 13

[Rail Aliiev [:rail]]

Comment on attachment 8424292 [details] [diff] [review]
Bump blobuploader version to 1.1.4

The next patch obsoletes this one

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8424426 [details] [diff] [review]
Bump blobuploader version to 1.1.5

https://hg.mozilla.org/build/mozharness/rev/98c98ca24f41

default only