1742629 - gz file download with content-encoding: gzip as well as content-type application/x-gzip (e.g. from phpmyadmin) gets gz extension but decoded (non-gzip) content

Status: RESOLVED INVALID

(Core :: Networking: HTTP, defect)

Description

[web2mc]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:94.0) Gecko/20100101 Firefox/94.0

Steps to reproduce:

Going on phpmyadmin on local environment (MAMP) or online (at my hoster) and asking to download a database at gzip format

Actual results:

My file is truncated and not gzip encoded. In my test, for a given database, the downloaded file weight 80ko with firefox. it weight 175ko with Chrome

Expected results:

Have a correct gzipped file.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::File Handling' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[:Gijs (he/him)]

Can you capture the local transfer with something like wireshark or similar (when https is not in use) so we can see what is actually happening?

Comment 3

[Clara Guerrero ( Need Info Brindusa Tot Please)]

Hi!
Thanks for the report.
I tried downloading https://www.gnu.org/software/gzip/manual/gzip.texi.tar.gz and both chrome and firefox are handling it ok, weight is same in both.
Can you provide a sample file?
Best,
Clara

Comment 4

[web2mc]

Hello.
Sorry for this late response.
I can reproduce the problem on
https://demo.phpmyadmin.net/ using username root with a blank password.

If i ask an export of any database in gzip format, i've got a gzip file that are not valid (mac os display an error message when trying to decompress)

Thank you.

Comment 5

[:Gijs (he/him)]

(In reply to web2mc from comment #4)

Hello.
Sorry for this late response.
I can reproduce the problem on
https://demo.phpmyadmin.net/ using username root with a blank password.

If i ask an export of any database in gzip format, i've got a gzip file that are not valid (mac os display an error message when trying to decompress)

Thank you.

Can you elaborate on how to do this? If I open the site and click the "try demo" link on the top right, I don't get prompted for a password. If I then select a table (I tried a few, but "table" or "students" or "world" all repro the problem easily) on the left-hand side and then pick the "Export" tab at the top, I get this error:

Error in processing request
Error code: 500
Error text: Internal Server Error (rejected)
It seems that the connection to server has been lost. Please check your network connectivity and server status.

in a little box inside the webpage (ie not from Firefox). If I first click one of the tables for those databases and then try the export tab, I get the same error again. So I'm guessing you're doing something else, but I'm not sure what.

Comment 6

[:Gijs (he/him)]

Oh, never mind, figured it out... if you use the "STABLE" site, https://demo.phpmyadmin.net/STABLE/ , you can log in with root and no password, and there export works.

The problem (at least from my tests...) is that phpmyadmin sends you gzip but also sends Content-Encoding: gzip, which means our networking layer decodes it, instead of writing it out as gzip.

content-disposition: attachment; filename=\"students.sql.gz\"
content-encoding: gzip
content-transfer-encoding: binary
content-type: application/x-gzip

If this stream should not be content-decoded, that'd have to be changed in the networking layer, I think.

Comment 7

[Martin Thomson [:mt:]]

This is a relatively simple HTTP problem. It's not a bug in Firefox, it's a bug in the server in that it is misapplying header fields.

When a server uses content-encoding: gzip it is saying that the content it is sending has been encoded using gzip. When it then describes the content, it describes the thing after the gzip encoding has been removed. In this case: the content is SQL.

So either let the file remain compressed:

content-disposition: attachment; filename=\"students.sql.gz\"
content-type: application/x-gzip

or describe the contents:

content-disposition: attachment; filename=\"students.sql\"
content-encoding: gzip
content-type: application/sql

Note that content-transfer-encoding is an email header field, not an HTTP header field.

Comment 8

[web2mc]

I made a download test on the database "sakila" from phpamdin demo stable
https://demo.phpmyadmin.net/STABLE/index.php?route=/database/export&db=sakila

If i made a gzip export from Firefox, i've got a 75ko file wich i can't open.
If i made the export from Chrome, i've got a 742ko valid file.

Comment 9

[:Gijs (he/him)]

(In reply to web2mc from comment #8)

I made a download test on the database "sakila" from phpamdin demo stable
https://demo.phpmyadmin.net/STABLE/index.php?route=/database/export&db=sakila

If i made a gzip export from Firefox, i've got a 75ko file wich i can't open.
If i made the export from Chrome, i've got a 742ko valid file.

I don't know why the response gets truncated in Firefox (my guess was a content-length header but it doesn't seem like we get one, per devtools), though the server sending bogus headers can't be helping... Martin?

This was previously fixed in phpmyadmin, per comments in bug 424306. On your local server, dropping the $content_encoding = 'x-gzip'; line from export.php may fix this in phpmyadmin, if that line is still there / back again?

Comment 10

[:Gijs (he/him)]

x-ref https://github.com/phpmyadmin/phpmyadmin/issues/16042 and https://bugzilla.mozilla.org/show_bug.cgi?id=233047 .

Comment 11

[Martin Thomson [:mt:]]

I just tried this twice with different databases and both appear to be complete files. sakila.sql is the file that is saved. There is no content-encoding on the response that I'm seeing. I'm using HTTP/2 and Firefox Nightly.

However, I'm getting a truncated response in Firefox release (content-encoding still absent). That suggests that maybe something has changed. Kershaw pointed at Bug 1030660, which landed in the right sort of timeframe to maybe affect this. I didn't trace the connection carefully to see what was causing the truncation.

I was maybe incorrect previously. As bz declared the intent to download without decoding in Bug 233047, so we maybe should be saving the content without removing the encoding. This is somewhat ambiguous in specifications as the current HTTP specification says:

Likewise, an origin server might choose to publish the same data as multiple representations that differ only in whether the coding is defined as part of Content-Type or Content-Encoding, since some user agents will behave differently in their handling of each response (e.g., open a "Save as ..." dialog instead of automatic decompression and rendering of content).

My interpretation of that text is that if the intent is to have a gzipped file saved, then the content should be served as "application/x-gzip" with no content-encoding. Serving as "application/sql" with content-encoding: gzip would also be acceptable, but what a browser does with that would not be assured. If we were to follow the declared intent (which is 13 years old), then we might save the .gz file. I haven't checked what we do with gzipped content right now (I've not the time to spin up a test).

I was unable to find any language about content-encoding for downloads other than the above. Fetch always processes content codings that are understood and RFC 6266 (content-disposition) says nothing.

As an aside, the sizes mentioned in https://github.com/phpmyadmin/phpmyadmin/issues/16042 indicate that maybe the server is sending a content-length that we're mis-applying. That is, the truncation point appears to match the length of what a gzip-compressed SQL file would match. My own experience shows truncation much, much sooner though, so there is probably something else at play.