Closed Bug 121001 Opened 23 years ago Closed 23 years ago

Mozilla gzip-compresses this file on download, but it opens fine directly in browser.

Categories

(Core Graveyard :: File Handling, defect)

Product:
Core Graveyard
Component:
File Handling
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
mozilla1.2alpha

People

(Reporter: jesse.houwing, Assigned: bugs)

References

(
URL
)

Details

Open the url. Rightclick the top faq/walkthrough (the one of +- 500 kb). and choose save link as. Save to you harddisk and open in notepad or something simular. The result is a unreadable file.
Confirming with 2002-01-19-22 on Linux. For some reason mozilla seams to gzip compress this file on download. I ran 'file' on the downloaded file and it reported this: baldurs_gate_c.txt: gzip compressed data, deflated, last modified: Thu Jan 1 01:00:00 1970, max compression, os: MS-DOS I renamed the file to baldurs_gate.gz, ran gzip -d on it and then I could view it. If I download it with 'wget' it detects it as plain/text and you can view it right away: --15:47:26-- http://db.gamefaqs.com/computer/doswin/file/baldurs_gate_c.txt => `baldurs_gate_c.txt' Connecting to db.gamefaqs.com:80... connected! HTTP request sent, awaiting response... 200 OK Length: 564,597 [text/plain]
Summary: Files opens fine in browser (only a few characters not showing up) bus save as is garbled → Mozilla gzip-compresses this file on download, but it opens fine directly in browser.
Probably a dupe of bug 107991.
This sounds alot like bug 51852 which I notice got marked WFM a few days ago, but it's got quite a number of dupes.... Andre, when you tested with wget did you send Accept Encoding header? Mozilla sends |gzip, deflate, compress;q=0.9| for that header value, but wget does not.
OS: Windows 2000 → Linux
Christopher, I just tried with wget --header="Accept Encoding: gzip, deflate, compress;q=0.9" and got this out of wget: Invalid specification `Accept Encoding: gzip, deflate, compress;q=0.9'. What is the right parameter to send?
Hmm, I guess Accept-Encoding is the right one. Here's what I got when I used that: --16:47:59-- http://db.gamefaqs.com/computer/doswin/file/baldurs_gate_c.txt => `baldurs_gate_c.txt' Connecting to db.gamefaqs.com:80... connected! HTTP request sent, awaiting response... 200 OK 2 Server: Microsoft-IIS/5.0 3 Connection: keep-alive 4 Date: Sun, 20 Jan 2002 15:38:28 GMT 5 Content-Type: text/plain 6 Accept-Ranges: bytes 7 Last-Modified: Tue, 03 Jul 2001 18:50:21 GMT 8 ETag: "aac9c22f13c11:802" 9 Content-Length: 564597 10 0K .......... .......... .......... .......... .......... 9% @ 45.25 KB/s 50K .......... .......... .......... .......... .......... 18% @ 59.59 KB/s 100K .......... .......... .......... .......... .......... 27% @ 57.94 KB/s 150K .......... .......... .......... .......... .......... 36% @ 59.67 KB/s 200K .......... .......... .......... .......... .......... 45% @ 59.59 KB/s 250K .......... .......... .......... .......... .......... 54% @ 58.00 KB/s 300K .......... .......... .......... .......... .......... 63% @ 59.52 KB/s 350K .......... .......... .......... .......... .......... 72% @ 57.08 KB/s 400K .......... .......... .......... .......... .......... 81% @ 60.46 KB/s 450K .......... .......... .......... .......... .......... 90% @ 59.59 KB/s 500K .......... .......... .......... .......... .......... 99% @ 57.74 KB/s 550K . 100% @ 1.33 MB/s 16:48:09 (57.47 KB/s) - `baldurs_gate_c.txt' saved [564597/564597] The saved file was in plain text format.
I just tried with telnet, and it is sent gzcompressed over the line. If you set content-encoding in wget it should decode at save (and so it does) telnet won't offer this for you so it's a good alternative.
The server sends us gzip'd content as seen with NSPR_LOG_MODULES=nsHttp:5 Here's a partial log of the transfers of that file. 1024[8086de8]: http request [ 1024[8086de8]: GET /computer/doswin/file/baldurs_gate_c.txt HTTP/1.1 1024[8086de8]: Host: db.gamefaqs.com 1024[8086de8]: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7+) Gecko/20020119 1024[8086de8]: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain; q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1 1024[8086de8]: Accept-Language: en-us 1024[8086de8]: Accept-Encoding: gzip, deflate, compress;q=0.9 1024[8086de8]: Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 1024[8086de8]: Keep-Alive: 300 1024[8086de8]: Connection: keep-alive 1024[8086de8]: Cache-Control: max-age=0 1024[8086de8]: ] =================== 1026[8128880]: http response [ 1026[8128880]: HTTP/1.1 200 OK 1026[8128880]: Server: Microsoft-IIS/5.0 1026[8128880]: Content-Encoding: gzip 1026[8128880]: Date: Sun, 20 Jan 2002 15:51:37 GMT 1026[8128880]: Content-Type: text/plain 1026[8128880]: Accept-Ranges: bytes 1026[8128880]: Last-Modified: Tue, 03 Jul 2001 18:50:19 GMT 1026[8128880]: Etag: "80277c1f13c11:802" 1026[8128880]: Content-Length: 163389 1026[8128880]: Expires: Wed, 01 Jan 1997 12:00:00 GMT 1026[8128880]: Cache-Control: max-age=86400 1026[8128880]: Vary: Accept-Encoding 1026[8128880]: ]
Based on that log, I think this probably *is* a dupe of bug 51852 in which case it needs to be re-opened.
OS: Linux → All
See bug 87449 and bug 108688 for the same problem on gamefaqs. Dupe. *** This bug has been marked as a duplicate of 51852 ***
Status: UNCONFIRMED → RESOLVED
Closed: 23 years ago
Resolution: --- → DUPLICATE
Please see my comments in bug 51852. This is _not_ a duplicate, but rather a new bug that's in a totally different area of the code and needs to be dealt with by a totally different person.
Reopening per bz's comments
Status: RESOLVED → UNCONFIRMED
Resolution: DUPLICATE → ---
->ben
Assignee: darin → ben
Note that per the HTTP1.1 rfc it's not clear whether content-encoding should or should not be decoded if the data is just being saved...
Component: Networking: HTTP → File Handling
WFM using 2002011503 Win2k. Maybe a Linux-only bug?
Does not WFM 2002011604 win2k
*** Bug 96490 has been marked as a duplicate of this bug. ***
confirming, based on bug 96490...
Status: UNCONFIRMED → NEW
Ever confirmed: true
Confirmed on 2002012203 WinXP Also not that bug #51852 (closed at the moment) has 10 votes. This vote has replaced #51852 Please look into this and set it to some near future milestone because this behavior is EXTREMELY annoying. It also happens when you use File->Save Page As... on a Gzipped/deflated page. BTW why is this bug still marked as 'new' and why was #51852 closed and not this one?
Please see comment 10.
This bug is NEW because no one has started working on it yet. This is because it's not clear what we should do: 1) You want text files that are gzipped to get unzipped. 2) You want tar files that are gzipped to _not_ get unzipped 3) What do you want with gzipped postscript files? Oh, and servers send basically identical data in cases #1, #2, and #3... :)
What to do is simple: The layer which is responsible for receiving the HTTP-Response must unzip the content. Reasons: - The browser requested the content as gzipped ("accept-encoding: gzip") (Maybe the accept-encoding is a std request header without brain) - The user do not request it as compressed. - The information that the response is compressed ("content-encoding: gzip") inside the http-header is lost after leaving the http-layer. Another solution is to request a file not as compressed if the decompression cannot be done. How it is implemented inside the renderer? How it knows its compressed, by checking the first bytes or knowing the http-response?
> Maybe the accept-encoding is a std request header without brain It is. We _always_ list the encodings we support. > The user do not request it as compressed Not true. When I request a .tar.gz file I _do_ request it as compressed. When I request a text file, I usually want it uncompressed. For postscript, I tend to want one or the other depending on the size of the file. > How it is implemented inside the renderer? The renderer asks the HTTP layer to always decompress everything before passing it on. We can't do that when saving files because servers send archive files with type and encoding headers such that they would be decompressed. _That_ is the crux of the problem, as I said. Servers make no distinction between files they send that should be decompressed before saving and files they send that should _not_ be decompressed before saving. This happens because Transfer-Encoding is unsupported in nearly all servers and browsers (including Mozilla) so servers have to use Content-Encoding to mean both content-encoding and transfer-encoding. Then we're left trying to guess which they meant... > Another solution is to request a file not as compressed if the decompression > cannot be done. Hmm. That's possibility, actually.... It doesn't help the case when we click on a link, then the user picks "save" from the dialog. At that point, the response is long since made... But it could help the "save as" case. I'm glad we're having this discussion.... This thing is pretty broken as it stands. :)
> > Maybe the accept-encoding is a std request header without brain > > It is. We _always_ list the encodings we support. I understand supporting if the response of the request is decoded. So accept-encoding is not static like the agent statement, it should be set per request. > > > The user do not request it as compressed > > Not true. When I request a .tar.gz file I _do_ request it as > compressed. When > I request a text file, I usually want it uncompressed. For > postscript, I tend > to want one or the other depending on the size of the file. No. If the browser accepts the encoding as gzipped and the web server may compress it again (like a tar.gz.gz) and says in the response "content-encoding: gzip". And the http-engine must it unzip to save a tar.gz and not a tar.gz.gz file. Or the webserver says nothing about the contentencoding and the file is transfered as .tar.gz . In this case the response data is compressed but this doest matter in the view of the http-engine. > > > How it is implemented inside the renderer? > > The renderer asks the HTTP layer to always decompress > everything before passing > it on. We can't do that when saving files because servers > send archive files > with type and encoding headers such that they would be > decompressed. _That_ is > the crux of the problem, as I said. Servers make no > distinction between files > they send that should be decompressed before saving and files > they send that > should _not_ be decompressed before saving. This happens because > Transfer-Encoding is unsupported in nearly all servers and > browsers (including > Mozilla) so servers have to use Content-Encoding to mean both > content-encoding > and transfer-encoding. Then we're left trying to guess which > they meant... The engine must not try to decompress a file, but it must decompress it in case of the content-encoding. (A file .tar.gz is not marked with content-encoding: gzip, because a zip file is not zipped again) > > > Another solution is to request a file not as compressed if > the decompression > > cannot be done. > > Hmm. That's possibility, actually.... It doesn't help the > case when we click > on a link, then the user picks "save" from the dialog. At > that point, the > response is long since made... But it could help the "save as" case. > > I'm glad we're having this discussion.... This thing is > pretty broken as it > stands. :) >
> I understand supporting if the response of the request is decoded. For transfer-encoding, sure. For content-encoding, I'm not convinced... Like I said, the problem is servers using content-encoding to mean both... > And the http-engine must it unzip to save a tar.gz and not a tar.gz.gz file. Too bad web servers send a .tar.gz _singly_ compressed and just claim it to be type application/tar, encoding gzip. > (A file .tar.gz is not marked with content-encoding: gzip, because a zip file > is not zipped again) In an ideal world, sure. In practice, a .tar.gz is sent as either content-type: application/tar content-encoding: gzip or content-type: application/x-gzip content-encoding: gzip by Apache. Since a large fraction of the servers out there are Apache, especially for unix-oriented sites, we have to deal with these broken behaviors.
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.2
Depends on: 69306
Fixed by checkin for bug 69306
Status: ASSIGNED → RESOLVED
Closed: 23 years ago → 23 years ago
Resolution: --- → FIXED
I can verify that the original testcase works with 2002-03-28-08 on Linux. VERIFIED FIXED.
Status: RESOLVED → VERIFIED
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.