Closed Bug 140301 Opened 23 years ago Closed 22 years ago

tgz files get silently unzipped

Categories

(Core Graveyard :: File Handling, defect)

Product:
Core Graveyard
Component:
File Handling
Platform:
x86
Windows 2000
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: jamesrome, Assigned: law)

References

(
URL
)

Details

Attachments

(3 files)

nsHttp:5 log from James (gzipped)
74.04 KB, application/x-gzip
Details
log file asked for by zbarsky
36.88 KB, text/plain
Details
Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp
309 bytes, patch
Biesinger
: review+
bzbarsky
: superreview+
asa
: approval1.4+
Details | Diff | Splinter Review
When one downloads files with a tgz extension (and I think gz also), they get silently unzipped when they are stored (even if one right-clicks and does "save as"). But the extension is not fixed up to reflect this. So, when one tries to open the files with, say, WinZip, it says the file is invalid. gz and tgz files that are being saved (and not displayed) should not be unzipped without 1) informing the user and 2) changing the extension to reflect it.
Which build are you using? This should be fixed for tgz files being sent as application/tar or application/x-tar and content-encoding: gzip (which is what this site does)... More to the point, the files are _not_ unzipped for me in Linux trunk build 2002-04-25-07.
Assignee: blaker → law
Component: Download Manager → File Handling
I have the 4/16 build.
It does work on today's build. But it appends .tar to the file name instead of replacing the .tgz. I would still like the choice of being able to keep the file in its compressed form.
That's exactly what should be happening (and what I see) -- the file getting saved as a .tgz file, compressed. Are you saving with shift-click or by clicking on the link? Are you spoofing your useragent? Could you set NSPR_LOG_MODULES to nsHttp:5, set NSPR_LOG_FILE to some filename, and do a minimal run: start up mozilla, point it to that page, save the file, exit. Then attach the log file to this bug.
That log shows what happens in http-land when doing right-click-and-save-link. In the process, the file gets a ".gz" extension in the filepicker and then gets saved unzipped. (The second case, when just clicking on the file, the file gets saved with a .tar extension, still unzipped). Here are the relevant parts of the log, with my annotations: We ask the server what sort of data this is: 0[234400]: http request [ 0[234400]: HEAD http://www.laurentconstantin.com/common/lcrzo/download/v4/lcrzo-4.08-src.tgz HTTP/1.1 0[234400]: Host: www.laurentconstantin.com 0[234400]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.9+) Gecko/20020426 0[234400]: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/j peg,image/gif;q=0.2,text/css,*/*;q=0.1 0[234400]: Accept-Language: en-us, en;q=0.50 0[234400]: Accept-Encoding: gzip, deflate, compress;q=0.9 0[234400]: Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 0[234400]: Keep-Alive: 300 0[234400]: Proxy-Connection: keep-alive 0[234400]: Pragma: no-cache 0[234400]: Cache-Control: no-cache 0[234400]: ] It responds that it's application/x-tar, encoding gzip: 2740[114fe50]: http response [ 2740[114fe50]: HTTP/1.0 200 OK 2740[114fe50]: X-Cache: MISS from cache.nxs.net 2740[114fe50]: Content-Type: application/x-tar 2740[114fe50]: Content-Encoding: gzip 2740[114fe50]: Accept-Ranges: bytes 2740[114fe50]: Proxy-Connection: close 2740[114fe50]: Content-Length: 660205 2740[114fe50]: Server: Apache 2740[114fe50]: Last-Modified: Mon, 08 Apr 2002 05:28:53 GMT 2740[114fe50]: Etag: "2c32f-a12ed-3cb12a95" 2740[114fe50]: Date: Fri, 26 Apr 2002 18:37:44 GMT 2740[114fe50]: Connection: close 2740[114fe50]: ] We put up the file save dialog and append ".gz" to the filename, since apparently the system does not have ".tgz" set as a valid extension for gzipped files. We also set a "don't decompress this data" flag. User picks a filename, we contact the server to get the data: 0[234400]: http request [ 0[234400]: GET http://www.laurentconstantin.com/common/lcrzo/download/v4/lcrzo-4.08-src.tgz HTTP/1.1 0[234400]: Host: www.laurentconstantin.com 0[234400]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.9+) Gecko/20020426 0[234400]: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/j peg,image/gif;q=0.2,text/css,*/*;q=0.1 0[234400]: Accept-Language: en-us, en;q=0.50 0[234400]: Accept-Encoding: gzip, deflate, compress;q=0.9 0[234400]: Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 0[234400]: Keep-Alive: 300 0[234400]: Proxy-Connection: keep-alive 0[234400]: Pragma: no-cache 0[234400]: Cache-Control: no-cache 0[234400]: ] And it responds: 2740[114fe50]: http response [ 2740[114fe50]: HTTP/1.0 200 OK 2740[114fe50]: X-Cache: MISS from cache.nxs.net 2740[114fe50]: Content-Type: application/x-tar 2740[114fe50]: Accept-Ranges: bytes 2740[114fe50]: Proxy-Connection: close 2740[114fe50]: Server: Apache 2740[114fe50]: Last-Modified: Mon, 08 Apr 2002 05:28:53 GMT 2740[114fe50]: Etag: "2c32f-a12ed-3cb12a95" 2740[114fe50]: Date: Fri, 26 Apr 2002 18:37:59 GMT 2740[114fe50]: Connection: close 2740[114fe50]: ] Notice that the type is still application/x-tar, but there is no content-length and there is no content-encoding! So it looks like at this point the server is sending un-encoded data back... ------------------------------------- I have asked James to try this also without the proxy he normally uses...
so, in other words, this server is not implementing HEAD correctly :-(
Not quite so simple... On Linux the GET has a Content-Encoding and all that. So this is either Windows-specific, or a proxy problem, or _something_ weird.... No sane server should actually _decompress_ .tgz files and serve them unzipped....
More comments from James: With the proxy off, I get the same 2 file names, but the tgz.tar version is definitely unzipped (4.630 KB) while the .tgz.gz is 645 KB. -------------------------------------------------- Well, that's definitely correct behavior... it seems like the server is sometimes sending the data unzipped (and the proxy somehow screws us up there)... In that test, did the .gz version come first? Or the .tar version?
I first did the straight download via clicking the link and then right-clicking with save as
QA Contact: sairuh → petersen
James, is this still a problem? I'm in the middle of doing stuff to this code, and it'd be good to know so I can fix it if so. ;)
The above URL now asks what I want to do. But it offers to use .tgz.tar for the name rather than the correct I think) .tgz. I used the .tgz extension and winzip then says it contains a tar file and offers to expand it.
So this is now basically bug 144334?
It is 144334 for sure. But originally, This bug saved the file unzipped with a .gz extension.
Right. I realize that. :) I made a bunch of changes to this code designed to prevent that... sounds like one of them did the trick. Mind if I mark this worksforme, since neither of us can reproduce it anymore?
Fine by me!
okee. ;)
Status: UNCONFIRMED → RESOLVED
Closed: 23 years ago
Resolution: --- → WORKSFORME
This has raised its head again (11/5 win2k build). I downloas a .zip file. It asks me if I want to open it with winzip or save it. I select save. It downloads it and stores it with a .zip extension. However, the file has already been unzipped and should have a .exe extension.
Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---
Which zip files does that happen for? URL please?
This is getting worse and worse.(11/19 build, win2k) I just received e-mail with .png attachments. When I try to open them, it claims they are zip files with extensions .png.zip. They are not.and winzip comfirms this! Here is the header: ------_=_NextPart_000_01C290E3.DD9D1370 Content-Type: application/octet-stream; name="SFO_ITPUT_vis_box_trellis.PNG" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="SFO_ITPUT_vis_box_trellis.PNG"
Sounds like you have a mapping from .zip to application/octet-stream in your helper app preferences. Which you should really not do unless you know exactly what you're doing. How about answering my questions from comment 19?
http://download.com.com/3000-2141-10154259.html Click the top-left download item. I put this up one time before. I have no idea where it went. You were right about the helper app. I did not put this in. But there was just a new version of winzip. Maybe it did it!
Unlikely, since winzip does not edit the mozilla prefs. Did you ever have a .zip file that was served as application/octet-stream and you chose to open in winzip and said remember this decision? That would have done it... (yes there is a bug on the fact that the UI makes this so easy to do).
Not that I recall. Does the dialog tell you what mime type it is serving?
Yes, it does. In square brackets.
Depends on: 86640
I suspect the problem that caused this bug can never happen again now that bug 86640 is fixed.
This is broken again. (build 2003020208 win2k) Clicking on a link such as file.tar.gz now saves the file as file.tar.gz.tar. And the download manager always shows the file as being 1 kb of 1 kb, even though it was a lot bigger as the file was transferred. For an example, click the top tar link on http://www.cs.dartmouth.edu/~beej/bgp/java/ The action on .exe files is even worse. They get a .jpg extension and get opened in my photo application. See bug 189552
Status: UNCONFIRMED → NEW
Ever confirmed: true
I know what it _NAMES_ it. But is it actually gunzipped? Or just named idiotically?
they both have the same size as the right-clicked save-as version, so the name is probably wrong. Of course, it is hard to tell what is in the file on a PC without trying it. I did. it appears to have been unzipped in both versions. file.tar.gz (right-clicked save as) file.tar.gz.tar (direct save) It should nbot unzip them without my telling it to.
Hmm... how big are your saved versions of http://www.cs.dartmouth.edu/~beej/bgp/java/BGP4_1.4.15.tar.gz (the "tar" link off http://www.cs.dartmouth.edu/~beej/bgp/java/)? (The reason I'm asking all these questions is that I cannot reproduce the problem on Linux, and the same code runs on both platforms....)
1380kb. Download manager says 1 kb
Hmmm... That _is_ unzipped, then. When I download here I get 293kb. Not good. ;) What is your "network.http.accept-encoding" preference set to? (Use about:config to see.)
Keywords: helpwanted
how do I access about:config? It iusn't in the help menu or edit preferences that I can see....
You type "about:config" in the URL bar, then hit enter.
On two different win2k machines, it says default string gzip,deflate,compress,q=0.9 It says that on my linux machine also
Ok, that's what I have too... See, here's the problem. That site sends the file as application/x-tar, encoding gzip. We have code to explicitly check for application/x-tar and _NOT_ decompress it. In my testing, this code is called properly... it sounds like it's not being called in your build. Which is odd, since there is no codepath that allows saving without calling that code. Do you have any extensions (multizilla or the like) installed? Or a plain vanilla Mozilla build from Mozilla.org?
I have the tool bar and spell checker
Which tool bar? And more to the point, does the problem appear _without_ the tool bar?
http://www.xulplanet.com/downloads/prefbar/ Not sure how to remove it. Interesting. On my laptop it worked (which doesn't have the prefbar). There it popped up a box saying application/x-tar one time (but not again). The action is identical on my desktop and laptop for the prefs zip-file, save to disk).
OK... that would not affect the direct-click version of saving. What else is different between the laptop and desktop?
Hard to say since my desktop has 400 GB of disks filled with stuff! And the download manager is also saying all files are 1 Kb except for .exe files.
Let's focus on the problem at hand, ok? The UI behavior is not relevant; what's relevant is the unzipping. Does this happen if you create a new Mozilla profile? I'm assuming that this was a clean Mozilla install following the release notes, bot just unzipping on top of a previous build?
1) I did a clean install last time because I was unable to do anything. So I wiped the directory after an uninstall and then did a reinstall. 2) Using a new profile worked!
Ok.... could you possibly mail me your mimeTypes.rdf and prefs.js files?
James, does the account having the problem include a helper app mapping for application/x-tar files? What about the one that is _not_ having the problem?
WinZip is registered for application/x-tar. When I go to this site, and when I do the download, the saved file is listed as lczroex-4.17.0-src.tgz.tar. However, the helper application settings for this have save to disk checked, NOT "open it with the default application."
James, which profile is that in? The "broken" one or the "non-broken" one?
The only one I use, which is I guess broken in the sense that it does not properly fix up the file names so that you know what is happening.
Wait. Is this bug about files getting UNZIPPED? Or about them being MISNAMED? See comment 42 again.
They are getting unzipped and misnamed!
OK. Could you create a new profile as per comment 43? If that does not show the problem (again per comment 43), could you add an entry for application/x-tar to that profile that is identical to the one in the "broken" profile? Then test with that new profile again?
With a profile with NO helper apps defined, I get the identical behavior. The file gets automatically unzipped and saved with the name lcrzoex-4.17.0-src.tgz.tar.
That's not the behavior you described in comment 43.... Are you using a proxy? Are the steps to reproduce still the same as in comment 0?
No proxy. But I have a much more recent Mozilla build 2003041704. What happens when you try it with the URL?
No proxy. But I am using a much later build 2003041704. I had been just clicking on the file link in the last 2 posts.. I went back to the profile with no helper apps and right-clicked and did save target as. It gave me the original name src.tgz, but the saved file was saved unzipped. And since the extension is wrong, it is hard to know what form it is in (It was .tar).
When I save, no matter how I do it, the file is not unzipped. biesi, can you reproduce this? I suppose it could be Windows-only....
Ahhh. Yes, it works fine on Linux! I am installing a more recent build to see if it still does. I will post if it stops working on Linux.
wfm with win2k build 20040319..
Yup, it works just fine on Linux with the 040105 build. On windows, I also note that the download manager lists the file size as 1KB of 1KB rather than 329KB of 329KB. (Actually, there is another bug here because it should be a lowercase k in kB).
Hmmm.... It worked fine on my laptop (win2k). It also has WinZip registered to handle these files with "Save to Disk" checked.
> On windows, I also note that the download manager lists the file size as 1KB of > 1KB rather than 329KB of 329KB. Is the server returning the same headers on Windows and Linux? See comment 6 again; though there I think it was a proxy bug and you say you are not using a proxy now... (And no, "KB" is in fact correct; a "kB" is 1000 bytes, while a "KB" is 1024 bytes.)
win2k build 2003041609: downloading http://www.laurentconstantin.com/common/lcrzoex/download/v4/lcrzoex-4.17.0-src.tgz offers a filename of "lcrzoex-4.17.0-src.tgz.tar" but does not uncompress the file (ie. it is really a tgz file)
I don't remember where to put the commands to make the file you requested. I put user_pref("NSPR_LOG_MODULES", "nsHttp:5"); user_pref("NSPR_LOG_FILE", "tgz.log"); in prefs.js, but that did not seem to make a file
You have to do it at the prompt (using "set"): set NSPR_LOG_MODULES=nsHttp:5 set NSPR_LOG_FILE=c:\log.txt mozilla
This is zipped text
That log shows exactly what the first log shows -- the server responds to the HEAD with a reasonable response, then responds to the GET with a response that has no content-length, no content-encoding, and type application/x-tar. Darin, do you have any idea what's up with that?
Blocks: 205904
I have seen some servers use "Content-Type: x-tar-gz" and "Content-Encoding: x-gzip" for .tgz files. For example, http://heanet.dl.sourceforge.net/sourceforge/rox/archive-1.9.1.tgz gives the following headers at the moment... Date: Mon, 02 Jun 2003 01:34:18 GMT Server: Apache/2.0.46 (Unix) Last-Modified: Wed, 05 Feb 2003 13:04:21 GMT ETag: "75003b3e-425d-6cadf40" Accept-Ranges: bytes Content-Length: 16989 Connection: close Content-Type: application/x-tar-gz Content-Encoding: x-gzip Mozilla has no clue about "application/x-tar-gz" or the ".tgz" extension, so it uncompresses the file and saves it with the native .tgz extension, creating a filesystem lie (a .tgz file that's really uncompressed). This is arguably a broken server and should be fixed (we've notified the SourceForge admins), but I believe that ".tgz" should be added to the list of nonDecodableExtensions in nsExternalHelperAppService.cpp; such a change will almost certainly break nothing (since we try to avoid uncompressing tarballs in general) and fix this problem on a number of servers of questionable configuration. It might also make sense to add "application/x-tar-gz" to the list of nonDecodableTypes, but I'm not sure I want to give that MIME "type" so much credibility...
Dan, you're right. We should just add .tgz to that extension list. There was another bug that I recall where that came up... Could you post a patch? (I have no tree right now to diff against.) Don't add that application/x-tar-gz type to the type list, though...
Comment on attachment 124709 [details] [diff] [review] Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp sr=me. biesi, would you review (and check in, if you could?)
Attachment #124709 - Flags: superreview+
Attachment #124709 - Flags: review?(cbiesinger)
Attachment #124709 - Flags: review?(cbiesinger) → review+
sure, no problem. checked in: Checking in uriloader/exthandler/nsExternalHelperAppService.cpp; /cvsroot/mozilla/uriloader/exthandler/nsExternalHelperAppService.cpp,v <-- nsExternalHelperAppService.cpp new revision: 1.187; previous revision: 1.186 done
Status: NEW → RESOLVED
Closed: 23 years ago → 22 years ago
Keywords: helpwanted
Resolution: --- → FIXED
Comment on attachment 124709 [details] [diff] [review] Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp a=asa (on behalf of drivers) for checkin to the 1.4 branch. Please add the "fixed1.4" keyword when this has landed on the branch. Thanks.
Attachment #124709 - Flags: approval1.4? → approval1.4+
fixed on 1.4 branch too: Checking in uriloader/exthandler/nsExternalHelperAppService.cpp; /cvsroot/mozilla/uriloader/exthandler/nsExternalHelperAppService.cpp,v <-- nsExternalHelperAppService.cpp new revision: 1.186.4.1; previous revision: 1.186 done
Keywords: fixed1.4
Verified on win32 (2003-06-06-08) trunk and (2003-06-06-05) branch
Status: RESOLVED → VERIFIED
Keywords: fixed1.4 → verified1.4
*** Bug 205904 has been marked as a duplicate of this bug. ***
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: