Closed Bug 67940 Opened 24 years ago Closed 21 years ago

For application/octet-stream, set MIME type from extension/data

Categories

(Core Graveyard :: File Handling, enhancement)

Product:
Core Graveyard
Component:
File Handling
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED WONTFIX
Milestone:
Future

People

(Reporter: matt, Assigned: law)

References

Details

Attachments

(1 file, 1 obsolete file)

patch to get MIME type from Content-Disposition filename and URI that *doesn't* crash
6.94 KB, patch
Details | Diff | Splinter Review
Sometimes a webserver gives the MIME type "application/octet-stream" for a file that has a defined MIME type (for example, I often get octet-stream for PDF files). It would be nice if Mozilla would, for octet-stream MIME types, try to guess the actual MIME type from the file extension.
This would also be useful in mail/news because certain email clients will not attach the correct mime type to a file. A particular example is when I receive png files I often have to save them to disk, and then open them again with mozilla because mail will not display them in-line if they have the application/octet-stream mime type.
Blocks: 61688
Target Milestone: --- → Future
This bug is very annoying for me. I use a fax->mail gateway that sends me Content-Type: application/octet-stream; name="990087634_C3G01F01.TIF" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="990087634_C3G01F01.TIF" Even if I configure TIFF in Helper Apps, I still get a normal download dialog, with not even the tiff helper app prefilled. I guess, they use octet-stream, because some another app would otherwise choke, I don't know. neeti, any hints on how I could fix this? Changing SUMMARY: s/guess/set/, because we know the extensions of the mimetypes - you can set them in the helper app dialog.
Keywords: mozilla1.0
Summary: For application/octet-stream, guess MIME type from extension → For application/octet-stream, set MIME type from extension
Open Networking bugs, qa=tever -> qa to me.
QA Contact: tever → benc
A more general description of this bug is in bug 66677. (f.ex. HTML attachments that arrive as text/plain).
I received mail with the following relevent headers (according to show message source): MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C12F4F.C6F96700" Content-Length: 3705 This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C12F4F.C6F96700 Content-Type: text/plain; charset="iso-8859-1" ... text message... ------_=_NextPart_000_01C12F4F.C6F96700 Content-Type: application/octet-stream; name="DAVIS-T33549-PASCALET.htm" Content-Disposition: attachment; filename="DAVIS-T33549-PASCALET.htm" ... HTML source... Quite reasonably, Mozilla doesn't display the HTML page inline, as its disposition is attachment. If I select the attachment and choose Open, I am presented with a dialog box which proclaims You have chosen to download a file of type "Hyper Text Markup Language" [text/html] from imap... What should Mozilla do with this file? Open using <no application specified> Save this file to Disk It appears to have interpreted the file type from the extension, but doesn't offer the option of displaying the attachment internally (or doing this automatically, since I did say I wanted it opened).
Scott: you're being bitten by bug 78943 and its byblows
Here's a preliminary patch to simulate discussion. If people feel this is the right approach, I'd like to reimplement it a bit more cleanly and fix some other code that wants access to the Content-Disposition filename to go through the new interface it creates. This patch ignores a "Content-Type: application/octet-stream" sent by the server on the grounds that such a header is as useless as sending no "Content-Type" header whatsoever. When faced with such unknown content (i.e., a missing "Content-Type" header or a "stupid" content type like "*/*" or "application/octet-stream"), it *first* tries a filename given in a "Content-Disposition" header. If that gives an extension that maps to a useful MIME type, it uses that. Otherwise, it falls back to trying to derive a MIME type from a file extension in the URI. This is under-tested, but with this patch Mozilla now works as expected when accessing attachments from the Microsoft Outlook Web Access. In this case, the attachment is sent by the server with "Content-Type: application/octet-stream" but with an appropriate filename in a "Content-Disposition" header. With this patch, if there's a matching extension in the MIME types for that content disposition filename, Mozilla will get it right. (Hooray!) Comments? Also, let me note that bug 164996 is really a duplicate of this, but they're both assigned to different people, so I'm hesitant to change anything.
hmm... i do like the idea of adding Content-Disposition to the list of HTTP atoms, since that'll avoid growing the atom table when that header is encountered, but as for the rest, can it move into uriloader/exthandler along with the rest of the content-disposition code?
Oh, you probably want the patch that *doesn't* crash. Here's a second try. I wouldn't be entirely surprised if there weren't more bugs lurking in there though.
Attachment #99252 - Attachment is obsolete: true
Darin: It's not clear to me how to move it into uriloader. The problem is that, for HTTP channels, the MIME type should be decided by the channel's idea of the content type (from a Content-Type header) unless it's a "stupid" value like "application/octet-stream", then by the Content-Disposition filename extension, and finally by an extension calculated via GetTypeFromURI or similar. The first and the last case are already handled by nsHTTPChannel::GetContentType. We'd have to split them up if we wanted to check the content disposition in the middle there. My thought was just the opposite---the content disposition parsing code could be moved *out* of exthandler and into nsHttpResponseHead. There would be nsIHttpChannel and nsIMultiPartChannel methods that exthandler could use to fetch the relevant *parsed* Content-Disposition information it needed. However, I don't really understand the role of exthandler very well. Is it involved in every bit of content or *only* when the content has been recognized as something to be handled externally?
Kevin, the code Darin refers to lives in http://lxr.mozilla.org/seamonkey/source/uriloader/exthandler/nsExternalHelperAppService.cpp#248 (nsExternalHelperAppService::DoContent). It does the type lookup, then does the extension lookup if there's nothing registered for the type. The reason it's not a good idea to change the http channel as you did is that sites will often send HTTP content as octet-stream expecting it to be saved (that's what NS4 and IE do, no?). So they send the content with a Refresh header that redirects to a "done downloading" page or something. Handling such content inline would just mean the user has no way of getting to it.... This problem is inherent in any fix to this bug, including one via the uriloader (the helper app service would need to set the type on the channel and kick the load back to the originating window, basically.... but that encounters the same issues.)
Okay, I think I understand, but I'll have to ponder this a bit. It seems like there should be an extra channel interface method, so we have a pair like nsHTTPChannel::GetContentType and nsHTTPChannel::GuessExternalContentType. The idea would be that, at least for the HTTP channel, GetContentType would always deliver the Content-Type header, even if it was "application/octet-stream" (indicating "download suggested"). If there was no Content-Type header, it could fall back on a channel-dependent algorithm to guess the (inline) content type. For the HTTP channel, it would probably *only* check the extension from the URI and try to map that for a MIME type, as it does now. Then, GuessExternalContentType would be used from the uriloader module as a channel-dependent way to make a better guess when the content type is still unknown or something useless like "application/octet-stream". We could either do this in nsExternalAppHandler::OnStartRequest so we bring up a "what do you want to do with this content?" dialogue that has an appropriate MIME type based on the Content-Disposition filename. Or, we could do it in nsDocumentOpenInfo::DispatchContent just before we're about to pass the request off to the helperAppService. I don't think it makes a difference. Anyway, I'll think about it this week and try testing some things, including the "redirect to `download complete' page" scenario you mentioned.
kevin: you can QI to nsIHttpChannel and call GetResponseHeader("content-type") to access the raw server specified MIME type. GetContentType on the other hand returns the guessed MIME type. so, we basically have the functionality that you're looking for... perhaps the uriloader code just needs to ask for things differently??
Whoa. Let's keep knowledge of the content-type header out of the loader ok? That's _so_ an implementation detail of http... Not to mention that other channels have the same issues as http with application/octet-stream (multipart channels come to mind).
well, my point was that it is possible to infer these things from the channel. of course, if we can find a protocol-agnostic way to do the same, then that's always better.
Not sure, if it's relevant or already considered: Some filesystems store the mimetype in a special field. ext2, XFS and (I think) Mac OS have this ability. Presumably, it would be a good idea to set it for downloads as well. However, I am not sure, who is responsible to figure out a good mimetype, if the provider didn't give one (should we guess and set it or leave it blank, leaving it to OS/apps to figrue it out?). Also, did you consider using |file| on Unix to figure out the correct mimetype? It often has a much better guess than an filename extension lookup.
Ben, your suggestion would be a great improvement to the file channel... Please file a bug on that; cc me.
*** Bug 164996 has been marked as a duplicate of this bug. ***
moving neeti's futured bugs for triaging.
Assignee: neeti → new-network-bugs
Hello -- I often receive messages with application/octet-stream for PDF, EXE, TIF, GIF, EFX (fax). I need some resolution from Mozilla on determining applications for each file suffix/extension. Mail/News is _very_ frustrating to use when viewing a non-zero # of emails with attachments (of various type). This may be a trivial "perceived" functionality -- but difficult in implementation?? I don't doubt there's some issue with lack of standards support -- but end-user functionality should remain high on priority list. I'd be surprised to learn the Netscape7 release has this same problem. Thanks -GA
Start being surprised -- it has the same "problem". One issue is that on the Web authors rely on being able to set application/octet-stream to force "save as" behavior (that is currently the only reliable method). For mail, of course, this is not an issue... The mail channel impls could certainly be fixed to deal. I'll look into doing that.
Summary: For application/octet-stream, set MIME type from extension → For application/octet-stream, set MIME type from extension/data
Depends on: 177026
Referring to comment #20 by Garretta: I've got the same problem. I had a text file (.txt) with the famous MIME type "application/octet-stream". Connected it to Notepad. Now every downloaded .exe file (and others) are saved as ".exe.txt". Now I checked the settings for helper applications: Editing the entry for "application/octet-stream" there, I noticed, that the MIME type must be specified. My suggestion: If I define a helper application for files with a certain extension and the MIME type "application/octet-stream", ignore the MIME type and just check the extension. This includes several options: a) If there's another MIME type with this extension connected to a helper application, use this type. b) Several helper application can be defined for the type "application/octet-stream". c) Alternatively you might introduce a "dummy" MIME type for this or the option "every type". I prefefred option a). Thanks for listening to a generally satisfied Mozilla user. RK
*** Bug 191730 has been marked as a duplicate of this bug. ***
This bug is fairly important for the average user. For example, my mother often gets emails containing images that have the incorrect content-type assigned to them. A quote from her: "Previous to downloading Netscape 7.0, it would have automatically opened up the picture. Hope you have some ideas how to fix this." (She previously used netscape 4.7 as her mail client.) Certainly sounds like a 4xp issue to me
Keywords: 4xp
Now that I think about it, every channel that wants this just needs to stick an nsUnknownDecoder stream converter into the data stream before calling OnStartRequest (as the http channel does for HTTP/0.9 responses, eg). That's it. No changes needed to the uriloader, channels can make their own decisions (eg HTTP should _not_ do this, in my opinion. But for mail, it would make a good deal of sense to do it). Thoughts? (Note that all comments but comment zero talk about this in the context of mail, and all the issues I've raised are only problems in the context of HTTP.)
bz: HTTP doesn't push a nsUnknownDecoder... the uriloader does so on behalf of any channel that cannot provide a specific content-type. the OnStartRequest is delayed by nsDocumentOpenInfo (or whatever the class name is) until nsUnknownDeocder does its thing.
Darin, see nsHttpChannel::CallOnStartRequest (you reviewed that patch, man!)
*oh yeah* :-)
*** Bug 195978 has been marked as a duplicate of this bug. ***
->file handling?
From a QA point, maybe.... This is a Necko bug at heart, but this particular part of necko is classified as "file handling" in the component descriptions.
*** Bug 173236 has been marked as a duplicate of this bug. ***
*** Bug 158050 has been marked as a duplicate of this bug. ***
->file handling
Assignee: new-network-bugs → darin
whoops....
Assignee: darin → law
Component: Networking → File Handling
QA Contact: benc → cpetersen0953
So are we back to Kevin's suggestion in comment #12 ?
Opera has a feature similar to what this is proposing. Their option calls it "determine file type based on extension for unreliable MIME types" or something, and I think that's a pretty good description of what this should be. I would like to see an option to allow filetypes to be defined based on extension (priority over MIME type) since many servers do not properly define many MIME types.
Re: Comment 25, my take is that this is a user-empowerment issue. I'm a pretty savvy developer (though not so much in the web arena), and my first attempt to "do something" about this implementation was to try changing my "helper applications" preference to handle "application/octet-stream" files with extensions of "jpg" (which were the ones that were annoying me) to be handled internally. Hilarity ensued. As for the web sites that want to "force" a download... what if I don't want to be forced? Where am I going with this, you might ask? Why does the GUI for configuring MIME type handlers allow only one entry for each MIME type? It seems to be that this could be handled by allowing users to specify their *own* actions for different extensions for these kinds of meaningless and widely wrongly used MIME types. Then we could have arguments about what the defaults for these should be :-).
Try http://www.testitonline.no/. It tries to download the html pages. Is this problem within the scope of this "bug"?
comment#39: The server is broken and they should send the correct mime/type for the document (text/html) but this is a workaround for this misconfigured server.
But comment#39 is a good example of a web site that is displayed in M$IE even when you think forced downloading should be triggered (it does in virtually all other browsers though). But this behavior probably depend on a file type/content that IE know how to display internally (i.e. text/html/image).
Why is this an enhancement, and not a bug? Is the email client really doing anything wrong? What mime-type should it use for unknown file types? It just gets files to attach, and often know nothing about them except their extentions. How should it be able to figure out the mime-type of all possible file types? I think this should be of hight priority to be fixed, as I assume many of the users are going to experience this problem and no good workaround exists.
>Why is this an enhancement, and not a bug? interesting that you ask, personally I wonder "why is this an enhancement and not WONTFIX" hm... comment 25 suggests to make this mail-only, in which case it may make sense, but why is this in the browser product then? >Is the email client really doing anything wrong? What mime-type should it use >for unknown file types? It just gets files to attach, and often know nothing >about them except their extentions. How should it be able to figure out the >mime-type of all possible file types? That's indeed a quite good question. Why do you think Mozilla can do a better job at extension->type mapping than your mail client can?
> hm... comment 25 suggests to make this mail-only, in which case it may make > sense, but why is this in the browser product then? I suppose the same problem exists for web also, or is it less because of less M$-percentage of servers than for email clients? I also agree that the issue is different for web. When you put something on a http server, you should know what type it is, that is, you are not displaying user content. >> Is the email client really doing anything wrong? What mime-type should it use >> for unknown file types? It just gets files to attach, and often know nothing >> about them except their extentions. How should it be able to figure out the >> mime-type of all possible file types? > That's indeed a quite good question. Why do you think Mozilla can do a better > job at extension->type mapping than your mail client can? Think of this scenario: -I have acroread/acrobat creator installed on my PC, and create a PDF file, place it on a file server. -B, which has no PDF applications installed on his PC, attach the file from that file server to an e-mail and send it to you -you have PDF applications installed, and know that .PDF (usually) are PDFs, a signature for PDF contents and know the MIME-type for PDF, get the attachment as application/octet-stream Here the MIME-type is lost because of the storage on a harddisk, with a file system which doesn't save the MIME-type. Do you think B should install a mapping of filename extentions of mime-types, or be forced to install some PDF applications? If the sender doesn't know the mime type and the receiver knows, I think the receiver should guess. Is there a better mime type for unknown than octet-stream? What should I tell people that say the exact same thing is working as expected in other mail clients (i.e. outlook)?
>I suppose the same problem exists for web also, or is it less because of less >M$-percentage of servers than for email clients? Well, see comment 11. >Do you think B should install a mapping of filename extentions of mime-types, >or be forced to install some PDF applications? That was a good example. Yeah, doing it for mail would be ok for me. But then this bug should be in the mail product. > Is there a better mime type for unknown than octet-stream? No, afaik octet-stream means "I don't know what this is, but it's binary" What should I tell people that say the exact same thing is working as expected in other mail clients (i.e. outlook)?
comment#11: "that's what NS4 and IE do, no?" No, IE doesn't use the content-type: http://www.mversen.de/mozilla/octet/ The server should send a attachment-header if the client should download the file. (And I saw a few servers send content-types like "application/x-download" to forcing a download) It would make sense to use the unkown-content decoder (not the extension) for application/octet-stream because application/octet-strema means = I don't know what this is. I think that the client should handle this file because the server doesn't know what this file is. I don't know if this is true but I think you can't tell Apache to send an unkown File without content-type. The mime-config only allows you to send a general mime-type for unkown files and the mime-list in Apache only contains IANA Mime-types (no mime-types for many known files like .rar or .ace) The xitami web-server does the same (but the default config is a broken "*/*" mime-type for unkown files ) And this would make us compatible with Opera or Safari
Keywords: mozilla1.0
If this is a mail-only bug, it's a trivial matter of hooking the unknown content decoder into streamlistener chain for mailnews channels that have application/octet-stream for the type. The http channel does this already for cases when the server sends no content-type header. Note that: 1) Comment 0 is not about mail 2) We have a separate bug on the mail behavior, filed in the mailnews product, as far as I can recall.
Comment#43 asked "Why do you think Mozilla can do a better job at extension->type mapping than your mail client can?" The correct question should be "why do you think your Mozilla can do a better job at extension->type mapping than someone else's mail client or web server can?" A fix to this bug will allow Mozilla users to have control over how they handle content received from poorly configured mail senders or web servers.
The mail-bug (or at least one of the relevant bugs) is bug 59631 : M$ Virus Outbreak (= Outlook) sends vcards as application/octet-stream.
*** Bug 189391 has been marked as a duplicate of this bug. ***
WONTFIX based on comments. Please file a bug on the mail components if this is to be done only in our mail clients, assuming such a bug is not already filed.
Status: NEW → RESOLVED
Closed: 21 years ago
QA Contact: chrispetersen → ian
Resolution: --- → WONTFIX
that's bug 59631 as mentioned in Comment 49. so please do not file duplicates of that bug. vrfy wont
Status: RESOLVED → VERIFIED
This isn't only for the mail component. Why was this closed WONTFIX "based on comments"?
Because for non-mail cases, this is actually invalid per the specs, as described in comments above.
Exactly. The wontfix is for the non-mail part of this bug. I concur, and so does darin last I checked.
What enhancement bug then contains a feature similar to the opera option mentioned in comment #37?
None that I know of. And I don't think we have any plans to implement something like that (I would say that the wontfix in this bug applies to that idea too).
Seeing as how this is a huge pain for real users, I think something should be done. However, it is true that the specs say that application/octet-stream is for unknown binary files that should be saved to disk. Given that, isn't it a violation of the specs for Mozilla to even *allow* setting a handler for this MIME type (that's what gets most people into trouble)? You can't have it both ways. Personally, I suggested something in comment 38 that I think would be an ideal solution, and that would actually be useful in other situations as well (allowing multiple extensions/handlers per MIME type). Yes, it's a bit of a pain, and something that only advanced users would get any benefit from... but it's not a violation of this spec at least...
> to even *allow* setting a handler for this MIME type At this point, you can't do that accidentally. The only way to do it is to purposefully open up prefs and set the handler for that type, typing in the type. As in, the helper app dialog no longer auto-saves the handler when the type is application/octet-stream. If I want to set the handler to something like a hex editor, no reason why I shouldn't. ;) Please do file a _separate_ clear bug on your suggestion. It's something that we would take if someone implements it (neither biesi nor I are likely to have time to work on it anytime soon, and we seem to be the only people willing to touch this code....)
My Workaround for Linux/KDE: This just passes the mail attachment to KDE/Konqueror to run the default app. You can probably do something similar on Windows with Explorer. I don't know exactly about Gnome. Add the following line to /etc/mailcap: application/octet-stream; /usr/local/bin/attachment.sh "%s" Put the following script in /usr/local/bin/attachment.sh: #!/bin/bash kfmclient exec "$1" Make it executable: chmod ugo+x /usr/local/bin/attachment.sh In Edit:Preferences:Navigator:Helper Applications, Add a New Application: For MIME Type, use 'application/octet-stream' Add some file extensions: 'wpd doc pdf' Select "Open it using the default application" Uncheck "Always ask me before handling files of this type" As long as the KDE file manager is setup with the right application, this will work. With my system (Fedora Core 1) Mozilla copies the attachment to "/tmp" then the scripts pass the URL of the attachment to kfm-client which opens it with the correct application. It takes a couple of seconds on a slow system, but it works.
Wouldn't that also open shell scripts and exes (using wine) etc.? If so, that's extremely dangerous.
> then the scripts pass the URL of the attachment to kfm-client surely the filename of the /tmp file, not a url?
> surely the filename of the /tmp file, not a url? Sorry: of course, the filename. I was playing around with passing url's for a while so I was still in that mindset when I wrote that comment. I couldn't figure out whether Konq supports the mailbox:// url and I couldn't really find a good reference for the % variables, so it just saves the attachment and passes the filename. > Wouldn't that also open shell scripts... ? Afaik, not unless Moz saves the attachment with the execute bit, in which case, yes, that would be a huge problem. Although it would be a problem with Moz, not my script. You might be able to force the (wrong) behaviour through Konqueror by associating scripts with their interpreter, but I don't think Konq is set up to do that automatically. On my system at least, I don't see any scripts or Wine executables that are associated. Even so, I would hope that Konq would respect the filesystem permissions, but who knows.
*** Bug 222132 has been marked as a duplicate of this bug. ***
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: