90558 - Disk cache should create files with MIME-based extension when possible [was: Some plugins (e.g. acrobat in some cases, netscape's LiveAudio always) require file extension.]

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect, P2)

Description

[Joseph Elwell]

The LiveAudio plugin is being migrated over from Netscape 4.x. The plugin file
"npaudio.dll" supports many file types.

LiveAudio *requires* the file extension be saved in the cache in order to figure
out the file type.

Visiting the above URL on a branch build from July 11th 2001 gives the following
error: "Unknown extension on file c:\documents and settings\jelwell\application
data\mozilla\users50\default\fq1rmaf1.slt\newcache\e91553c7d01"

Renaming "e91553c7d01" to "e91553c7d01.wav" and dragging it onto the browser
plays the file just fine. Since there is no way (that I see) to turn off the
cache, this pretty much makes basic sound files worthless in Mozilla.

From the about:plugins page:

LiveAudio

    File name: C:\Program Files\Netscape\Communicator\Program\Plugins\npaudio.dll
Sound Player for Netscape Navigator, v.1.1.1515

Mime Type Description Suffixes Enabled
audio/x-nspaudio Netscape Packetized Audio(*.LA,*.LMA) la,lma Yes
audio/nspaudio Netscape Packetized Audio(*.LA,*.LMA) la,lma Yes
audio/midi MIDI (*.mid,*.midi) mid,midi Yes
audio/x-midi MIDI (*.mid,*.midi) mid,midi Yes
audio/wav WAV (*.wav) wav Yes
audio/x-wav WAV (*.wav) wav Yes
audio/aiff AIFF (*.aif,*.aiff) aif,aiff Yes
audio/x-aiff AIFF (*.aif,*.aiff) aif,aiff Yes
audio/basic AU (*.au) au Yes

Comment 1

[benc]

Gordon: What do you think of this?

Comment 2

[Peter Lubczynski]

This bug is caused by bug 90570 which is a regression from bug 89407. Bug 89407
now has a new patch that won't cause LiveAudio to be picked up, so this may not
need to be fixed for the branch.

However, if this isn't that hard to do, I think other 3d party plugins will
benefit from using the correct file extension in the cache. I don't have any
examples, though besides LiveAudio. 

Comment 3

[av (gone)]

>Bug 89407 now has a new patch that won't cause LiveAudio to be picked up, so 
>this may not need to be fixed for the branch.

...assuming that the user has QT too. But this may not always be the case.

Comment 4

[gordon]

Adding extensions to cache files is non-trivial, and would most likely require
changing/adding interfaces to the cache.

Comment 5

[gordon]

Darin, Doug, we may need to revisit the "cache as file" functionality if this
problem is wide spread.

Comment 6

[Doug Turner (:dougt)]

do plugin's even need to use the cache?

Comment 7

[Darin Fisher]

why don't we just support file extensions in the disk cache?  would that really
be so difficult?

Comment 8

[gordon]

How is the disk cache supposed to know what the extension is?  Perhaps we could
add an API to set it...

Comment 9

[Darin Fisher]

yeah.. that's what i was thinking.  i'm not sure if we'd want the extension or
the content-type to be passed through the api.  perhaps just the content-type,
since that could be used for other things as well, and we could invoke the mime
service
to map the content-type to the appropriate file extension.

 interface nsICacheEntryInfo : nsISupports {
     // ...
+    attribute string contentType;
 };

Comment 10

[gordon]

Yes, I prefer content type to extension as well.  We would still need a
mechanism for the disk cache device to know how to convert content type to
extension though.

Comment 11

[av (gone)]

Why not to have both?

Comment 12

[Darin Fisher]

gordon: take a look at |nsIMIMEService::GetFromMIMEType|

Comment 13

[Peter Lubczynski]

Note: This bug really hurts us in being compatible with older 4.x plugins and is
another reason we must "gate" what Mozilla can use. File extensions used to be a
big deal and our own LiveAudio plugin that shipped with 4.x won't work with
Mozilla because of this :(

Comment 14

[Arun Ranganathan]

OK, I'm nominating this nsbeta1 .  See bug 122300 for a related problem .  This
follows our conference call with Apple in which we contemplated fixing this
issue from the cache.

Comment 15

[gordon]

I thought they were going to use a different work around.

Comment 16

[gordon]

This feature will not land by Tuesday night.  Moving to 1.0.

Comment 17

[Arun Ranganathan]

ADT wants this one for a future Netscape release, so we're adding "nsbeta1+".

Comment 18

[rubydoo123]

this is actually fairly critical for plugins to work correctly, is this actually
going to be checked in by 1.0?

Comment 19

[serge (gone)]

Attachment: the temp hack

an example of ugly hack to force the files for certain mimetype to be saved
(with file extension) into local plugin cache

Comment 20

[Winnie Lam]

adding adt1 to status whiteboard as per discussion with beppe.

Comment 21

[selmer (gone)]

So, LiveAudio is affected.  Is QT affected?  Prior comments seem inconclusive on
that.  If it's just LiveAudio, what percentage of the user base is that?  Are
there other plugins *known* to have this problem as well?  I agree that this is
severe if you're using an affected plugin, but it's not clear how many people
that is.  If the percentage isn't large, this should be [adt2].

Comment 22

[Jaime Rodriguez, Jr.]

If the answer to Steve's question above, is that a large percentage of users are
effected by this issue, than pls keep it as an ADT1, and add an ETA to the
Status Whiteboard.

Comment 23

[Peter Lubczynski]

Comment on attachment 74388 [details] [diff] [review]
the temp hack

In a conference call with Quicktime a few months ago, they said that in some
circustances their plugin does depend on the file extension. However, I don't
know of any specifc bugs.

I don't like this hack of hard-coding mimetypes that need special caching. What
about using NPN_SetValue or NPP_GetValue?

Comment 24

[Jaime Rodriguez, Jr.]

Changing to [ADT2] per ADT triage.

Comment 25

[serge (gone)]

it's appears not only LiveAudio plugin needs file ext, but in case of stream as 
file acrobat plugin expects file+extension also. (bug 131153 has a temp fix for 
this problem, it's slightly better than hack above, though it actually does 
the same, it copies file content into the temp location, depends on stream type 
plugin requests). 

Comment 26

[Darin Fisher]

-> me since gordon is on sabbatical.

Comment 27

[Peter Lubczynski]

Serge, did bug 131153 contain a hack for this so that the priority and
importance can be lowered? Does LiveAudio now work all the time, or just in some
situations?

Comment 28

[serge (gone)]

well, I don't like that hack at all, because
1. we are doing the extra job to copy a file from mozilla cache into 
temp location,
2.  the way how it works (creating unique temp file) is not good enough for 
plugins:( 
e.g. if plugin N times will call get stream as file from the same url  we'll  
end up with N copies of the same file in temp dir, file name is modified by 
additing "-N", this is how nsLocalFile::CreateUnique() works
http://bonsai.mozilla.org/cvsblame.cgi?&rev=1.38&file=mozilla/xpcom/io/nsLocalFi
leCommon.cpp#77,
and in addition there is awful performance hit when nsLocalFile::CreateUnique()  
fails N-1 times to create filename-N.ext file:(
The problem is we have to keep a temp file alive during live time of the plugin, 
of course, all file are properly deleted on exiting of plugin,
but anyway if cache will have the file extension I think it isn't only plugins 
code will benefit from that.

Comment 29

[Darin Fisher]

serge: sounds to me like the plugin temp file system could use some work then. 
it should be smart enough to reuse a file if it already exists.  making that
work wouldn't be too difficult given the nsICachingChannel::isFromCache() method.

plus, there is an intrinsic problem with referencing cache files directly from
plugins... the cache may decide to delete the file at any time!!  but that's
another bug ;-)

at any rate, fixing _this_ bug would require a non-trivial patch, and
unfortunately gordon is on sabbatical... i know he had concerns about this
patch... that it would seriously impact some of the disk cache data
structures... which makes me really worried about trying to solve this safely
before 1.0.1 :(

Comment 30

[serge (gone)]

darin: I agree plugins temp file system should be revised w/o any dependencies 
on this bug, because plugins use that with https:// protocol

Comment 31

[Darin Fisher]

i'm on vacation and won't be able to work on this before RTM, and i think this
can wait...

-> 1.2alpha

Comment 32

[Darin Fisher]

back to gordon

Comment 33

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 94561 has been marked as a duplicate of this bug. ***

Comment 34

[Andrei Boros]

I think some consideration should go into the following aspect: 

Should cache files be stored with extension based on MIME-type? 
Or should they be stored using the extension of the original filename as 
supplied by the server (URL)?

I would support the second option. Plugins would receive exactly the extension 
of the file as in the requested URL, and avoiding any confusion on the fact that 
a MIME-type can have _multiple extensions_ associated to it!!!
Also, exploring the cache would be easier. 
If a .js or .css is sent as text/plain, would it be cached as a .txt? Confusion.

On the other hand, a consideration might go into the fact that some CGI scripts 
generate images as output. I don't really see any collisions in the second 
option, naming them using the script's extension, whatever the MIME-type they 
sent. I can hardly imagine a script generating a real audio stream, but it's a 
side of the problem that should be considered. 

Comment 35

[Michael Buckland]

Discussed in edt, and sent mail to arun.  Minusing.

Comment 36

[j]

I suppose I do not understand why adding extensions to the filenames in the
cache is 'non trivial'.  Just write the file to the same name as on the server!
 This and bookmark dysfunction were the major reasons delaying my migration from
NS4.8.  

Comment 37

[Anthony Jenkins]

This bug also affects WiMP 9 on Windows XP.  I have Media Player 9.0 (with the
plugins installed for Mozilla), and the following address:

http://us.rd.yahoo.com/launch/promotions/eam/franzferdinand/takemeoutaudio/*http://playlist.yahoo.com/makeplaylist.dll?SID=6039119

will cause media player to horque ("The selected file has an extension that is
not recognized..." followed by "[WMP] cannot play the file...").

The URL provided does not lead to a proper ASX file, but a DLL file that
dynamically generates the ASX, and returns it with the video/ms-asf mime type. 
(So this would be a vote against using the filename as stored on the server).

This is made harder by the fact that many servers are misconfigured, and will
send unknown filetypes as either a text or octet-stream mime type.

Additionally, I don't know if this is a bug or not, or a different one, if it
is, but if I assign that mime type to WMP, Mozilla gives me the spiel about
"Mozilla can handle this type internally...", but whether the helper app is
defined or not, it passes it to WMP, rather than displaying it in the page (I
know, it's not embedded).

Comment 38

[Christian :Biesinger (don't email me, ping me on IRC)]

(In reply to comment #37)
> Additionally, I don't know if this is a bug or not, or a different one, if it
> is, but if I assign that mime type to WMP, Mozilla gives me the spiel about
> "Mozilla can handle this type internally...", but whether the helper app is
> defined or not, it passes it to WMP, rather than displaying it in the page (I
> know, it's not embedded).

Uh, what the plugin decides to do with the URL is not really up to Mozilla to
decide.