Allow simple audio format (wav, aiff) sound files to be played internally

RESOLVED FIXED

Status

()

Core
Audio/Video
--
enhancement
RESOLVED FIXED
17 years ago
8 years ago

People

(Reporter: Steve Chapel, Unassigned)

Tracking

Trunk
Points:
---
Dependency tree / graph
Bug Flags:
blocking1.8b3 -
blocking-aviary1.5 -

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: parity-opera, URL)

Attachments

(1 attachment)

(Reporter)

Description

17 years ago
When I download an audio file of type audio/aiff or audio/wav, I'm able to open
the audio file with a helper app and play the sound. However, this method of
playing the sound is quite distracting because the File Download dialog box
flashes momentarily, then the window of the helper app comes to the front.

I would prefer an option to have Mozilla play the audio file internally so that
the only result of clicking on a link to an AIFF or WAV audio file would be that
the audio plays. This option would be a welcome relief to the users of a web
tool I developed that allows users to review audio and graphics files.
Over to XP miscellany for starters
Assignee: asa → waterson
Status: UNCONFIRMED → NEW
Component: Browser-General → XP Miscellany
Ever confirmed: true
QA Contact: doronr → brendan

Comment 2

17 years ago
It seems to be related to (more general) 92073

Updated

17 years ago
Depends on: 92073

Comment 3

17 years ago
Setting depends on rather than duplicate as I don't know for sure that Mozilla
can handle AIF and WAV internally. In fact, in general one would need to know
for each format whether Mozilla can do it, as 4.X (bugilly) did. What about AU
for example?

Comment 4

16 years ago
I'm sure Mozilla can play sound using the nsSound component, I'd have to check
out what formats it supports though. Anyway, that's how the mail component does
the New Mail sound for instance.

thanks -mike

Updated

16 years ago
Target Milestone: --- → Future

Updated

16 years ago
Status: NEW → ASSIGNED

Comment 5

15 years ago
.
Assignee: waterson → blaker
Status: ASSIGNED → NEW
Component: XP Miscellany → XP Apps: GUI Features
QA Contact: brendan → paw
Summary: [RFE] Allow simple audio to be handled internally → Allow simple audio to be handled internally

Comment 6

14 years ago
Reassigning obsolete bugs to their respective Seamonkey owners (i.e. nobody). 
If you want this fixed for Firefox, change the Product and Component accordingly
and reassign back to me.
Assignee: firefox → guifeatures
Target Milestone: Future → ---
Product: Core → Mozilla Application Suite

Comment 7

13 years ago
Created attachment 173623 [details]
testcase

Updated

13 years ago
Whiteboard: parity-opera

Comment 8

13 years ago
See http://www.phon.ucl.ac.uk/home/mark/audio/play.htm (Example #1) for a
description of this technique for playing a sound.

Updated

13 years ago
Summary: Allow simple audio to be handled internally → Allow simple audio format (wav, aiff) sound files to be played internally

Comment 9

13 years ago
*** Bug 293410 has been marked as a duplicate of this bug. ***
See:
   http://whatwg.org/specs/web-apps/current-work/#sound

Comment 11

13 years ago
Mozilla Application Suite must not be the right product for this bug.

Over to core...
Assignee: guifeatures → nobody
Component: XP Apps: GUI Features → XP Miscellany
Product: Mozilla Application Suite → Core
QA Contact: pawyskoczka → brendan

Comment 12

13 years ago
I am working on a product that will have a lot of usage and could really take
advantage of the change proposed here. Would be great to see this in 1.1.
Flags: blocking-aviary1.1?

Updated

13 years ago
Flags: blocking1.8b3?

Updated

13 years ago
Flags: blocking1.8b3? → blocking1.8b3-

Comment 13

13 years ago
*** Bug 113560 has been marked as a duplicate of this bug. ***
not on the radar for 1.8, unless a solid and low-risk patch appears
Flags: blocking-aviary1.1? → blocking-aviary1.1-

Comment 15

12 years ago
Taking for preliminary investigation.
Assignee: nobody → pkasting

Comment 16

12 years ago
Peter, 

should you choose to take this mission on, would you please consider SVG.
I've been asking for sound for almost a decade :-)

if you want a test page http://www.peepo.co.uk isn't quite right, and I'll be happy to provide test cases

cheers

Comment 17

12 years ago
My knowledge of SVG is basically 0.  I think the best course, if I can do something on this bug, would be to get an initial patch that fixes the HTML cases, then see what more would need to be done for SVG at that point.

Unless I'm misunderstanding your request.
The spec anchor has changed since Hixie originally linked it:

http://whatwg.org/specs/web-apps/current-work/#scs-sound

Updated

12 years ago
Blocks: 334920

Comment 19

12 years ago
re #16 https://bugzilla.mozilla.org/show_bug.cgi?id=334920
If this becomes about implementing the WHATWG Audio object (and I think it should be -- this would be an awesome thing to have), the bug description should probably morph into that.  I had some comments regarding Audio on the whatwg mailing list -- see the thread starting at http://listserver.dreamhost.com/pipermail/whatwg-whatwg.org/2006-April/006260.html .

I do think that the whatwg spec needs some more input here before being implemented; I know the Opera folks have gone ahead and implemented what Hixie specced, but I think it's premature.  Some of the questions got raised in the thread linked above.

I would ignore SVG for now (audio has no business being in a 2d vector graphics format, grumble); whatever internal interfaces end up being created to handle audio (some more advanced version of nsISound) can be reused for whatever svg wants to expose.

Comment 21

12 years ago
(In reply to comment #20)
> If this becomes about implementing the WHATWG Audio object...

Could you file a new bug report about that, please? I think the original request is just as valid as ever.

Comment 22

12 years ago
(1) Let's definitely leave the WHATWG Audio stuff to a separate bug.  It's already unclear to me what this bug is really about, I don't want to confuse the issue further.
(2) So, speaking of: what _IS_ this bug about?  I have some reluctance to do what the original requester seems to want, because we eliminate the ability for helper apps or plugins to handle audio at all -- and if we're going to do that, our in-browser implementation better be quality, i.e. have a set of transport controls etc.  As far as I can tell, IE doesn't do this -- it lets WMP (or another plugin) handle audio.  And frankly, for clicked-on audio links, that seems sufficient to me.

So what we're left with is things like <bgsound>, <object>, and <embed>.  Perhaps we should implement support for bgsound natively -- that seems doable and I don't think it will interfere with anything or require us to draw play controls.  As for <object> and <embed>, IE seems to be passing those off to plugins, why shouldn't we?

The point I'm trying to make is that taking over audio playback from cases where plugins currently support it is both risky and not as simple to get right as it sounds.  I don't see much upside there.

I'm open to others telling me why I'm stupid and wrong.  Until then, I'm inclined to consider this bug a request to implement <bgsound>, and I believe we should implement the WHATWG Audio spec (or a future version), tracked under a separate bug.

Comment 23

12 years ago
(In reply to comment #22)
> I have some reluctance to do
> what the original requester seems to want, because we eliminate the ability
> for helper apps or plugins to handle audio at all...

No, as the original reporter, eliminating ability is not what this bug is about at all. Currently, when the browser plays an audio file, you *must* pass that functionality off to a helper app. This bug requests that *in addition* to that ability, you also have the option of having the browser play the sound itself. Having a helper app come up every time you click on an audio link can be very annoying. Ideally, every way of playing a sound described at <http://www.phon.ucl.ac.uk/home/mark/audio/play.htm> except those that use a plugin should be able to be handled by the browser itself, at least for the simplest audio formats such as wav and aiff and possibly au.

> consider this bug a request to implement <bgsound>

That's not what this bug is about either. Requests to support <bgsound> have been correctly marked WONTFIX.

Comment 24

12 years ago
First, on IE, every one of those methods uses a plugin, as far as I can tell.  IE does not seem to play .WAV or .AIFF or anything else internally in the case where you click a link to such a file, and I still don't understand why we should either.  What's wrong with letting the user's plugin kick off to handle it?  --Because I don't want to be in the business of adding a full-fledged audio player with transport controls to the core, and that's what you'd need to do to be able to play such a link internally properly, as far as I can tell.

Second, from what I can tell on bug 92073, we currently mandate that internally-handlable types get handled internally, and don't allow them to be passed to plugins.  Hence my comments about eliminating ability.  To do what you want, 92073 must be fixed first, as far as I can tell.  Yes?

So, if we implement 92073, and then implement this, we gain the ability for users to play linked .wav files internally.

The effort to fix each of these properly (UI, specifically) versus the benefit over letting Quicktime or some other plugin handle the linked .wav file like it already will on every other, more common type of audio playback, doesn't seem compelling to me.  It ESPECIALLY doesn't seem compelling if we implement the WHATWG Audio spec so there IS a way for pages to play sounds when they need to -- and implementing that is going to be less work than implementing both these other bugs.

Finally, in passing, why do you say it's "correct" for requests to implement <bgsound> to have been marked WONTFIX?  If there are particular bug numbers you're thinking of feel free to link me.

Comment 25

12 years ago
(In reply to comment #24)
> First, on IE, every one of those methods uses a plugin, as far as I can tell. 
> IE does not seem to play .WAV or .AIFF or anything else internally in the case
> where you click a link to such a file, and I still don't understand why we
> should either.  What's wrong with letting the user's plugin kick off to handle
> it?  --Because I don't want to be in the business of adding a full-fledged
> audio player with transport controls to the core, and that's what you'd need to
> do to be able to play such a link internally properly, as far as I can tell.
> 

When we implemented Gmail chat, we experimented with adding audio notifications, but the pluggin model in Firefox caused us endless problems. Each pluggin had a different implementation of the API and different UI. For example, Real Player and sometimes QuickTime would often popup external UI and make the user accept some licensing agreement or make them download some other component. This inconsistent experience resulted in us having to abandon this approach to playing sound. Developers need a consistent and deterministic API. We couldn't add this feature since it degraded the Gmail experience. In IE, we actually had a much more consistent experience for whatever reason.



(In reply to comment #25)
> This inconsistent experience resulted in us having to abandon this
> approach to playing sound. Developers need a consistent and deterministic API.

I think this is exactly one of the use cases that the WHATWG audio spec tries to solve; filed bug 336164 regarding implementing it.

Comment 27

12 years ago
(In reply to comment #25)
> This inconsistent experience resulted in us having to abandon this
> approach to playing sound. Developers need a consistent and deterministic API.

I agree with vlad; for this case the WHATWG stuff is the way to go.

I wonder if we control any API or UI stuff like what you described, where we could fix some of those issues (though I doubt there's anything we can do here).

As for whatever else should be done, I suspect we could (and perhaps should) implement it in terms of the work done for the WHATWG stuff.  I'm going to make this bug depend on that one.
Depends on: 336164

Comment 28

12 years ago
It might be worth contemplating that MS has recently lost a ~$600M court case regarding the way that plugins are handled: see Eolas Technologies patents.

Comment 29

12 years ago
Just to clarify my previous comment. I didn't mean to suggest an implementation - I was just trying to explain a pain point we had and why the current implementation wasn't working for us. I'm hoping the future design will be better.
jdpederlow: can you elaborate (here or in a wiki page) about the approaches you tried, and the differences in behaviour?  It'd be good to capture the practical use-cases here to make sure that they work well with whatever combination of internal-play and <audio> we end up implementing.

(Welcome, pkasting!)

Comment 31

12 years ago
(In reply to comment #24)
> What's wrong with letting the user's plugin kick off to handle
> it?  --Because I don't want to be in the business of adding a full-fledged
> audio player with transport controls to the core, and that's what you'd need to
> do to be able to play such a link internally properly, as far as I can tell.

Many plugins have a horrible UI for playing quick audio snippets, which is usually what aiff and wav files are used for. I've seen a user with dozens of helper app windows open, because each click on an audio file link opened up a new window. That horror is what led me to file this bug report, I believe. Opera already plays audio internally, and there should already be an audio player built in to Firefox and SeaMonkey to play UI sounds such as when find-as-you-type fails to find the text. It seems like it would be easy to do, and would benefit the user so much that at least one other browser has implemented this feature already.

> Second, from what I can tell on bug 92073, we currently mandate that
> internally-handlable types get handled internally, and don't allow them to be
> passed to plugins.  Hence my comments about eliminating ability.  To do what
> you want, 92073 must be fixed first, as far as I can tell.  Yes?

It looks that way, yes. In a way, this bug is merely an extension of bug 92073 specifically for the ability to play audio internally. My guess is that bug not being fixed yet is the main reason this bug is still open.

> Finally, in passing, why do you say it's "correct" for requests to implement
> <bgsound> to have been marked WONTFIX?  If there are particular bug numbers
> you're thinking of feel free to link me.

Sorry, I should have said INVALID. I was thinking of bug 226635. Surely a bug requesting <bgsound> functionality would be quickly closed as a dupe of that bug, whether this bug morphs into that request or we open a new bug report.

Comment 32

12 years ago
(In reply to comment #31)
> Opera already plays audio internally, and there should already be an audio
> player built in to Firefox and SeaMonkey to play UI sounds such as when
> find-as-you-type fails to find the text. It seems like it would be easy to do,
> and would benefit the user so much that at least one other browser has
> implemented this feature already.

Actually playing the audio, without any user-visible controls or any way to pass the file off to a plugin, wouldn't be too bad.  The audio interface you're thinking of is NSISound, which is indeed what we use for the find bar "fart noise" and similar audio events.  I'm not sure this would be sufficient to implement what you want, but certainly an implementation of bug 336164 would be.

No, what I'm much more concerned about is the effort to "do this right" such that it's not an annoyance to users who want transport controls or the ability to let their existing plugins handle things.  It's certainly doable, but it seems low-priority and high effort given the amount of end-user payoff.  If both dependencies of this bug are fixed, this bug should be possible to implement; creating and hooking up the transport controls will be the most difficult thing.  Maybe we could just punt on that aspect to some degree and provide only play and stop buttons.

> > Finally, in passing, why do you say it's "correct" for requests to implement
> > <bgsound> to have been marked WONTFIX?  If there are particular bug numbers
> > you're thinking of feel free to link me.
> 
> Sorry, I should have said INVALID. I was thinking of bug 226635. Surely a bug
> requesting <bgsound> functionality would be quickly closed as a dupe of that
> bug, whether this bug morphs into that request or we open a new bug report.

All right, thanks very much for the bug number.

Comment 33

12 years ago
I don't know if WhatWG is necessarily the answer to this because there's piles upon piles of legacy pages that use audio that won't be using whatever interface WhatWG comes up with because it doesn't exist yet. And there's also plenty of sites with regular old non-embedded links to .wav files.

The answer of "just use a plugin" would be more satisfactory if the plugins available weren't so terrible as far as bloat, etc. QuickTime seems to be the de facto standard, but QuickTime has issues like long gaps when playing looping sounds before the looping starts working properly. Plus, most people don't have any Firefox plugin at all set up to handle audio so whenever I send someone a link to a page with audio embedded I get a chorus of "it says additional plugins are required, I can't be arsed, why doesn't it just work like in IE". If there were a small, free, non-full-of-commercial-crap audio playing plugin that I could just point people to, that would take care of some of the problems (vlc has one but it's hard to install and has no control UI at all) but of course I'd still have to evangelize it.

Netscape used to come bundled with a small audio playing plugin called LiveAudio, a straight resurrection of that with MP3 thrown in would be perfect IMO.
There's a bug on file that I can't find right now about hooking up internal handlers for some content types; if that was fixed, this could be implemented as a simple audio-playing plugin that had some native components for the necessary interfaces.

Comment 35

12 years ago
Thought: perhaps there is a way to better autodetect and integrate with WMP on initial install so we can work more like IE on Windows (which uses plugins for those sorts of things, but integrates nicely out-of-the-box so it provides a good user experience).

Comment #33 and comment #34 about shipping a plugin to do audio might make sense, and would address the problems people have with <object> and <embed>, as well as handling the "link to a .wav" case that this bug is currently about.

Vlad, can you clarify more about "internal handlers" and "plugin" and such?  Maybe I'm getting my terminology mixed up, but I thought "internal handler" was something like what we do for JPEGs where we just decode it internally.  What bugs would therefore need to be fixed to make this work?  I don't understand how we need an "internal handler" if we are shipping a "plugin".  Also would this mean bug 92073 doesn't need fixing if we go this route?

Comment 36

12 years ago
Does this bug also depend on bug 258012, which requests that Firefox be allowed to be picked as the helper application that views the file?

Comment 37

12 years ago
I don't think it does; although I suppose that would be an alternate way of solving the problem in bug 92073.  I don't see a lot of use cases where it would be good to have for this bug, though, because I think you either want .wav links to open in your helper app or in Firefox, and if you want them to open in Firefox, having to explicitly pick that every time gets old fast.
In the download/helper/etc. dialog, the

  [X] Do this every time

would help here, but I'm not entirely convinced that we want to do this for all simple audio format files.  For a long WAV, not having pause/etc. controls might well be quite frustrating.

It'd be interesting to get jpederlow's notes on what IE does, at the least, to start to think through those cases.  (This bug should probably just be about the functionality for internal play, and the UI twiddles to happen in a Firefox-product bug.)

Comment 39

12 years ago
I'm not going to get to this one anytime in the foreseeable future; dumping back to nobody.
Assignee: pkasting → nobody

Updated

10 years ago
Component: XP Miscellany → General
QA Contact: brendan → general

Updated

10 years ago
Component: General → Video/Audio
QA Contact: general → video.audio

Updated

10 years ago
Status: NEW → ASSIGNED
Depends on: 449315
Fixed, now that HTML5 <audio> is implemented and supports WAV (and Ogg Vorbis).
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
I'm not convinced this is fixed until <audio> also supports AIFF.
You need to log in before you can comment on or make changes to this bug.