336164 - Implement WHATWG Audio spec

Status: RESOLVED FIXED

(Core :: Audio/Video, enhancement)

Description

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

We should implement the WHATWG audio spec, or at least some future incarnation of it (see http://listserver.dreamhost.com/pipermail/whatwg-whatwg.org/2006-April/006260.html for some more comments, also bug 92110).

Filing this in Widget as the closest-fit, since it will need some platform-specific audio playing infrastructure (with more control than nsISound).

Comment 1

[jonathan chetwynd]

Is Mozilla the only browser that doesn't support sound in SVG?
Now that Opera, Safari-webkit and IE with ASV all support sound.
see bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=334920

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #1)
> Is Mozilla the only browser that doesn't support sound in SVG?
> Now that Opera, Safari-webkit and IE with ASV all support sound.
> see bug:
> https://bugzilla.mozilla.org/show_bug.cgi?id=334920

The example given in that bug has nothing to do with SVG; it simply adds an XHTML embed element pointing to a sound file on mouseover.  Under Windows, that sound plays in Firefox, however, as with all other <embed> sounds, there's no control over it.

I hope we'll get around to implementing the WHATWG audio spec as soon as the remaining issues are resolved with it (I also hope that we never implement any sVG-specific sound/audio elements).

Comment 3

[jonathan chetwynd]

Vladimir,

If you had opened the attachement provided in bug #334920 you would see it uses the WHATWG Audio spec, which currently works in Opera9b.

I have attached the same file to this bug.

Comment 4

[jonathan chetwynd]

Attachment: HTML 5 Audio - Opera9b

Comment 5

[jonathan chetwynd]

Vladimir wrote,

"Under Windows, that sound plays in Firefox,"  it doesn't work for me:

Using the <embed> file:
http://bugzilla.opendarwin.org/attachment.cgi?id=8239
with FF1.5.0.3 in Windows'98

 perhaps you could specify your version of windows?

should this perhaps be moved to another bug? this was my purpose in setting up the SVG audio bug, but perhaps the issue is more general? should that bug be reopened in respect of the fact that sound support through embed is evidently patchy?

cheers!

Comment 6

[Sylvain Pasche]

I have started a page on the wiki with some notes on what libraries could be used for this. Feel free to complete or add comments.

Comment 7

[Christian Fredrik Kalager Schaller]

In regards to that wiki page, I updated the license information as GStreamer and OpenAL where mistakenly listed as GPL. I would also say that it is probably a big mistake to try to split audio and video as separate unrelated tasks. The lets first do audio and then add in video later strategy is something many have burned themselves on before. Audio and video sync issues tend to be the most common issue with such an approach.

Comment 8

[Sylvain Pasche]

Thanks for your corrections and advices. I think whether to use GStreamer or not is a very important question to answer before going further on this.

I'm no licence expert, but LGPL code is already used indirectly as in the Gtk example. Would it be possible to ship GStreamer as a shared library as part of the installer? Of course, if other options are possible that would be great.

Do you have some experience with gstreamer on Win32 and OSX? I saw the songbird's people are using vlc on these OS'es, I was wondering what were the reasons about it (I didn't ask yet).

Comment 9

[cajbir (:cajbir)]

I would *love* to be able to use OpenAL, simply because that's what I'm most familiar with, having implemented theora and vorbis players using it before.

There was a discussion on moz.dev.platform about audio libraries recently:

http://groups.google.com/group/mozilla.dev.platform/browse_frm/thread/1ebfc03ee131ca63/6645b4396a2cea75?lnk=gst&q=portaudio&rnum=1#6645b4396a2cea75

There was also a discussion in the video session at the 'all hands' meeting. The consensus at the time was portaudio seemed to be the likely best option due to their license, it being fairly active, and it already being integrated on the zap branch.

I'm working on audio for the <video/> element and portaudio was the path I was intending to take based on that feedback.

Comment 10

[Christian Fredrik Kalager Schaller]

I have not personally had any experience, but the Songbird guys seems happy so far. There are some further developments in that regard making GStreamer even better suited for cross-platform use, but I don't want to announce it before the Songbird guys do so.

I can't help but feel that the use of portaudio comes from a situation where you 
are not aware of the true difficulty of getting the media handling correct will be. Not having the usecase of audio and video sync as the top priority will land you with a **** solution on the level of the flash video player, where audio and video er more out of sync than in sync. Everytime I see something on youtube I wonder how they managed to produce something that bad.

Comment 12

[Christian Fredrik Kalager Schaller]

Since its public know I thought I should mention that not only have Songbird announced that they will be using GStreamer across all platforms now, the GStreamer plugins developed on their behest is also out. Those plugins provide support for outputting audio and video on Windows and MacOSX and they also provide access to native codecs on both platforms.

Comment 13

[cajbir (:cajbir)]

Bug 382267, the implementation of <video>, has been refactored to allow <audio> to share code with it. It includes the start of an <audio> implementation.

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

This is implemented.