Specific mp3 file does not decode correctly when using `decodeAudioData`

VERIFIED FIXED in Firefox 25

Status

()

defect
--
critical
VERIFIED FIXED
6 years ago
6 years ago

People

(Reporter: kael, Assigned: rillian)

Tracking

23 Branch
mozilla26
x86_64
Windows 7
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox25 verified, firefox26 fixed)

Details

()

Attachments

(1 attachment, 2 obsolete attachments)

As of Nightly 23.0a1 (2013-04-16) (if not slightly earlier), Firefox now exposes enough of the AudioContext and related Web Audio APIs that it is very hard to tell the difference between them and a working implementation.

Unfortunately, this is still not a working implementation. In particular, Firefox will gleefully tell you yes if you use canPlayType to ask whether mpeg3 is a supported audio codec (presumably because on Windows, Firefox will use native codecs to decode audio/video), but when you try to decode it with the Web Audio API it fails:

[21:19:11.553] "Error while loading 'Audio/hit_wood5': Unknown audio decoding error"
[21:19:11.553] The buffer passed to decodeAudioData contains an unknown content type. @ http://127.0.0.1:81/Lumberjack/Lumberjack.html

This is probably more of an oversight in the design of Web Audio than a problem with Firefox, but the reality is that making Web Audio appear fully supported while not supporting all the same codecs as <audio> will make it incredibly difficult to write applications that gracefully handle various browsers and audio backends.

The only alternative I can think of would be downloading both the MP3 and OGG versions of every sound, and trying to decode one and falling back to the other if it fails. This is unacceptable. If there's an alternative I've missed (perhaps a recent revision to the Web Audio spec? I checked and didn't see anything) then my apologies.
(In reply to Kevin Gadd (:kael) from comment #0)
> As of Nightly 23.0a1 (2013-04-16) (if not slightly earlier), Firefox now
> exposes enough of the AudioContext and related Web Audio APIs that it is
> very hard to tell the difference between them and a working implementation.
> 
> Unfortunately, this is still not a working implementation. In particular,
> Firefox will gleefully tell you yes if you use canPlayType to ask whether
> mpeg3 is a supported audio codec (presumably because on Windows, Firefox
> will use native codecs to decode audio/video), but when you try to decode it
> with the Web Audio API it fails:
> 
> [21:19:11.553] "Error while loading 'Audio/hit_wood5': Unknown audio
> decoding error"
> [21:19:11.553] The buffer passed to decodeAudioData contains an unknown
> content type. @ http://127.0.0.1:81/Lumberjack/Lumberjack.html

That's not true.  We are able to decode any type that canPlayType says is supported.

Looking at your test case, I can get audio playback to work just fine in Nightly...  Can you please create a small test case which shows the problem you're seeing?

> This is probably more of an oversight in the design of Web Audio than a
> problem with Firefox, but the reality is that making Web Audio appear fully
> supported while not supporting all the same codecs as <audio> will make it
> incredibly difficult to write applications that gracefully handle various
> browsers and audio backends.

FWIW, we're using the exact same decoding infrastructure for decodeAudioData that our HTMLMediaElement implementation uses, and all of the same codecs are supported through both APIs.

> The only alternative I can think of would be downloading both the MP3 and
> OGG versions of every sound, and trying to decode one and falling back to
> the other if it fails. This is unacceptable. If there's an alternative I've
> missed (perhaps a recent revision to the Web Audio spec? I checked and
> didn't see anything) then my apologies.

Nope, canPlayType is indeed the right thing to use here.
Blocks: webaudio
https://dl.dropboxusercontent.com/u/1643240/mp3decodetest/test.html


Output (FF23):
canPlayType(audio/mpeg) === maybe
canPlayType(audio/ogg) === maybe
starting buffer load
buffer load failed. error=Unknown audio decoding error

Output (Chrome Canary):
canPlayType(audio/mpeg) === maybe
canPlayType(audio/ogg) === maybe
starting buffer load
buffer load successful. length=2229167 numberOfChannels=2



Contents of about:support:

  Application Basics

        Name
        Firefox

        Version
        23.0a1

        User Agent
        Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20130416 Firefox/23.0

        Build Configuration

          about:buildconfig

  Extensions

        Name

        Version

        Enabled

        ID

        Firebug
        1.11.3b1
        true
        firebug@software.joehewitt.com

        Gecko Profiler
        1.12.0
        true
        jid0-edalmuivkozlouyij0lpdx548bc@jetpack

        JIT Inspector
        0.4.0.0
        true
        bhackett@mozilla.com

  Important Modified Preferences

      Name

      Value

        accessibility.typeaheadfind.flashBar
        0

        browser.cache.disk.capacity
        1048576

        browser.cache.disk.smart_size.first_run
        false

        browser.cache.disk.smart_size_cached_value
        1048576

        browser.places.importBookmarksHTML
        false

        browser.places.smartBookmarksVersion
        4

        browser.startup.homepage_override.buildID
        20130416030901

        browser.startup.homepage_override.mstone
        23.0a1

        browser.tabs.warnOnClose
        false

        dom.max_script_run_time
        0

        dom.mozApps.runUpdate
        false

        dom.mozApps.used
        true

        dom.w3c_touch_events.expose
        false

        extensions.lastAppVersion
        23.0a1

        gfx.direct3d.last_used_feature_level_idx
        0

        gfx.direct3d.prefer_10_1
        true

        javascript.options.mem.notify
        true

        network.cookie.cookieBehavior
        1

        network.cookie.prefsMigrated
        true

        places.database.lastMaintenance
        1366864272

        places.history.enabled
        false

        places.history.expiration.transient_current_max_pages
        104858

        plugin.disable_full_page_plugin_for_types
        application/pdf

        plugin.importedState
        true

        privacy.clearOnShutdown.cookies
        false

        privacy.clearOnShutdown.history
        false

        privacy.clearOnShutdown.sessions
        false

        privacy.sanitize.migrateFx3Prefs
        true

        privacy.sanitize.sanitizeOnShutdown
        true

        security.warn_viewing_mixed
        false

        storage.vacuum.last.index
        1

        storage.vacuum.last.places.sqlite
        1365176844

  Graphics

        Adapter Description
        NVIDIA GeForce GTX 670

        Adapter Drivers
        nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um

        Adapter RAM
        2047

        Device ID
        0x1189

        Direct2D Enabled
        true

        DirectWrite Enabled
        true (6.2.9200.16492)

        Driver Date
        3-14-2013

        Driver Version
        9.18.13.1422

        GPU #2 Active
        false

        GPU Accelerated Windows
        1/1 Direct3D 10

        Vendor ID
        0x10de

        WebGL Renderer
        Google Inc. -- ANGLE (NVIDIA GeForce GTX 670)

        AzureCanvasBackend
        direct2d

        AzureContentBackend
        direct2d

        AzureFallbackCanvasBackend
        cairo

  JavaScript

        Incremental GC
        true

  Accessibility

        Activated
        false

        Prevent Accessibility
        0

  Library Versions

        Expected minimum version

        Version in use

        NSPR
        4.9.6
        4.9.6

        NSS
        3.15 Basic ECCBeta
        3.15 Basic ECCBeta

        NSSSMIME
        3.15 Basic ECCBeta
        3.15 Basic ECCBeta

        NSSSSL
        3.15 Basic ECCBeta
        3.15 Basic ECCBeta

        NSSUTIL
        3.15 Beta
        3.15 Beta
Right, so the problem here is that you are running this on Windows. Our Windows nightly support mp3 decoding, so they return "maybe" when doing canPlayType("audio/mpeg"). This is expected behavior. I'm running this testcase on Linux, and I get an empty string for the mp3 test (and of course it fails to decode, because we don't support mp3 on Linux yet).

However, we should be able to decode mp3 files on Windows using |decodeAudioData|, and this is what this bug is really about. This file plays correctly when using a <audio> tag.
Summary: Web Audio API landing has made it impossible to correctly detect supported audio formats → Specific mp3 file does not decode correctly when using `decodeAudioData`
Well, it fails for every single mp3 file in my larger test case, not just one mp3 file. But they are all produced by the same encoder, so maybe that's the problem? I don't know why they would all play fine in <audio> but not Web Audio if it's using the same decoding backend.
I can probably look into it, to at least get the cause. Feel free to steal if someone has an idea.
ehsan says he tested on OS X, so it's possible this is a bug in the Windows mp3 decoding backend that isn't present in the OS X backend. Need someone else on W7 to test (and perhaps W8, too, since I've been told before that the MP3 decoder is less buggy on W8...)
There is no mp3 decoding backend on OSX at the moment. I have tested on Win7 and it does not work. I have not tested on Win8. However, because we are able to play the file using <audio>, this means we should be able to get the decoded buffer using decodeAudioData.
This is a dupe of bug 862088, and it happens because the MP3 file doesn't have an ID3 tag so we fail to sniff its content type.

A workaround for now would be to use MP3 files with ID3 tags.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 862088
Mass moving Web Audio bugs to the Web Audio component.  Filter on duckityduck.
Component: Video/Audio → Web Audio
The duplicate bug of this bug was marked as fixed, but this problem still exists in the current nightly build (25.0a1 2013-07-06). You can't decode mp3-files without ID3 tag on Windows 7.
Per comment 10, reopening.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Could I have access to one of the files that is not decoding properly when you use webaudio?

My theory is that it is a high-bitrate file, and when we try to sniff for the second mp3 frame using the offset present in the header, it is too far in the bitstream (over 512 bytes, which is a number mandated by the spec), and we fail, because we respect the sniffing spec (chrome/safari does not and successfully sniffs the file media as being mp3, by going past the first 512 bytes). Alternatively, just telling me the bitrate and wether it is VBR/CBR would help characterizing the issue your are experiencing.

We are planning to fix this issue by fixing the spec and then our code (or maybe in the reverse order), this work is going to happen in bug 879429.
Flags: needinfo?(bernhard.robert.pichler)
Hi, sorry for the late reply.

Okay here are two files which don't work:
http://www.stagexl.org/games/escape/sounds/Intro.mp3
http://www.stagexl.org/games/escape/sounds/ChainRotate.mp3

The error message is this:
The buffer passed to decodeAudioData contains an unknown content type.

And here is another file which works fine:
http://www.stagexl.org/games/escape/sounds/ChainBlast.mp3

All the mp3 files are encoded in Windows with the "lame mp3 encoder" from wav files:
FOR /R %1 %%G IN (*.wav) DO lame.exe -V2 "%%G"

All those audio samples and some others are used in this game:
http://www.stagexl.org/games/escape/escape.html

You could run this game in Chrome to hear the sounds effect as desired.
Flags: needinfo?(bernhard.robert.pichler)
Ralph, what happens here is that we don't find the sync pattern at the expected offset when sniffing for mp3 (in the function |is_mp3|), but we should be able to, since the file plays fine. Do you mind having a look?
Flags: needinfo?(giles)
Intro.mp3 is an 'MPEG 2.5' file. I didn't implement length calculation for that variant. Patch or further details pending.
Flags: needinfo?(giles)
Looks like Ralph volunteered in comment 15.  ;-)
Assignee: nobody → giles
Yep. Sorry for not taking the bug earlier.
Posted patch WIP mpeg-2 support (obsolete) — Splinter Review
Posted patch mpeg-2 support (obsolete) — Splinter Review
Attachment #778732 - Attachment is obsolete: true
Attachment #781990 - Flags: review?(paul)
Comment on attachment 781990 [details] [diff] [review]
mpeg-2 support

Review of attachment 781990 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/components/mediasniffer/mp3sniff.c
@@ +40,4 @@
>    header->layer = 4 - ((p[1] & 0x06) >> 1);
>    header->errp = (p[1] & 0x01);
>  
> +  header->bitrate = bitrates[(header->version&1) ? 0:1][(p[2] & 0xf0) >> 4];

Spaces around :, please.

@@ +60,4 @@
>    int size;
>    int scale;
>  
> +  if ((header->version&1) == 0) scale = 72;

Spaces around binary operators, please.
Attachment #781990 - Flags: review?(paul) → review+
Address formatting nits. Carrying forward r=padenot.
Attachment #781990 - Attachment is obsolete: true
Attachment #788255 - Flags: review+
Ralph, can you please nominate this for aurora?  Thanks!
https://hg.mozilla.org/mozilla-central/rev/1f5c5635e875
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
checkin-needed for Aurora, a=webaudio
Status: RESOLVED → REOPENED
Keywords: checkin-needed
Resolution: FIXED → ---
Target Milestone: mozilla26 → ---
checkin-needed for Aurora, a=webaudio
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
Comment on attachment 788255 [details] [diff] [review]
bug865553v4.patch

[Approval Request Comment]

Bug caused by (feature/regressing bug #): 865553
(tagless mp3 files don't work with decodeAudioData)

User impact if declined: Many webaudio demos will not work.

Testing completed (on m-c, etc.): Landed on m-c 3 days ago.

Risk to taking this patch (and alternatives if risky): 

We have pre-approval for webaudio fixes. Asking for approval because this code is also called by the html <audio> element for files served with application/octet-stream to determine playability. I think risk is low: the code changes are simple, and such servers are misconfigured and not common; the primary path to this code is the new webaudio decodeAudioBuffer call.

String or IDL/UUID changes made by this patch: None.
Attachment #788255 - Flags: approval-mozilla-aurora?
Removing checkin-needed pending approval for aurora.
Keywords: checkin-needed
Pushed to try on top of today's Aurora.
Assuming the tbpl goes green, approving.
Attachment #788255 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Whiteboard: [checkin-needed-aurora]
Target Milestone: --- → mozilla26
While testing for the pre-beta sign-off of this feature, I tried to verify this bug, so here are my results, with latest Aurora (build ID: 20130901004005):

1) http://127.0.0.1:81/Lumberjack/Lumberjack.html is no longer available
2) http://hildr.luminance.org/Lumberjack/Lumberjack.html behaves the same with Chrome and Aurora
3) with https://dl.dropboxusercontent.com/u/1643240/mp3decodetest/test.html I get the same results with Aurora and Chrome
4) with http://www.stagexl.org/games/escape/sounds/Intro.mp3 I hear the same sound with Aurora and Chrome
5) with http://www.stagexl.org/games/escape/sounds/ChainRotate.mp3 I hear the same sound with Aurora and Chrome
6) with http://www.stagexl.org/games/escape/sounds/ChainBlast.mp3 I hear the same sound with Aurora and Chrome
QA Contact: manuela.muntean
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.