Closed Bug 1552194 Opened 1 year ago Closed 1 year ago

Audio Mime Type detection differs on M4A-files depending on call by Javascript audio object or direct link

Categories

(Core :: Audio/Video, defect, P2)

66 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla69
Tracking Status
firefox69 --- verified

People

(Reporter: mmoreno, Assigned: jya)

Details

Attachments

(2 files)

Apologies if cloning this bug is inappropriate, but I could not figure out how to reopen the original bug report. As I mentioned in a comment on the original bug, I don't understand why this issue was closed. Is this not indeed a bug?

The attached screenshot shows how a mismatch between the audio's incorrect mimetype and the Content-Type from its response header. Why is the Content-Type not used? Even if the audio element is loaded via JS, it still needs to connect to the server and should be able to use the Content-Type that is provided. There should be no need to resort to sniffing the file. And even if all efforts fail, it certainly should not simply default to the first mimetype in a list (which is what it appears to do by assigning '3gpp').

If this is not a bug, can you explain why not and whether there is a workaround?

The problem continues to exist with Firefox 66.0.5 on both Mac and Linux Mint.

+++ This bug was initially created as a clone of Bug #1362358 +++

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36

Steps to reproduce:

I am having trouble to play m4a audio files correctly. I set up a test page to show the issue and the steps to reproduce it:

http://123-bb.de/tests/audiotest/index.html

Actual results:

I have 2 different m4a files tested. One where the server sent the mime type "audio/mp4" and one where the mime type was "audio/mpeg".

For the "audio/mp4" file the sound was played correctly when called directly without a javascript audio object.
But when called from a javascript audio object ist wasn' played because Firefox detected the mime type "audio/3gpp" which ist not supported.

For the second file with mime type "audio/mpeg" it was the opposite. It was only played from within a javascript audio object where Firefox detected the mime type "audio/mp4" although the file was send as "audio/mpeg". A direct call without javascript object was not played due to wrong mime type "audio/mpeg".

Expected results:

Firefox should detect the same mime type no matter if a m4a-file is called directly or via a Javascript audio object.

This is real weird:

When I click on the step 3 after loading, nothing happens. However when I clock on step 4, play the file, then go back and click on step 3, it now works.

Step 7 never works.

This is real fishy. Jean-Yves, can you have another look? It seems like a cache bug.

Flags: needinfo?(jyavenard)

(In reply to Marco Moreno from comment #0)

Apologies if cloning this bug is inappropriate, but I could not figure out how to reopen the original bug report. As I mentioned in a comment on the original bug, I don't understand why this issue was closed. Is this not indeed a bug?

The attached screenshot shows how a mismatch between the audio's incorrect mimetype and the Content-Type from its response header. Why is the Content-Type not used? Even if the audio element is loaded via JS, it still needs to connect to the server and should be able to use the Content-Type that is provided. There should be no need to resort to sniffing the file. And even if all efforts fail, it certainly should not simply default to the first mimetype in a list (which is what it appears to do by assigning '3gpp').

Content-Type must be ignored per spec. I mentioned this in comment 6.
There was a long discussion between the various user-agent maker (Microsoft, Google, Apple, Mozilla) and they came to the conclusion that the content type of a file must always be determined via sniffing.

As described in bug 1362358 comment 6; that 10.m4a file binary looks like this:
[ftyp] size=8+16
major_brand = 3gp4
minor_version = 0
compatible_brand = isom
compatible_brand = 3gp4

This tells us that the file type is 3gp4 which we can't play. This is why step 3 gives you nothing.

If this is not a bug, can you explain why not and whether there is a workaround?

the work around is to remux your file so that they have the proper binary format.
Something like this will do:
ffmpeg -i 10.m4a -c:a copy -c:v copy 10-new.m4a

Your file will now always play as you expect:
see https://jyavenard.github.io/htmltests/tests/1552194/1552194.html

The original file in the page above is named: 10-3gpp.m4a
you can test with 10.mp4 and 10.m4a which are the remuxed file.

For the second file with mime type "audio/mpeg" it was the opposite. It was only played from within a javascript audio object where Firefox detected the mime type "audio/mp4" although the file was send as "audio/mpeg". A direct call without javascript object was not played due to wrong mime type "audio/mpeg".

The type audio/mpeg is wrong though, that tells that it's MP3. audio/mp4 is the proper mimetype as your file is aac

Firefox should detect the same mime type no matter if a m4a-file is called directly or via a Javascript audio object.

For you to get consistent result regardless of how the file is loaded, it must be valid.

Now we could add 3gp4 as supported mimetype, however typically a 3gp4 file is not aac, it's AMR which we do not support. But you would get a decoding error there instead.

At a guess when the file is in the cache, as it's been played before the resource type got stored too and filled with the right value and not the incorrect one.

I'll do some experimentations on what happens if we accept 3gp4 in a ftyp

Flags: needinfo?(jyavenard)
Assignee: nobody → jyavenard

3gp4 is based on mp4. Should the codecs not be supported we will simply error later.

Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Priority: -- → P2
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/35d351f93ba0
Consider ftyp 3gp4 as being mp4. r=padenot

(In reply to Jean-Yves Avenard [:jya] from comment #2)

(In reply to Marco Moreno from comment #0)

Content-Type must be ignored per spec. I mentioned this in comment 6.
There was a long discussion between the various user-agent maker (Microsoft, Google, Apple, Mozilla) and they came to the conclusion that the content type of a file must always be determined via sniffing.

Ahh...I misunderstood. I thought you had to sniff the file because the content type was unavailable. I didn't realize that it was to be ignored. I presume you're saying that you can use the content type if loading the file directly, but not via JS - due to security concerns?

If this is not a bug, can you explain why not and whether there is a workaround?

the work around is to remux your file so that they have the proper binary format.
Something like this will do:
ffmpeg -i 10.m4a -c:a copy -c:v copy 10-new.m4a

Your file will now always play as you expect:
see https://jyavenard.github.io/htmltests/tests/1552194/1552194.html

Ok, I now understand that there is a subtle difference in many of my m4a files. FYI, it appears that older versions of neroAacEnc muxed m4a files as 3gpp, but the latest version (still old) muxes it as mp4. So many of my newer m4a files play properly via JS, but my older files do not.

For the second file with mime type "audio/mpeg" it was the opposite. It was only played from within a javascript audio object where Firefox detected the mime type "audio/mp4" although the file was send as "audio/mpeg". A direct call without javascript object was not played due to wrong mime type "audio/mpeg".

The type audio/mpeg is wrong though, that tells that it's MP3. audio/mp4 is the proper mimetype as your file is aac

This appears to be a common bug in /etc/mime.types that incorrectly specifies audio/mpeg instead of audio/mp4.

Now we could add 3gp4 as supported mimetype, however typically a 3gp4 file is not aac, it's AMR which we do not support. But you would get a decoding error there instead.

I'll do some experimentations on what happens if we accept 3gp4 in a ftyp

It would be great if this could be made to work! Thanks for all your help in this!

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
Flags: qe-verify+

Confirmed with 68.0.
Verified with 69.0b15 on Windows 10, MacOS 10.13, Ubuntu 18.04.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.