1119299 - Firefox isn't seeing multiple audio tracks in a particular mp4 file

Status: NEW

(Core :: Audio/Video: Playback, defect, P4)

Description

[Boris Zbarsky [:bzbarsky]]

STEPS TO REPRODUCE:

1)  Make sure "media.track.enabled" is turned on.
2)  Load http://www.stuart-pinfold.co.uk/videotest/
3)  Click on the video to start playing it.
4)  Open web console.
5)  Evaluate "video1.audioTracks"
6)  Evaluate "video1.audioTracks[0]"

EXPECTED RESULTS: Step 5 shows a list of length 4.  Step 6 shows an audio track with one of the four languages involved.

ACTUAL RESULTS: Step 5 shows a list of length 1.  Step 6 shows an audio track with language="".

As a result, the links on the page, which are expecting to switch between the different-language tracks in that video, don't work, since they never find a track with the matching language.

In general, it doesn't look to me like the tests for bug 744896 actually test multiple tracks, unless I'm missing something: they always use 0 as the index into the track list, and never check for lengths greater than 1.

Shelly, could you take a look at this, please?

Comment 1

[Stuart Pinfold]

Thanks Boris and Shelly.

Please also take a look at the test case in Internet Explorer. You will see that their baked-in HTML5 player (no plugins being used on the test page) even has a button to switch languages - just like the CC button in Firefox for subtitles. Although I cannot personall verify it, research suggests this was a feature in IE10, released over two years ago (October 2012). It certainly works (and has the button in the player) in the version of IE10 you can download from the MS website right now.

http://www.stuart-pinfold.co.uk/videotest/

Comment 2

[Shelly Lin [:shelly]]

(In reply to Boris Zbarsky [:bz] from comment #0)
> STEPS TO REPRODUCE:
> 
> 1)  Make sure "media.track.enabled" is turned on.
> 2)  Load http://www.stuart-pinfold.co.uk/videotest/
> 3)  Click on the video to start playing it.
> 4)  Open web console.
> 5)  Evaluate "video1.audioTracks"
> 6)  Evaluate "video1.audioTracks[0]"
> 
> EXPECTED RESULTS: Step 5 shows a list of length 4.  Step 6 shows an audio
> track with one of the four languages involved.
> 
> ACTUAL RESULTS: Step 5 shows a list of length 1.  Step 6 shows an audio
> track with language="".
> 
> As a result, the links on the page, which are expecting to switch between
> the different-language tracks in that video, don't work, since they never
> find a track with the matching language.
> 
> In general, it doesn't look to me like the tests for bug 744896 actually
> test multiple tracks, unless I'm missing something: they always use 0 as the
> index into the track list, and never check for lengths greater than 1.
> 
> Shelly, could you take a look at this, please?

Although the track interface is implemented, gecko's media pipeline yet not support multi-track plackback, nor switching between tracks (I doubt that a media with multiple video tracks or audio tracks is even playable). Further more, as for the metadata parsing, only Ogg format is supported (bug 1022524).

Comment 3

[Shelly Lin [:shelly]]

s/plackback/playback

The reason we haven't turn this interface to on by default is because gecko does not support muti-track playback, and I don't know if we are planning on supporting that currently. I've also updated the MDN page[1] since bug 1022524 has closed.

[1] https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement

Thank you very much!

Comment 4

[trzickel]

@stuart: did you get this to work with a different format than your sample file somehow? Audio Tracks are supported according to caniuse.com, yet it never shows more than one, regardless of video codec and container.

Comment 5

[Stuart Pinfold]

(In reply to trzickel from comment #4)

@stuart: did you get this to work with a different format than your sample file somehow? Audio Tracks are supported according to caniuse.com, yet it never shows more than one, regardless of video codec and container.

No I never got it to work 4-5 years ago despite it working flawlessly in out of the box IE and Edge. I didn't get it working in Chrome or Safari either so used a different method. No idea whether it works now.

Stuart

Comment 6

[trzickel]

Amazing. I guess I'll wait another 5 years and see what's changed. Maybe it'll work when I'm 30.

Comment 7

[Boris Zbarsky [:bzbarsky]]

Jan-Ivar, do we have plans to support this?

Comment 8

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

No plans. https://caniuse.com/#search=audioTracks shows low support.

Marking P4 = will review patches.

Comment 9

[pp]

To be honest: this is pathetic. I understand that the mainly english speaking developer world is happy with not supporting multi-language videos, but then: why do you supply the browser in other languages anyway?

Not having this feature is a PITA: It might require to have the same video content in multiple files just to offer different versions that people speaking different languages can choose from. This means additional required space and overhead to create multiple video artifacts from the toolchain.

And enabling "media.track.enabled" in about:config as indicated on https://caniuse.com/#search=audioTracks does not work either.

Comment 10

[protected]

People have been using mpeg-dash and the like to have multiple audio tracks, which probably makes this less urgent. It's heavier on the server side, though (typically requires prior reencoding and splitting), which limits the type of infrastructure you can use.

Comment 11

[Kyra]

Is there a dependency relation between this older report?

https://bugzilla.mozilla.org/show_bug.cgi?id=1057233

and/or this?

https://bugzilla.mozilla.org/show_bug.cgi?id=1169102