Can't dynamically change subtitle tracks on a <video> element

RESOLVED INVALID

Status

()

Core
Audio/Video: Playback
P1
normal
RESOLVED INVALID
3 years ago
3 years ago

People

(Reporter: djf, Assigned: gerald)

Tracking

(Blocks: 1 bug)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox44 affected)

Details

Attachments

(5 attachments)

(Reporter)

Description

3 years ago
I'm trying to modify the FirefoxOS Video app to display subtitles when there is a .vtt file on the device sdcard along side the video, and have run into a gecko bug, or maybe two gecko bugs.

1) A <track> element inside a <video> element works if I add the track before playing the video, or if I add the track element even after the video is already playing.  And if I remove the track element, the video element stops displaying subtitles. So far, so good. But now if I add a <track> element back to the <video> element, it does not display any subtitles.  It appears that once a <track> has been removed from a <video> element, that video will never again display captions or subtitles.  The FirefoxOS Video app reuses a single long-lived <video> element for all of its videos. So this bug means that I can display captions for one video. But then if I play a video with no captions (which causes the track to be removed) captions will never again work unti the user kills the app and restarts it.

2) I've tried working around this by leaving the <track> element in the video element, but just removing the src attribute and setting track.default = false.  This appears to have no effect at all... the video just keeps playing and displaying subtitles.

I'll attach files to demonstrate these bugs.  bug.html and bug2.html demonstrate bugs 1 and 2.  They use upc.ogv and upc.vtt as the test movie and captions.

This blocks my work in bug 1213390, so getting it fixed (or finding a workaround) is high-priority for me.
(Reporter)

Comment 1

3 years ago
Created attachment 8672850 [details]
bug.html: test file that demonstrates the first bug
(Reporter)

Comment 2

3 years ago
Created attachment 8672851 [details]
bug2.html: demonstrates the second bug
(Reporter)

Comment 3

3 years ago
Created attachment 8672853 [details]
upc.vtt: captions for the movie
(Reporter)

Comment 4

3 years ago
Created attachment 8672854 [details]
upc.ogv: video used by the test files
(Reporter)

Comment 5

3 years ago
This bug does not appear to be specific to .ogv videos. I've seen it with .mp4 as well.
(Reporter)

Updated

3 years ago
Blocks: 1213390
(Assignee)

Comment 6

3 years ago
Could it be the same issue as experienced in bug 1205644 and/or bug 1206304?
See Also: → bug 1206304, bug 1205644
(Reporter)

Comment 7

3 years ago
I should add that this bug is affecting me on FirefoxOS but that the demos I've attached demonstrate the bug on MacOS running Firefox Developer Edition (gecko 43)
(Reporter)

Comment 8

3 years ago
(In reply to Gerald Squelart [:gerald] from comment #6)
> Could it be the same issue as experienced in bug 1205644 and/or bug 1206304?

I just added a console.log(JSON.stringify(player.textTracks)); to my bug.html file. textTracks is not updated immediately after I add the <track> element, but it does get update asynchronously, so that if I do a setTimeout(function() {console.log(...)}); I do see textTracks having the right number of elements.

So I think that this is not the same issue, unless there is some code in gecko that is also being tripped up by the async textTracks update.
Thanks for the excellent bug report, David. Sounds like a real bug, but no one at moco is working on <track> at the moment. Might have to fix this yourself. :/

Gerald, would you be interested in this one.
Flags: needinfo?(gsquelart)
Priority: -- → P1
(Assignee)

Comment 10

3 years ago
Working on it now.
Assignee: nobody → gsquelart
Flags: needinfo?(gsquelart)
(Assignee)

Comment 11

3 years ago
Created attachment 8674660 [details]
bug-fixed.html bug.html with correct track mode setting

The first "bug" is due to the added tracks' mode staying "disabled".

According to the HTML specs, when the very first track is added to the video element, an algorithm is performed to select the relevant track(s) to be displayed. But after that, whether tracks are added or removed, this algorithm is not run anymore.
Search for "did-perform-automatic-track-selection" in the following page to see how this is specified:
http://www.w3.org/TR/html5/embedded-content-0.html#did-perform-automatic-track-selection

So our implementation actually follows the specs correctly!

As an exercise (and before finding out about the specs), I was able to "fix" the first bug by removing the did-perform-automatic-track-selection check:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6b44e56165a7

Attached is a slightly modified version of bug.html, which sets the track mode to "showing" *after* attaching the track to the video element; this one works fine, and even simultaneously displays all copies of the added tracks!
(Assignee)

Comment 12

3 years ago
As for the 2nd bug:

The specs describe the src attribute: "The [src] value must be a valid non-empty URL potentially surrounded by spaces. This attribute must be present."
http://www.w3.org/TR/html5/embedded-content-0.html#attr-track-src

So if I interpret this correctly, setting src to an empty string, or removing the attribute, are not valid operations.

Clearing the src attribute of the track reverts it to the html page URI, but somehow internally keeps a reference to the VTT cues (I didn't investigate further).

The <video> element probably handles this undefined behavior better: It produces an error, and modifies the video contents.
We could modify <track> to also produce an error and remove the captions (as probably intended by the webpage author), but I would argue that a separate low-priority bug should be created for that.
(Assignee)

Comment 13

3 years ago
David: Based on my analysis above, I'm tempted to close this bug as Invalid, but please let me know if you think I've missed something.

Hopefully my proposed solution in attachment 8674660 [details] should help you continue with bug 1213390 -- or as you've already experimented in the meantime, you may use a different <video> element for each video instead.
Flags: needinfo?(dflanagan)
See Also: bug 1206304, bug 1205644
(Reporter)

Comment 14

3 years ago
Thank you Gerald!  I had not ever read the spec, and didn't even know about track.mode. Sounds like that is my bad (though maybe MDN needs more docs, too.)

I agree that this is probably an invalid bug.
Flags: needinfo?(dflanagan)
(Assignee)

Comment 15

3 years ago
Thank you David.
Indeed the "TextTrack" type (referred to in https://developer.mozilla.org/en-US/docs/Web/API/HTMLTrackElement) is not documented, I'll open a bug for that.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → INVALID
(Assignee)

Updated

3 years ago
Blocks: 1216387
You need to log in before you can comment on or make changes to this bug.