1555197 - [webvtt] reuse cues' display state when the amount of displaying cue changes

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Alastor Wu [:alwu]]

During checking track-webvtt-two-cue-layout-after-first-end.html, I found that we would recompute cues' display state when the amount of displaying cue changes, which was implemented in bug1551385.

However, when the amount of displaying cue changes, if display cues have already had the display state, which are based on the amount of displaying cue we had last time, we should reuse them to keep them on the previous position.

If we recompute their display state, they would be different from the one we had last time.

EX. Cue0 [0:2], Cue1 [1:5]

  [playing at 1s]  
*-------------------*
|        Cue1       |                  
|        Cue0       |   
*-------------------*

  [playing at 3s]  (cue0 disappears, cue1 stays in the same position)
*-------------------*
|        Cue1       |               
|                   |      
*-------------------*

Comment 1

[Alastor Wu [:alwu]]

Attachment: Bug 1555197 - part1 : introduce new state to distinguish different situations for showing cues' display state.

Cue's display state is a DIV element with corresponding CSS style to display cue on the screen. When the cue is being displayed first time, we will compute its display state.

After that, we could reuse its state until following conditions happen.
(1) control changes : it means the rendering area changes so we should recompute cues' position.
(2) cue's hasBeenReset flag is true : it means cues' line or position property has been modified, we also need to recompute cues' position.
(3) the amount of showing cues changes : it means some cue would disappear but other cues should stay at the same place without recomputing, so we can resume their display state.

The new state includes REUSE, REUSE AND CLEAR and COMPUTE AND CLEAR, our current behavior doesn't handle `REUSE AND CLEAR, which would be implemented in next patch.

Comment 2

[Alastor Wu [:alwu]]

Attachment: Bug 1555197 - part2 : handle state `REUSE AND CLEAR`.

According to the spec [1] step 13, if we won't reset cues' display state, we should reuse them.

If the amount of displaying cues changes, which means some cues should disappear from the screen and some of them should still be showed, we have to clear the screen and resume displaying cues' display state.

As the amount of displaying cues might affect the computed result of cues' display state, if we recompute cues' display state in this situaion, the result would be different from the one we had last time.

For example, there are two cues, Cue0 [0:2], Cue1 [1:5]. When Cue0 disappears, Cue1 should stay in the same position.

  [playing at 1s]
*-------------------*
|        Cue1       |
|        Cue0       |
*-------------------*

  [playing at 3s] (correct)
*-------------------*
|        Cue1       |
|                   |
*-------------------*

  [playing at 3s] (incorrect)
*-------------------*
|                   |
|        Cue1       |
*-------------------*

[1] https://w3c.github.io/webvtt/#rules-for-updating-the-display-of-webvtt-text-tracks

Comment 3

[Alastor Wu [:alwu]]

Attachment: Bug 1555197 - part3 : modify log.

Add and modify some log content in order to help debugging.

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1555197 - part4 : modify wpt 'track-webvtt-two-cue-layout-after-first-end.html' and 'track-webvtt-two-cue-layout-after-first-end-ref.html'.

For track-webvtt-two-cue-layout-after-first-end.html

(1) modify cue1's start time

According to the text track cue order [1], "cues must be sorted by their start time, earliest first; then, any cues with the same start time must be sorted by their end time, latest first".

This order also decides which cue we would display first. As this test would like show cue1 at top and show another cue at bottom, we should modify cue1's start in order to put it before cue [0:3].

[1] https://html.spec.whatwg.org/multipage/media.html#text-track-cue-order

(2) listen for cue1's exit

As this cue would like to stop when the first cue ends, it should listen exit event.

---

For track-webvtt-two-cue-layout-after-first-end-ref.html, we should call video.play() in order to clear show-poster flag [2] and run TimeMarchesOn to show the cue.

If we didn't call it to reset the flag, we won't display any cue.

[2] https://html.spec.whatwg.org/multipage/media.html#playing-the-media-resource:show-poster-flag

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1555197 - part5 : enable wpt 'track-webvtt-two-cue-layout-after-first-end.html'.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2fd0da098622
part1 : introduce new state to distinguish different situations for showing cues' display state. r=heycam
https://hg.mozilla.org/integration/autoland/rev/c94e1169e00e
part2 : handle state `REUSE AND CLEAR`. r=heycam
https://hg.mozilla.org/integration/autoland/rev/4a93da5a440d
part3 : modify log. r=jya
https://hg.mozilla.org/integration/autoland/rev/323c42c5c377
part4 : modify wpt 'track-webvtt-two-cue-layout-after-first-end.html' and 'track-webvtt-two-cue-layout-after-first-end-ref.html'. r=jya
https://hg.mozilla.org/integration/autoland/rev/b8809dc8740c
part5 : enable wpt 'track-webvtt-two-cue-layout-after-first-end.html'. r=jya

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/17318 for changes under testing/web-platform/tests

Comment 8

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Comment 9

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/2fd0da098622
https://hg.mozilla.org/mozilla-central/rev/c94e1169e00e
https://hg.mozilla.org/mozilla-central/rev/4a93da5a440d
https://hg.mozilla.org/mozilla-central/rev/323c42c5c377
https://hg.mozilla.org/mozilla-central/rev/b8809dc8740c

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged