865401 - Implement :past and :future pseudo-classes

Status: REOPENED

(Core :: Audio/Video: Playback, enhancement)

Description

[Rick Eyre (:reyre)]

User Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31

Steps to reproduce:

The :future and :past pseudo-elements need to be implemented so that anonymous content created by the WEBVTT parser, relating to captions that are in the immediate past or future of the current caption displaying, can be accessed through it.

See:
http://dev.w3.org/html5/webvtt/#the-past-and-future-pseudo-classes

Comment 1

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Pseudo-classes are pretty easy to implement (unlike pseudo-elements; the distinction is important).

Comment 2

[Boris Zbarsky [:bzbarsky]]

Though it's not clear to me whether these WebVTT nodes are DOM nodes... perhaps whether they are or not is an implementation detail, in which case we can just make them so and then I agree: pseudo-classes are easy.

Comment 3

[Rick Eyre (:reyre)]

As far as I can tell they should be. See: http://dev.w3.org/html5/webvtt/#webvtt-cue-text-dom-construction-rules.

Comment 4

[Blake Wu [:bwu][:blakewu]]

This should be related to layout.

Comment 5

[Benjamin Chen [:bechen]]

From email by :heycam :

To support these I think we just need to:

(a) add two #defines NS_EVENT_STATE_* entries in
dom/events/EventStates.h for past and future
(b) add entries in nsCSSPseudoClassList.h using CSS_STATE_PSEUDO_CLASS
that references these two NS_EVENT_STATE_* values
(c) when you need to update which nodes in your DOM tree match these
pseudo-classes, you need to call the C++ functions
Element::AddStates(NS_EVENT_STATE_*) and
Element::RemoveStates(NS_EVENT_STATE_*) to set and clear these state
bits, but since you're in JS you might want to add a function to
nsIDOMWindowUtils that calls into Element::AddStates/RemoveStates

Comment 6

[Boris Zbarsky [:bzbarsky]]

AddStates/RemoveStates are for use from the ESM only.  That's why they're protected and have assertions about the set of states passed to them.

That said, we could do some renaming (e.g. ESM_MANAGED_STATES to EXTERNALLY_MANAGED_STATES) and allow whatever API we're talking about here to call Add/RemoveStates.

If we do that, you will be taking over management of these states completely, including for cases like node removal from the document, adoption into another document, etc (assuming that can happen for the relevant nodes).

That all said, depending on how many of these nodes you have and whether the perf/memory overhead is acceptable, you may be able to use inIDOMUtils.addPseudoClassLock to do this.

Comment 7

[Cameron McCormack (:heycam)]

(In reply to Boris Zbarsky [:bz] (still a bit busy) from comment #6)
> AddStates/RemoveStates are for use from the ESM only.  That's why they're
> protected and have assertions about the set of states passed to them.
> 
> That said, we could do some renaming (e.g. ESM_MANAGED_STATES to
> EXTERNALLY_MANAGED_STATES) and allow whatever API we're talking about here
> to call Add/RemoveStates.

Ah, indeed, I missed that we'd need to add them to ESM_MANAGED_STATES.

> If we do that, you will be taking over management of these states
> completely, including for cases like node removal from the document,
> adoption into another document, etc (assuming that can happen for the
> relevant nodes).

These are NAC nodes that will be controlled entirely from within webvtt.jsm, so I don't think we need to handle such cases.

> That all said, depending on how many of these nodes you have and whether the
> perf/memory overhead is acceptable, you may be able to use
> inIDOMUtils.addPseudoClassLock to do this.

Hmm, my initial thought was to suggest this, but then I realised that this would preclude devtools locking these state bits.  (Which may or not be something they'd want to do.)

Comment 8

[Boris Zbarsky [:bzbarsky]]

Ah, good point.  If devtools can examine these nodes (can they?) then I agree we shouldn't stomp on their API.

Comment 9

[Cameron McCormack (:heycam)]

I don't know the answer to that, but it seems plausible that it would be useful for developers producing WebVTT files to be able to inspect the stylable captions that we show.

Comment 10

[Alastor Wu [:alwu]]

FYI, Chrome supports :past and :future, here is the test case, 
https://people-mozilla.org/~alwu/WebVTT/vttTest.html

Comment 11

[Emilio Cobos Álvarez [:emilio]]

Moving to the media component. The complexity is there, the CSS bits are trivial. Alastor, do you have a sense of how easy / hard would it be to plumb these? Do we even support WebVTT inline stylesheets? I think we don't?

Comment 12

[Alastor Wu [:alwu]]

I might be able to find sometime this year to work on this, assign to myself to do a quick investigation for the current status/plan later. Keep my NI.

Comment 13

[Alastor Wu [:alwu]]

(In reply to Emilio Cobos Álvarez [:emilio] from comment #11)

Do we even support WebVTT inline stylesheets?

Supported today:

• bare ::cue from a document stylesheet, for the WebVTT-allowed properties: color, background, font, text-shadow...e.t.c.

Not supported:

• functional ::cue(<selector>)
• in-file STYLE blocks
• ::cue-region
• :past / :future

So basically only the bare ::cue-from-document case works. This matches the status I noted in bug 865395 comment 21. Nothing has changed since.

---

Do you have a sense of how easy / hard would it be to plumb these?

The CSS keyword bits are trivial as you say, but there's a prerequisite and then the WebVTT work.

1. Selector support

:past/:future are always authored as ::cue(:past) (they're general time-dimensional pseudo-classes, Selectors L5, but cue words are anonymous content, so ::cue() is the only way author CSS can reach them; a bare :past {} is meaningless for captions).

We don't parse the functional ::cue(<selector>) form at all today, so we first need at least a minimal ::cue() parser that accepts a pseudo-class argument. We can keep it minimal because the property set inside ::cue(:past) is the same as plain ::cue, so we skip the selector-dependent property-restriction machinery (the hard/weird part of full ::cue(<selector>)).

2. WebVTT Plumbing

In the WebVTT side, the direction is: make the inline timestamps stylable and flip a past/future state on the surrounding text as playback crosses each timestamp. WebKit does essentially this (markFutureAndPastNodes()/updateDisplayTree()).

3. Testing.

The WPT already exist, so this part is easy.

Conclusion

I'd put :past/:future (minimal ::cue() prerequisite + the WebVTT work) at roughly 1-2 months.

However, filling the whole gap is a lot bigger: full functional ::cue(<selector>) with its per-selector property restrictions, STYLE-block styling, and ::cue-region. That is what bug 865395 is tracking.