1922305 - [META] Make live captured element rendering work

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, task)

Description

[Emilio Cobos Álvarez [:emilio]]

Kinda falls from https://drafts.csswg.org/css-view-transitions-1/#update-pseudo-element-styles:

If capturedElement’s new element is not null, then:

• Let new be the ::view-transition-new() with the view transition name transitionName.
• Set new’s replaced element content to the result of capturing the image of capturedElement’s new element.

In order to implement this efficiently we should probably take advantage of the captured element not being in view (bug 1922304), and try to do something clever with the display list of the element... Maybe.

Comment 1

[Emilio Cobos Álvarez [:emilio]]

So the plan for this is to go for display lists. As per discussion with Markus and Tim::

emilio: heh, that's the part of view transitions that I was most scared of, and it was never clear to me what the right approach in firefox would be
emilio: I agree that it seems better to make it so that all the state can be recovered on the main thread and there's nothing required for "correctness" to be saved on the WR side
offscreen elements are one thing, surviving gpu process crashes is another factor that I can think of
I don't know if I'd call it "easier" to do on the display list side; traditionally, changing anything related to display lists can keep us busy for quite a while
I don't know if I'd call it "easier" to do on the display list side; traditionally, changing anything related to display lists can keep us busy for quite a while
emilio: my recommendation is to try doing it on the display list side and then seeing where it fails
(if RDL is in the way, I would be happy to see the RDL code go away)

Tim thought it might not be too hard in fact:

emilio: you're talking about building the display at the top level during building time? not moving an already built display list later? and this stuff is all stacking contexts? then, like you say about abs pos, we could use something similar to abs pos stuff and that doesn't seem too bad. RDL doesn't seem like it'd be too bad, assuming we have an easy way to make sure everything relevant gets marked as modified and/or disabling RDL for some amount of paints
no placeholder, but we'd prob have a list of active view transitions or something? so we'd always be able to map from the frame to the place we are painting it (always at the top) and back so that should be okay

The basic plan should be something like:

• While building the displaylist for the "new" view transition pseudo-elements, as part of building the view-transition layer (bug 1922300):
  • Get the referenced "live" element if present from the active view transition (that's part of capturing the "new" state, bug 1918402).
  • Build its display list at that point, as descendant of the view transition pseudo.

Part of the issue is, I think that we still need to make that display list behave like an image (e.g. support object-fit etc, and stretch it as needed...). I think it might not be too hard in practice? But might need some extra layout code and such for full correctness.

Or maybe we can build a WebRender image reference or so given a display list, and let WebRender do some of the potentially harder parts. Nical would be the expert here.

Comment 2

[Emilio Cobos Álvarez [:emilio]]

Just chatted with Nical, and it seems like, for the snapshot implementation, we'd either have:

• A stacking context tagged with an image key, which we'd have to somehow tell webrender not to render into the parent stacking context (seems doable).
• An iframe display item tagged with an image key, which would be a bit more flexible because it'd also solve some use cases APZ folks and such had for captures. This would basically implement "DrawSnapshot but on the GPU", which seems potentially very nice.

Either of those options would work for us because they would allow us to reuse nsImageFrame / nsImageRenderer in order to plumb through the image, I believe.