437554 - Implement BackgroundImage/BackgroundAlpha for filters

Status: NEW

(Core :: SVG, enhancement)

Description

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I had a go at implementing BackgroundImage/BackgroundAlpha until I realized it's a lot harder than it looks. My initial approach was to create the background by compositing together all the temporary surfaces created by active PushGroups. But this just doesn't work because those surfaces are clipped by ancestors, including clipping to the viewport, which is incorrect according to the spec. See
http://lists.w3.org/Archives/Public/www-svg/2008Jun/0007.html
I think Opera uses this approach.

I'm not sure of the best way to proceed and it's not a priority for me right now so I'm putting it on ice. One possible approach is to implement the spec verbatim. This seems to be what Batik does. Unfortunately that won't integrate well into our code, especially when applying filters to non-SVG content. Also it will perform extremely poorly, probably O(N^2) per paint where N is the number of filters using BackgroundImage.

Probably a better approach is to change the size of temporary buffers so they're big enough to capture all needed background, and to not apply ancestor clipping while drawing into those buffers. There are two major problems with that:
1) this means we can't use PushGroup, or we'll have to extensively modify it. We may even have to create new gfxContexts for the temporary buffers.
2) determining the size of the temporary buffers is nontrivial. It will require analysis of the scene to figure out what areas are covered by filters with BackgroundImage.

Invalidation is also very hairy for BackgroundImage. You have to invalidate filters that use BackgroundImage that are over other elements that change --- overlapping filters that use blur, for example, require an expansion of the invalid area. This is a new phenomenon and requires some amount of new infrastructure to handle. One approach would be to record the region painted by BackgroundImage-using filters in a document and invalidate that region whenever anything in the document is invalidated.

Comment 1

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Attachment: work in progress

Attaching a work-in-progress patch. Th s patch builds, if you start from the right place in my SVG branch, but doesn't work and corrupts memory. It's also the wrong approach. I probably shouldn't even attach it at all :-). It does add a user-data API to gfxContext though, and also a way to access the stack of pushgroup'ed surfaces, and partially implements the enable-background CSS property.

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #0)
> 1) this means we can't use PushGroup, or we'll have to extensively modify it.
> We may even have to create new gfxContexts for the temporary buffers.

We probably still can, it would just mean clipping to a different (bigger) rectangle before PushGroup, and then clipping to the smaller (actual) one after PopGroupToSource before the Paint happens.

Comment 3

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Yeah, but we have to somehow undo the clipping imposed from outside. Note that we may even have to render outside the viewport.

Comment 6

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Another thought:  if enable-background applies to HTML, it seems like enable-background != accumulate should create a stacking context (just like opacity != 1 and transform != none).

Comment 7

[jonathan chetwynd]

the spec states "This filter composites two objects together"
to state that feblend is fixed when one can composite with self is disingenuous
this and the lack of active progress is one of the reasons I stopped filing mozilla bugs

Comment 8

[Robert Scott]

From what I can tell, the lack of this feature cripples SVG to a single ("normal") blend mode. So I don't really understand why it's not getting more attention.

Comment 10

[Dirk Schulze]

(In reply to David Baron [:dbaron] (busy May 31-June 15) (UTC+2) from comment #6)
> Another thought:  if enable-background applies to HTML, it seems like
> enable-background != accumulate should create a stacking context (just like
> opacity != 1 and transform != none).

The enable-background property is no longer part of Filter Effects. It got replaced by the isolation[1] property which indeed creates a stacking context when set to isolate. The isolation property is part of CSS Compositing and Blending. It should be easier to implement BackgroundImage once isolation is implemented.

[1] http://dev.w3.org/fxtf/compositing-1/#isolation

Comment 11

[Sebastian Zartner [:sebo]]

(In reply to Dirk Schulze from comment #10)
> (In reply to David Baron [:dbaron] (busy May 31-June 15) (UTC+2) from
> comment #6)
> > Another thought:  if enable-background applies to HTML, it seems like
> > enable-background != accumulate should create a stacking context (just like
> > opacity != 1 and transform != none).
> 
> The enable-background property is no longer part of Filter Effects. It got
> replaced by the isolation[1] property which indeed creates a stacking
> context when set to isolate. The isolation property is part of CSS
> Compositing and Blending. It should be easier to implement BackgroundImage
> once isolation is implemented.
> 
> [1] http://dev.w3.org/fxtf/compositing-1/#isolation

The 'isolation' property was implemented in bug 1077872.

Sebastian

Comment 14

[flying sheep]

interestingly, as the two duplicates show, disabling (and optionally re-enabling) the filter CSS rule makes the paths visible.

that is maybe a separate bug…

Comment 15

[Robert Longson [:longsonr]]

Yes it is a separate bug.

Comment 16

[Alecazam]

Using mix-blend-mode isn't the same thing AFAIK, since it has to be set on the element (and not inside the filter).  You can only have one filter on an svg element, and when you have complex filters, you need to be able to mix them against the previous content (at least on the first filter).  mix has the needed set of blend types (12 types), but only feBlend (5 types) can be interspersed into the filter into the correct spots.

Comment 17

[Andres Gomez]

Just adding a reference test, in case it is useful:
https://rawgit.com/tanty/design-hodgepodge/master/svg/filter/feBlend-backgroundimage-tests.html

Notice that, if I'm understanding correctly how the isolation property works, when using a SVG filter we shouldn't worry about what it is behind the SVG document since the filter makes a mandatory isolated group:
https://drafts.fxtf.org/compositing-1/#csscompositingrules_SVG

Comment 18

[Andres Gomez]

Similar issues from other web engines:
https://bugs.webkit.org/show_bug.cgi?id=6022
https://code.google.com/p/chromium/issues/detail?id=137230
https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/10238658-implement-backgroundimage-backgroundalpha-input-so

Comment 22

[Tynach]

Now that the CSS property backdrop-filter is implemented and enabled by default, could the same mechanism used by it be used for implementing this?