1409486 - (wr-svg-filter-perf) [meta] [project] Render all SVG filters + CSS filters with WebRender

Status: NEW

(Core :: Graphics: WebRender, defect, P3)

Description

[Markus Stange [:mstange]]

At the moment, any unhandled CSS filters and any SVG filters cause us to go through the fallback path and handle the filters in Gecko.

It would be nice to avoid the fallback and handle the filters in WebRender instead. I've filed https://github.com/servo/webrender/issues/1880 with a proposed API.

Comment 1

[Milan Sreckovic [:milan] (needinfo for best results)]

I'm going to keep the meta out of the MVP, but individual filters can be blockers for MVP.

Comment 2

[Kartikaya Gupta (email:[e-mail])]

It might be good to file individual bugs for the unhandled CSS/SVG filters, or at least list them here. Right now this is just an empty metabug which doesn't seem too useful.

Comment 3

[Markus Stange [:mstange]]

The API in https://github.com/servo/webrender/issues/1880 has a list of all the individual filters that we need to support. We can file individual bugs / WR issues for them once we start going through that list and implement them. In the meantime I think it's still valuable to have a meta bug open for this.

CSS filters are already completely supported. It's just all the SVG filters that are missing.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

The most important filters to implement are probably: flood, composite and blend. Having these will let us use WebRender for filters on quite a few pages.

Comment 5

[Glenn Watson [:gw]]

A rough outline of some work we're considering for H1 this year, to support a wider range of SVG filter graphs without fallback:

Goal

Support a wider range of SVG filter graphs directly in WR, avoiding software rendering fallback paths. Refactor the existing SVG filter support in WR to simplify and optimize currently supported filters.

Current problems

1. When SVG filters were first implemented, the support in render_task_graph for multiple input / output dependencies was very limited. As a result, the SVG filter primitive effectively implements a simple but limited sub-graph itself. We should aim to instead take advantage of render_task_graph functionality to directly handle SVG filter graph dependencies (at least, we should attempt this, and if it's not possible, either extend render_task_graph or document why we're not choosing to do that).

2. Partially due to (1), we currently only support a single local rect for an entire SVG filter graph. This complicates the existing implementation, makes it difficult to add more filters, and is also sub-optimal in a number of cases where we want different sized targets for different parts of an SVG filter graph. Fixing this will require some design work - likely to extend PicturePrimitive or use a different abstraction to represent SVG filter graphs. Some prototyping will be required here.

Suggested steps

1. Identify and document a list of SVG filter cases we would like to support well. This should consider:

   • Existing backlog of real-world bugs with slow fallback rendering performance.
   • Existing backlog of correctness bugs related to SVG filters.
   • Common cases we want to ensure we will handle in an optimal way with whatever changes we make.
   • Complex / edge cases we want to ensure we will be able to handle correctly without too much complexity.

2. Design and prototype an implementation that fixes the problems above and can handle a subset of the identified cases we want to handle. This is likely to be the bulk of the work. I would suggest doing it in parallel with the existing SVG filter implementation, so that it's possible to work without those constraints, and without breaking the existing support. This doesn't need to be production code, just enough to validate the ideas.

3. Once we're confident we can support the cases we want, make the prototype able to handle all the existing cases we do currently support, and switch over to using this in production, removing the previous implementation.

4. Extend the new implementation to handle the wider range of SVG filter graphs that we don't currently support.

Comment 6

[Nicolas Silva [:nical]]

I think that nowadays, the render task graph handles having nodes with multiple inputs as well as having the output of a node be used by multiple nodes as input (which is what you mean by multiple output, but being extra clear that it is not a multiple render taget type of deal).

Important missing parts include:

• Reading a portion of a node's content (right now the input of a node is always the full content of the node it reads from and that's insufficient for SVG filers)
• Because of the way the graph is specified [Edit: in the SVG specification] with rectangles in layout space and mostly inferred, there is a bit of non-trivial logic needed to determine the device space area that needs to be rendered by each node. I think that this is the main challenge.
  • I prototyped that logic here https://github.com/nical/misc/blob/master/svg_filters/src/lib.rs way back (I don't remember how far that went).
  • Note that this effort was really looking at the spec/gecko and implementing the general case without consideration for implementing a simpler subset.
• Then there is missing shaders, but these are rather easy to add once the previous item is in place.
• And then there is the issue of supporting conversion between sRGB and linear in a performant way.
  • Right now it is implement as nodes which is correct but means two extra render passes (99% of the time, the inputs and outpus of the graph are in sRGB, but internally nodes work in linear space).
  • The performance impact of that is pretty strong with SWGL
  • I think that we can have a subset or all filters have inline optionallly convert the input from sRGB to linear and the output from linear to sRGB.

Also note that there is a bunch of code that looks like it is handling svg filters but is in fact dead code (an unfinished intern project).

Comment 7

[Glenn Watson [:gw]]

(In reply to Nicolas Silva [:nical] from comment #6)

I think that nowadays, the render task graph handles having nodes with multiple inputs as well as having the output of a node be used by multiple nodes as input (which is what you mean by multiple output, but being extra clear that it is not a multiple render taget type of deal).

Yes, that's right.

Important missing parts include:

• Reading a portion of a node's content (right now the input of a node is always the full content of the node it reads from and that's insufficient for SVG filers)

I don't think that's true - you can set the UV rect yourself with the with_uv_rect_kind method when creating a render task.

• Because of the way the graph is specified (with rectangles in layout space), there is a bit of non-trivial logic needed to determine the area that needs to be rendered by each node. I think that this is the main challenge.

I don't think this is quite right either - perhaps you're thinking of Pictures? For a render task, the parameter defining it is a device space size, anything else is arbitrary data that depends on the type of the render task itself.

- I prototyped that logic here https://github.com/nical/misc/blob/master/svg_filters/src/lib.rs way back (I don't remember how far that went).
- Note that this effort was really looking at the spec/gecko and implementing the general case without consideration for implementing a simpler subset.

• Then there is missing shaders, but these are rather easy to add once the previous item is in place.
• And then there is the issue of supporting conversion between sRGB and linear in a performant way.
  • Right now it is implement as nodes which is correct but means two extra render passes (99% of the time, the inputs and outpus of the graph are in sRGB, but internally nodes work in linear space).

It's probably reasonable for SVG filter shaders to optionally support that as part of each filter kind, sort of like how we (un)pre-multiply alpha at the start / end of a filter shader as required?

- The performance impact of that is pretty strong with SWGL
- I think that we can have a subset or all filters have inline optionallly convert the input from sRGB to linear and the output from linear to sRGB.

Also note that there is a bunch of code that looks like it is handling svg filters but is in fact dead code (an unfinished intern project).

Is this is Gecko? Could you link to it?

Comment 8

[Nicolas Silva [:nical]]

I don't think that's true - you can set the UV rect yourself with the with_uv_rect_kind method when creating a render task.

Cool!

I don't think this is quite right either - perhaps you're thinking of Pictures? For a render task, the parameter defining it is a device space size, anything else is arbitrary data that depends on the type of the render task itself.

I was referring to the SVG spec here, not the render task graph. SVG filter rects are in layout space and much of it is inferred from the input/output are of the graph. That logic doesn't exist in WR and ideally would be there. See the code in my incomplete prototype that is meant to handle that. The alternative is to have Gecko do it. Back when I investigated it I didn't like the idea of computing the filter device rects on the gecko side but I don't remember if it was because of some limitation or just me wanting to avoid adding C++ code.

Is this is Gecko? Could you link to it?

The supposedly dead code I was referring to was:

• https://searchfox.org/mozilla-central/source/gfx/wr/webrender/res/cs_svg_filter.glsl
• https://searchfox.org/mozilla-central/source/gfx/wr/webrender/src/filterdata.rs
• BUT checking it again it looks like it is actually hooked up. For some reason I convinced myself a while back that these pieces existed but were not used on the gecko display list building side. I'm still not 100% sure to be honest.
  • If the code is indeed hooked up and working, then it's worth investigating what it is missing, When I said during the meeting that trying to build on top of the existing filter code would probably be more friction than a headstart, I was referring to the CSS filter code (which does not do graphs), not the SVG filters which I assumed was in an unknown state.

Comment 9

[Ashley Hale [:ahale]]

Note to self: prior art on feTurbulence I can possibly refactor into a usable implementation: https://gist.github.com/mstange/41e89dba2030e43511ee1784ccc44f56

Comment 10

[Ashley Hale [:ahale]]

Attachment: Test page with all SVG filters

I created a test page that uses every SVG filter effect currently supported by Firefox. Some of these are from SVG 1.1, but also some belong to SVG 1.2+ working drafts for PDF rendering.

Comment 12

[Robert Longson [:longsonr]]

When implementing feTurbulence take care to implement bug 1122761 too.