1452979 - Support ImageRendering on canvas (NativeTexture external images)

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, enhancement, P1)

Description

[Kartikaya Gupta (email:[e-mail])]

There's some reftests that use -moz-crisp-edges. These are all failing on windows-qr due to lack of crispness, so presumably this isn't hooked up properly.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

For the record, these are the reftests that I believe to be failing due to lack of -moz-crisp-edges support:

layout/reftests/canvas/reftest.list
  == image-rendering-test.html image-rendering-ref.html
layout/reftests/w3c-css/submitted/images3/reftest.list
  == object-fit-fill-png-001c.html object-fit-fill-png-001-ref.html
  == object-fit-fill-png-002c.html object-fit-fill-png-002-ref.html
  == object-fit-contain-png-001c.html object-fit-contain-png-001-ref.html
  == object-fit-contain-png-002c.html object-fit-contain-png-002-ref.html
  == object-fit-cover-png-001c.html object-fit-cover-png-001-ref.html
  == object-fit-cover-png-002c.html object-fit-cover-png-002-ref.html

Comment 2

[Henrik Winnemöller]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Build ID: 20180709100247

Performing these tests using
gfx.webrender.all true
gfx.webrender.force-angle false
and thereby using OpenGL on Windows show crisp edges on all tests in Comment 1.

I guess the issue might be in ANGLE or some place between ANGLE and WR.

Comment 3

[Henrik Winnemöller]

Attachment: test moz-crisp without canvas, working on WR

Comment 4

[Henrik Winnemöller]

Attachment: test video moz-crisp, not working on WR

Comment 5

[Henrik Winnemöller]

STR:
0. (only macOS) Set gfx.canvas.azure.accelerated to true. 
1. Be on a new profile and set gfx.webrender.all to true.
2. Create a Canvas element with some content that has sharp edges, e.g. a QR code.
3. Set the Canvas' style to image-rendering: -moz-crisp-edges
4. Draw the Canvas.
5. Zoom the Page.

Expected Results:
Content is scaled using a nearest neighbour/pixelated filter algorithm. Edges appear sharp.

Actual Results:
Content is scaled using a linear filter algorithm. Edges appear blurry.


Context:
When using moz-crisp-edges on a background image of the whole body or a div, WR produces correct sharp results (see https://bugzilla.mozilla.org/attachment.cgi?id=9004731). But these reftests mentioned in comment https://bugzilla.mozilla.org/show_bug.cgi?id=1452979#c1 all use canvas to draw their images. I added the moz-crisp-edges property to a small h264 video, which also shows incorrect results (see https://bugzilla.mozilla.org/attachment.cgi?id=9004732).

To exclude any issues with DirectX, I used Renderdoc to verify that the correct filter is set. When looking at the expected sharp result, the filter is set to D3D11_FILTER_MIN_MAG_MIP_POINT. The actual result's filter is D3D11_FILTER_MIN_MAG_LINEAR_MIP_POINT. Thus, DirectX performs correctly.

ANGLE is also not responsible: when the Texture is created for the first time at https://searchfox.org/mozilla-central/rev/55da592d85c2baf8d8818010c41d9738c97013d2/gfx/angle/checkout/src/libANGLE/Context.cpp#2025, changing the SamplerState of the Texture to GL_NEAREST actually changes the result to the expected sharp result. So it looks like WR never updates the filter on the texture after it has been created.

Furthermore, this bug is also reproducible on macOS when setting gfx.canvas.azure.accelerated to true (deprecated since Bug 1468801).

Therefore, WR does not update the image-rendering CSS property on NativeTexture external images on Windows and MacOS(SkiaGL).

Comment 6

[Markus Stange [:mstange]]

https://github.com/servo/webrender/issues/2993 has been filed about this. Thanks for the investigation, Henrik!

Comment 7

[Markus Stange [:mstange]]

try push with Henrik's wip patch:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5aff262aadf05511c1f82a6ff08b0cf300af8934

Comment 8

[Markus Stange [:mstange]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4fc42b3e4e77699bff309491005f76ff1d3572fe

Comment 9

[Henrik Winnemöller]

Attachment: 2-Bug_1452979____Use_ImageRendering_property_to_provide_proper_filtering_on_NativeTexture_external_images__r_nical.diff

Comment 10

[Henrik Winnemöller]

Attachment: 3-Bug_1452979___Update_reftest_expectations__r_nical.diff

Comment 11

[Nicolas Silva [:nical]]

Comment on attachment 9006115 [details] [diff] [review]
2-Bug_1452979____Use_ImageRendering_property_to_provide_proper_filtering_on_NativeTexture_external_images__r_nical.diff

Review of attachment 9006115 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/webrender_bindings/RenderD3D11TextureHostOGL.cpp
@@ +81,5 @@
> +                                     mTextureHandle[1],
> +                                     aRendering);
> +      }
> +    }
> +    

nit: trailing space

::: gfx/webrender_bindings/RenderTextureHost.h
@@ +51,3 @@
>  protected:
>    virtual ~RenderTextureHost();
> +  

nit: trailing space

Comment 12

[Nicolas Silva [:nical]]

Comment on attachment 9006116 [details] [diff] [review]
3-Bug_1452979___Update_reftest_expectations__r_nical.diff

Review of attachment 9006116 [details] [diff] [review]:
-----------------------------------------------------------------

Glorious!

Comment 13

[Henrik Winnemöller]

Attachment: 1-Bug_1452979____Use_ImageRendering_property_to_provide_proper_filtering_on_NativeTexture_external_images__r_nical.diff

Comment 14

[Henrik Winnemöller]

Attachment: 2-Bug_1452979___Update_reftest_expectations__r_nical.diff

Comment 15

[Henrik Winnemöller]

These changes introduce the ability to WR to apply ImageRendering on NativeTexture external images that work for the canvas examples. For video support for ImageRendering, further changes are needed on Gecko. Bug 1488555 has been filed for this.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/65002b9b5bca
Use ImageRendering property to provide proper filtering on NativeTexture external images. r=nical
https://hg.mozilla.org/integration/mozilla-inbound/rev/5d90cbaeceb8
Update reftest expectations. r=nical

Comment 17

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/65002b9b5bca
https://hg.mozilla.org/mozilla-central/rev/5d90cbaeceb8