1441294 - Visual Artifacts with SVG Filter's feComposite using Operator Arithmetic

Status: NEW

(Core :: Graphics, defect, P3)

Description

[porcupine021]

Attachment: Reduced test case

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36
Firefox for Android

Steps to reproduce:

Visit this HTML page to view a reduced test case of the issue. (also attached)
https://porcupine021.github.io/alphajpg/demos/firefox-bug.html

An SVG filter takes two images and uses <feComposite> to create an image mask. The filter is applied to a <rect>. The <feComposite> is set to use the "arithmetic" operator. 


Actual results:

In Firefox 58, 59 and Firefox for Android, the photo of the falcon is drawn with transparency behind it (the checker pattern shows through). However, around the edges of the falcon and towards the bottom half of the image where there are more semi-transparent pixels, the SVG filter is introducing visual artifacts. The pixels can be various different colors, but should be fairly obvious.


Expected results:

I have tested this on Chrome, Safari, iOS11, Android 6, IE10, IE11, and Edge. The expected result would be that the falcon renders like these other browsers.

Comment 1

[porcupine021]

Attachment: firefox-bug.png

The attached png shows the rendering artifacts in FF versus other browsers.

Comment 2

[porcupine021]

I see the visual artifacts on:

Firefox 58.0.2, Macbook Pro, MacOS 10.12.6, NVIDIA GeForce GT 650M 512 MB

Firefox 58.0.2, Samsung Galaxy S6, Android 7.0

Comment 3

[Emil Pasca [:emilpasca]]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:58.0) Gecko/20100101 Firefox/58.0

I have tested this issue on Mac 10.12 with the latest Firefox release (58.0.2) and the latest Nightly (60.0a1-20180301024724) and managed to reproduce the issue.

After loading the provided URL in the browser, you can observe some visual artifcats around the edges of the falcon and towards the bottom half of the image where there are more semi-transparent pixels.

The issue is not reproducible on Windows 10 x64.

Moving this issue to "Core: SVG" which seems the most appropriate component for this.

Comment 4

[Alice0775 White]

I can reproduce this on Windows10 Nightly60 x64 if HWA disabled.

Regression window:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=024e7dc7a219e3f276bc2245746bbad67a574ea9&tochange=209072396aa5ab5c5a0a28109a980dbbcd884922

Triggered by:
209072396aa5	Mason Chang — Bug 1007702. Enable skia on unaccelerated windows. r=lsalzman

Comment 5

[porcupine021]

I believe these visual artifacts we're seeing may be to due to how <feComposite> is clamping the color channel values. Here is how I've come to that conclusion.

According to Mozilla's documentation of <feComposite> (https://developer.mozilla.org/en-US/docs/Web/SVG/Element/feComposite):

"a component-wise arithmetic operation (with the result clamped between [0..1]) can be applied."
"If the arithmetic operation is chosen, each result pixel is computed using the following formula:"
"result = k1*i1*i2 + k2*i1 + k3*i2 + k4"

In my test cases, I have set the constants to k1=0, k2=1, k3=-1, and k4=0, which yields:
result = i1 - i2

In other words, the filter's resulting image should be a RGBA component-wise subtraction of image2 from image1.

The SVG filter looks like this:
<filter id="myfilter" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse" x="0" y="0" width="768" height="512">
  <feImage xlink:href="./img/debug2-img1.png" result="img1"/>
  <feImage xlink:href="./img/debug2-img2.png" result="img2"/>
  <feComposite in="img1" in2="img2" operator="arithmetic" k1="0" k2="1" k3="-1" k4="0"/>
</filter>

To test this hypothesis, I created a set of simplified test cases. Each test case consists of compositing two images. Each image consists of a solid color.

Test 1:
https://porcupine021.github.io/alphajpg/demos/firefox-test1.html
 128, 128, 128, 1.0 (image 1 RGBA)
 255, 128, 128, 0.5 (image 2 RGBA)
-------------------
-127,   0,   0, 0.5 (pre-clamped result after subtraction)
 0  , 127, 127, 0.5 (Chrome actual result)
 0  , 127, 127, 0.5 (Firefox actual result)

In this test case, it appears that all browsers look for the lowest negative color channel and then add that value to the RGB channels. In this case, 127 is added to each RGB channel producing a dark cyan color rgba(0, 127, 127, 0.5) at 50% opacity. All browsers treat the color channel clamping the same and render identical images.

Test 2:
https://porcupine021.github.io/alphajpg/demos/firefox-test2.html
 128, 255, 255, 1.0 (image 1 RGBA)
 255,   0,   0, 0.5 (image 2 RGBA)
-------------------
-127, 255, 255, 0.5 (pre-clamped result after subtraction)
   0, 255, 255, 0.5 (Chrome result)
   0,   0,   0, 0.5 (Firefox result) Fail

In this test case, I have chosen colors for the source images that make clamping the final color values more difficult. After the feComposite subtraction, the pixels' color values are rgba(-127, 255, 255, 0.5). I believe non-Firefox browsers are looking for the lowest negative color channel (-127) and are adding 127 to each RGB channel. That produces rgba(0, 382, 382, 0.5). The GB channels 382 are too high and a secondary clamping check is made which brings the final color to rgba(0, 255, 255, 0.5), which is a 50% transparent cyan.  Firefox on the otherhand appears to not perform the secondary clamping the same way. Firefox seems to set all RGB channels to 0, which yields a black image with 50% transparency.

Additionally, I have tested each major version of Firefox from 59 down to 48 and the behavior is consistent. Due to the age of this behavior, I have doubts that it is an issue with Firefox's core graphics system. I think it is more probable that it is specific to the implementation of <feComposite> and how it handles color channel clamping. That would explain why it has flown under the radar for so long.

Here is where things start to get very weird though. In Firefox version 47.0.2 and below (I tested down to 40.0), the rendering issue changes slightly. By slowly changing the width of the browser window, Test 2 toggles between two different rendering artifacts. The rectangle will flash from cyan with the wrong opacity to black with the correct opacity. Vertical scrolling can also cause the artifacts as well as changing the page's current Zoom level. Very odd.

I hope this additional info will prove valuable in finding the source of the bug.

Comment 6

[porcupine021]

Attachment: firefox-1px-offset-bug.png

Another quick note about the strange flashing behavior in FF 47 and below. It appears that there is still some flashing artifacts in FF 58 as well. When resizing the browser window's width for the Test 2 case (https://porcupine021.github.io/alphajpg/demos/firefox-test2.html), a red 1px border appears to the right of the image. I have attached a zoomed in version of how it appears. I'm not sure if this is related to the artifacts we are seeing or is a different rendering bug entirely.

Comment 7

[Nicolas Silva [:nical]]

Thanks a lot for the detailed information.

Lee, assigning to you since there is a chance the skia update you are doing might fix this. Feel free to reassign otherwise.

Comment 8

[Lee Salzman [:lsalzman]]

This does not appear to be impacted by or related to the Skia update. Unassigning myself for now to give other people a chance to look into this.

Comment 9

[Jan Bösenberg]

I have another very simple test case that is rather bizarre.

Open the following SVG in Firefox with hardware acceleration disabled:

<svg xmlns="http://www.w3.org/2000/svg" style="width: 1000; height: 1000" viewBox="0 0 1024 1024">
	<defs>
		<filter id="filter1">
			<!-- sets color to opaque red -->
			<feColorMatrix values="0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1" result="r1"/>
			<!-- sets color to opaque black -->
			<feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1"/>
			<!-- combine result of primitives -->
			<feComposite operator="arithmetic" k1="0" k2="-1" k3="1" k4="0.998039156198" in2="r1"/>
		</filter>
	</defs>
	<rect x="100" y="100" width="200" height="200" fill="#000000" filter="url(#filter1)"/>
</svg>

You will see a cyan square. The expected behaviour would be a white square.

Now if you change the k4 parameter in feComposite from 0.998039156198 to 0.998039156199 the square will become white. So a change of 0.000000000001 on one parameter has this much of an effect.

I have created a Codepen that contains both cases: https://codepen.io/anon/pen/aYwQYm

Comment 10

[porcupine021]

Jan,
You're right, that is very bizarre.  

This issue seems to only happen on Windows machines when hardware acceleration is turned off, however on Macs it seems to be present regardless of whether hardware acceleration is enabled or not.

I was able to replicate your test case using these two setups on FF 59.0.1:
1) MacBook Pro (Retina, 15-inch, Mid 2015), MacOS 10.13.3, Intel Iris Pro 1536MB
2) MacBook Pro (15-inch, Mid 2012), MacOS 10.12.6, NVIDIA GeForce GT 650M 512MB, Intel HD Graphics 4000 1536 MB

My first assumption was that this was due to floating point rounding errors because the k4 value is so small. To test that hypothesis I put together my own simple test case based upon yours. You can see it at:

https://porcupine021.github.io/alphajpg/demos/firefox-test5.html

The main difference between our two test cases is: 
1) I have used <feFlood> to generate the images for compositing rather than <feColorMatrix>
2) I have used larger values for k4.

When k4 = 0.998 the test case fails (cyan square). When k4 = 0.999 the test case passes (white square).  A difference of 0.001 is too large for the behavior to be caused by hardware floating point rounding errors.

I then looked at the math using the <feComposite> k constants. Using the formula for <feComposite> in arithmetic mode and these constants, the math looks like the following. Note that I'm expressing color values ranging from 0.0 to 1.0.

k1=0 k2=-1 k3=1 k4=0.998

result = k1*i1*i2 + k2*i1 + k3*i2 + k4
result = 0 + -1*i1 + 1*i2 + 0.998
result = i2 - i1 + 0.998

i1 = rgba(0, 0, 0, 1.0)
i2 = rgba(1.0, 0, 0, 1.0)

r = r2 - r1 + 0.998 = 1.0 - 0 + 0.998 = 0.002
g = g2 - g1 + 0.998 = 0 - 0 + 0.998 = -0.998
b = b2 - b1 + 0.998 = 0 - 0 + 0.998 = -0.998
a = a2 - a1 + 0.998 = 1.0 - 1.0 + 0.998 = -0.998

rgba(0.002, -0.998, -0.998, -0.998) // pre-clamping values

We can see that the numbers for the color channels are outside the allowed color values of 0.0 to 0.1.  This should be true whether k4 is 0.998 OR 0.999 however.

It still smells like this is related to color channel clamping, but your test case does show that the same types of clamping required do not always produce the same results on the screen.

One more note regarding the 1 pixel border described in a previous comment. These failing test cases also show a 1 pixel border on the right of the square. In these test cases it is white with 50% transparency. This one pixel border only appears on my non-retina display. The retina display renders without a right border.

So weird.