1834651 - WPT failure in css-masking/clip-path/animations/two-clip-path-animation-diff-length2.html , related to timing of `takeScreenshot()`

Status: RESOLVED FIXED

(Core :: CSS Transitions and Animations, defect)

Description

[Daniel Holbert [:dholbert]]

This interop-2023-relevant WPT test fails in Firefox:
https://wpt.fyi/results/css/css-masking/clip-path/animations/two-clip-path-animation-diff-length2.html

The screenshot on the WPT.fyi harness shows a small green circle for the testcase, vs. a green square for the reference case.

If I load the test side-by-side in Firefox vs. Chrome, it looks like it plays through the exact same animation. So the issue here must be to do with the fiddliness around how the test is timing its screenshot (which is potentially still a Firefox bug).

Note that we have this test flagged as intermittent, per bug 1789156. So we may occasionally get the rendering that the test expects.
https://searchfox.org/mozilla-central/rev/8e1b221afcdae76284b1439c547b032d1f84d236/testing/web-platform/tests/css/css-masking/clip-path/animations/two-clip-path-animation-diff-length2.html#40-46

I tried adding some logging to the test, to log when we call takeScreenshot();, and in fact the test looks correct (green square) when we perform that logging. But the screenshot still seems to capture a later state. So maybe there's an issue with how we handle takeScreenshot() in WPT tests where it's capturing a later rendering (e.g. due to waiting for pending paints or somesuch)?

Comment 1

[Daniel Holbert [:dholbert]]

I think this is just a test bug. The test is playing two animations, one starting at t=0s and shrinking an element ~impercetibly over 10s, and the other starting at t=1s and doing a much more rapid animation.

The test is assuming that the t=1s animation will not have started yet, by the time that the screenshot is captured (when the animations are reporting themselves as ready). But it doesn't have that guarantee -- particularly in e.g. a debug build or under similar slowdown constraints.

The test should probably be using much larger times here, to avoid the chance for times to be reached inadvertently.

(Side note: while editing the test and running it in Firefox/Chrome locally via ./mach wpt, I'm seeing it sometimes fail locally in Chrome, where the 10-second animation has played through enough to cause the outer edge of the rectangle to be slightly fainter in the testcase as compared to the testcase.)

Comment 2

[Daniel Holbert [:dholbert]]

Attachment: Bug 1834651: Increase time thresholds in WPT two-clip-path-animation-diff-length2.html, to avoid inadvertently tripping over them. r?#style,#layout

This patch addresses two issues that I ran into with this test:

• Sometimes the test fails in Chrome (locally for me at least) with the outer
  edge of the green rect being slightly too faint, due to the first animation
  having advanced a perceptible amount through its 10-second duration.

• Sometimes the test fails in Firefox due to the takeSnapshot() operation
  capturing a snapshot of the rendering after the delayed animation (simply
  because the 1-second delay sometimes ellapses before takeSnapshot does its
  work).

This patch avoids these issues by increasing the times all by an order of
magnitude.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e08a7a0f4ba3
Increase time thresholds in WPT two-clip-path-animation-diff-length2.html, to avoid inadvertently tripping over them.  r=hiro

Comment 4

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/40202 for changes under testing/web-platform/tests

Comment 5

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/e08a7a0f4ba3

Comment 6

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot