1707513 - Create a reftest harness option for using drawSnapshot to capture

Status: RESOLVED FIXED

(Testing :: Reftest, task)

Description

[Matt Woodrow (:mattwoodrow)]

The reftest harness currently uses drawWindow with the USE_WIDGET_LAYERS flag to capture the current contents of the compositor via readback (and thus test what we actually paint).

We don't have much test coverage for non-composited drawing paths (including snapshot drawing though), and this is going be to required for bug 1540737 which will rewrite all that code.

If we add an option to capture using drawSnapshot, we can still get images from the parent process (and correctly handles e10s+fission), but covers the new drawing path with the existing set of reftests.

Comment 1

[Matt Woodrow (:mattwoodrow)]

Attachment: Bug 1707513 - Add 'drawSnapshot' mode to reftest harness, and annotate all current failures. r?tnikkel

Comment 2

[Matt Woodrow (:mattwoodrow)]

Attachment: Bug 1707513 - Add reftest-snapshot task using the 'drawSnapshot' reftest mode. r?ahal

Depends on D114188

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b76d28f3a159
Add 'drawSnapshot' mode to reftest harness, and annotate all current failures. r=tnikkel
https://hg.mozilla.org/integration/autoland/rev/f9bdd1b929f2
Add reftest-snapshot task using the 'drawSnapshot' reftest mode. r=ahal

Comment 4

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/b76d28f3a159
https://hg.mozilla.org/mozilla-central/rev/f9bdd1b929f2

Comment 5

[Alexandru Michis [:malexandru]]

Backed out 2 changesets (Bug 1707513) for causing reftest failures in partial-prerender-expansion-with-resolution-1.html
Backout link: https://hg.mozilla.org/integration/autoland/rev/1296fe99b37f19bb54118d4040ceafa66bac3862
Push with failures, failure log.

Comment 6

[Matt Woodrow (:mattwoodrow)]

Looks like the issue is in here: https://hg.mozilla.org/integration/autoland/rev/b76d28f3a1599a412caf83ff85e9e8e1f2c4e256

That patches changes the reftest harness slightly so that it uses async messages with a reply for requesting a canvas update, instead of a sync message.

Looking at the log for the failing test, it seems that we keep detecting that there's a pending paint and so we wait for the MozAfterPaint event. This repeats for what looks to be ~3000 paints.

I think we're just waiting for the entire 100s animation to complete (log timestamps aren't useful, because the reftest harness dumps the whole test log in one go), and then capturing a final screenshot without the transform applied.

I guess if isMozAfterPaintPending is returning true, then it means we're not running this animation async?

Hiro, any ideas what to do here? Looks like this used to fail intermittently (but not very often) and now with my timing changes it's failing consistently.

Comment 7

[Cosmin Sabou [:CosminS]]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/1296fe99b37f

Comment 8

[Hiroyuki Ikezoe (:hiro)]

Looks like it's a partial predernder issue. We keep detecting jank on the animation, and it's actually wrong. That's causing the pending paints. Presumably it's been accidentally passing for some reasons.

Can you please skip the test on WebRender&&Android and file a bug blocking bug 1656473 to re-enable the test?

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e68e8ca94bbf
Add 'drawSnapshot' mode to reftest harness, and annotate all current failures. r=tnikkel
https://hg.mozilla.org/integration/autoland/rev/d86c0cbb97eb
Add reftest-snapshot task using the 'drawSnapshot' reftest mode. r=ahal

Comment 10

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/e68e8ca94bbf
https://hg.mozilla.org/mozilla-central/rev/d86c0cbb97eb

Comment 11

[Alexandru Ionescu (needinfo me) [:alexandrui]]

(In reply to Cosmin Sabou [:CosminS] from comment #7)

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/1296fe99b37f

== Change summary for alert #30058 (as of Wed, 12 May 2021 13:48:03 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
6%	build times		linux64-shippable	nightly taskcluster-m5d.4xlarge	2,463.00 -> 2,307.70

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=30058