1875182 - Print WPTs that are rotated with 'page-orientation' end up clipping (i.e. not comparing) the top or bottom portion of the page

Status: NEW

(Testing :: web-platform-tests, defect)

Description

[Daniel Holbert [:dholbert]]

STR:

1. Run the attached WPT test (which is a mismatch test, i.e. testcase and reference case are not expected to render the same)

EXPECTED RESULTS:
The mismatch test should pass (testcase and reference case should not render the same, since the bottom part of the testcase differs from the reference case).

ACTUAL RESULTS:
The mismatch test fails (testcase and reference case do render the same as far as the harness is concerned).

If you capture reftest output with --log-tbpl /tmp/tbpl.txt and view it in reftest analyzer, you can see that the image (the "virtual paper") is portrait-oriented, while the content is overlaid onto it in a landscape-oriented manner, with the extreme end of the content being clipped as a result.

Really, we should probably be rendering to an image ("virtual paper") that is landscape-oriented for page 2 here.

Comment 1

[Daniel Holbert [:dholbert]]

This is essentially a "robustness bug", in that we'll end up not-comparing some portion of the testcase and reference case. In the example WPT that I'll be attaching on this bug as a testcase, this makes the test fail; but in the real world, this could be making tests spuriously pass (if they have page-orientation and have some mismatching content in the part of the test that is clipped, which should result in a test failure but isn't screenshotted/compared due to this bug).

Comment 2

[Daniel Holbert [:dholbert]]

Attachment: testcase 1, as patch: WPT that fails as a result of this bug

Comment 3

[Daniel Holbert [:dholbert]]

Chrome passes the test (by correctly detecting a rendering difference on page 2). If I run the WPT locally with --product chrome to test Chrome, they show:

 0:08.68 TEST_START: /css/css-page/page-orientation-on-portrait-002-print.html
 0:09.38 INFO Image hashes didn't match on page 2, checking pixel differences
 0:09.42 INFO Found 1910 pixels different, maximum difference per channel 255
 0:09.42 TEST_END: PASS

Comment 4

[Daniel Holbert [:dholbert]]

Attachment: reftest screenshot showing the rotation/clipping

If I run the test like so...

./mach wpt --log-tbpl /tmp/tbpl.txt testing/web-platform/tests/css/css-page/page-orientation-on-portrait-002-print.html

...then this is the data URI that ends up getting included in /tmp/tbpl.txt as the screenshot of page 2 for the testcase and reference case (which matches because we're clipping the bottom portion of it which is where the rendering-difference-that-should-induce-a-mismatch happens to live).

Comment 5

[Daniel Holbert [:dholbert]]

For completeness: if I tweak the WPT to use rotate-right instead of rotate-left, then we end up clipping the top of the content rather than the bottom. (The content still runs off of the right side of the reftest screenshot, but it's just the top part of the original content rather than the bottom part that falls on the right side of the image, due to rotate-right rather than rotate-left.)

Comment 6

[Daniel Holbert [:dholbert]]

Comment on attachment 9373398 [details] [diff] [review]
testcase 1, as patch: WPT that fails as a result of this bug

I landed a patch with two WPTs for this in bug 1875186. That makes the first attachment here no longer necessary, since the tests now live in-tree.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jgraham, could you have a look please?

For more information, please visit BugBot documentation.