1968678 - perma fail ubuntu 24.04 layout/generic/test/ test_page_scroll_overlap.html | test_selection_expanding.html

Status: RESOLVED FIXED

(Core :: Layout, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

when running tests in CI on a newer version of linux (18.04 -> 24.04) we run into some new perma failures.

in this debug log, you can see these failure:

[task 2025-05-26T19:48:14.594Z] 19:48:14     INFO - TEST-PASS | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 100% and 9999 
[task 2025-05-26T19:48:14.594Z] 19:48:14     INFO - TEST-PASS | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging up with pref vals 100% and 9999 
[task 2025-05-26T19:48:14.595Z] 19:48:14     INFO - TEST-PASS | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 200% and 9999 
[task 2025-05-26T19:48:14.596Z] 19:48:14     INFO - TEST-PASS | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging up with pref vals 200% and 9999 
[task 2025-05-26T19:48:14.596Z] 19:48:14     INFO - Buffered messages finished
[task 2025-05-26T19:48:14.597Z] 19:48:14     INFO - TEST-UNEXPECTED-FAIL | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 200% and 1 - got 23, expected 18 epsilon: +/- 3
[task 2025-05-26T19:48:14.597Z] 19:48:14     INFO -     SimpleTest.isfuzzy@SimpleTest/SimpleTest.js:522:14
[task 2025-05-26T19:48:14.597Z] 19:48:14     INFO -     myIs@layout/generic/test/test_page_scroll_overlap.html:59:14
[task 2025-05-26T19:48:14.597Z] 19:48:14     INFO -     testPrefVals@layout/generic/test/test_page_scroll_overlap.html:80:7
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO -     async*doTests@layout/generic/test/test_page_scroll_overlap.html:150:9
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO -     async*@layout/generic/test/test_page_scroll_overlap.html:29:3
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO -     callStackHandler@SimpleTest/SimpleTest.js:285:24
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO -     EventHandlerNonNull*this.addLoadEvent@SimpleTest/SimpleTest.js:314:7
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO -     @SimpleTest/SimpleTest.js:1804:15
[task 2025-05-26T19:48:14.598Z] 19:48:14     INFO - Not taking screenshot here: see the one that was previously logged
[task 2025-05-26T19:48:14.599Z] 19:48:14     INFO - TEST-UNEXPECTED-FAIL | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging up with pref vals 200% and 1 - got 23, expected 18 epsilon: +/- 3
[task 2025-05-26T19:48:14.599Z] 19:48:14     INFO -     SimpleTest.isfuzzy@SimpleTest/SimpleTest.js:522:14
[task 2025-05-26T19:48:14.599Z] 19:48:14     INFO -     myIs@layout/generic/test/test_page_scroll_overlap.html:59:14
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     testPrefVals@layout/generic/test/test_page_scroll_overlap.html:91:7
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     async*doTests@layout/generic/test/test_page_scroll_overlap.html:150:9
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     async*@layout/generic/test/test_page_scroll_overlap.html:29:3
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     callStackHandler@SimpleTest/SimpleTest.js:285:24
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     EventHandlerNonNull*this.addLoadEvent@SimpleTest/SimpleTest.js:314:7
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO -     @SimpleTest/SimpleTest.js:1804:15
[task 2025-05-26T19:48:14.600Z] 19:48:14     INFO - Not taking screenshot here: see the one that was previously logged

[task 2025-05-26T19:49:42.486Z] 19:49:42     INFO - TEST-PASS | layout/generic/test/test_selection_expanding.html | The contents of Custom element contents (checking on parent) are selected (div1-input, all boxes are overflow: visible;): Selected String: "" 
[task 2025-05-26T19:49:42.487Z] 19:49:42     INFO - TEST-PASS | layout/generic/test/test_selection_expanding.html | The contents of iframe are selected (div1-input, all boxes are overflow: visible;): Selected String: "" 
[task 2025-05-26T19:49:42.487Z] 19:49:42     INFO - TEST-PASS | layout/generic/test/test_selection_expanding.html | The contents of input are selected (div1-input, all boxes are overflow: visible;): Selected String: "" 
[task 2025-05-26T19:49:42.489Z] 19:49:42     INFO - TEST-PASS | layout/generic/test/test_selection_expanding.html | The contents of textarea are selected (div1-input, all boxes are overflow: visible;): Selected String: "" 
[task 2025-05-26T19:49:42.489Z] 19:49:42     INFO - Buffered messages finished
[task 2025-05-26T19:49:42.489Z] 19:49:42     INFO - TEST-UNEXPECTED-PASS | layout/generic/test/test_selection_expanding.html | The contents of div1 aren't selected (div1-textarea, all boxes are overflow: visible;): Selected String: "aaaa aaaaaaa aaaaaaa\naaaaaaa aaaaaaa aaaaaaa aaaaaaa
[task 2025-05-26T19:49:42.489Z] 19:49:42     INFO - aaaaaaa
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - bbbbbbb bbbbbbb bbbbbbb
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - bbbbbbb bbbbbbb bbbbbbb
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - bbbbbbb
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - ccccccc tttttt tttttt tttttt" - expected FAIL
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - TEST-INFO | expected FAIL
[task 2025-05-26T19:49:42.490Z] 19:49:42     INFO - TEST-UNEXPECTED-PASS | layout/generic/test/test_selection_expanding.html | The contents of div2 aren't selected (div1-textarea, all boxes are overflow: visible;): Selected String: "aaaa aaaaaaa aaaaaaa\naaaaaaa aaaaaaa aaaaaaa aaaaaaa
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - aaaaaaa
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - bbbbbbb bbbbbbb bbbbbbb
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - bbbbbbb bbbbbbb bbbbbbb
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - bbbbbbb
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - ccccccc tttttt tttttt tttttt" - expected FAIL
[task 2025-05-26T19:49:42.491Z] 19:49:42     INFO - TEST-INFO | expected FAIL

this bug will be for tracking investigations, I will be skipping these tests in a manifest in a larger patch to green stuff up.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:boris, could you have a look please?

For more information, please visit BugBot documentation.

Comment 2

[Boris Chiou [:boris]]

It seems dholbert wrote this test before (and hiro reviewed it), so I cc you just in case.

Comment 3

[Daniel Holbert [:dholbert]]

The test failures here all look like this:

TEST-UNEXPECTED-FAIL | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 200% and 1 - got 23, expected 18 epsilon: +/- 3

We're 5 beyond the expected value here (greater than epsilon of 3)

TEST-UNEXPECTED-FAIL | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 200% and 2 - got 46, expected 36 epsilon: +/- 6

We're 10 beyond the expected value here (greater than epsilon of 6)

TEST-UNEXPECTED-FAIL | layout/generic/test/test_page_scroll_overlap.html | Should get expected overlap, when paging down with pref vals 200% and 10 - got 230, expected 180 epsilon: +/- 30

We're 50 beyond the expected value here (greater than epsilon of 30)

Our epsilon here is based on this constant in the test:
https://searchfox.org/mozilla-central/rev/7c10f97fd575f1c3a6898197a0c73ba88d12c91a/layout/generic/test/test_page_scroll_overlap.html#35

const EPSILON_PER_LINE = 3; // see documentation

Maybe we just need to bump EPSILON_PER_LINE to 5 here? At first glance, that would address these failures. I don't recall the exact significance of the epsilon value though.

Comment 4

[Daniel Holbert [:dholbert]]

Ah right, the EPSILON_PER_LINE thing here is just a guess at how much taller the line is, vs. the font-size (which depends on the font metrics and is not predictable).

Documentation on that here:
https://searchfox.org/mozilla-central/rev/7c10f97fd575f1c3a6898197a0c73ba88d12c91a/layout/generic/test/test_page_scroll_overlap.html#138-149

// Test various line-count pref values (with extremely high percent values,
// which makes the percent pref irrelevant, since we use whichever pref
// produces a smaller amount of overlap). Note that the "lines" here are
// sized using the font metrics of the iframe's root scroll frame, which uses
// the initial values for all of the CSS font properties.  So our lines here
// have whatever line-height gets used for the default font at the default
// 16px font-size.  (Unfortunately we can't force a more predictable font
// like Ahem here; even if we set that set on the root node, it doesn't
// affect the styles on the root scroll frame.)
// So: we set our expectations here by assuming that the lines are a little
// bit taller than the default font-size (16px), with a few pixels of epsilon
// (per line-height) to allow for platform differences.

So right now the test assumes that a line with the default font (at default font-size 16px) will be between 13px and 19px in height (16px +/-3px). And with this new platform's default font, the lines are apparently a bit taller than that (16px +/-5px instead).

So: it's entirely reasonable to just bump EPSILON_PER_LINE to 5 here and remove the skip annotation.

jmaher: could I ask you to try that and see if it helps?

Comment 5

[Julien Cristau [:jcristau] (back August 18)]

That appears to work: https://treeherder.mozilla.org/jobs?repo=try&revision=2c73af2113a3bee34f0b2d3de0bdf421c3a1ee6a

Comment 6

[Daniel Holbert [:dholbert]]

Hooray! Thanks for trying that out. Let's just make that change then.

Comment 7

[Julien Cristau [:jcristau] (back August 18)]

Attachment: Bug 1968678 - adjust EPSILON_PER_LINE to work with ubuntu 24.04's default font. r?dholbert

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/2dc30a9979fb
https://hg.mozilla.org/integration/autoland/rev/e6f37a9a92a4
adjust EPSILON_PER_LINE to work with ubuntu 24.04's default font. r=emilio

Comment 9

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e6f37a9a92a4