Open Bug 1862592 Opened 11 months ago Updated 15 hours ago

Intermittent TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent scro

Categories

(Core :: Layout, defect, P5)

defect

Tracking

()

REOPENED
Tracking Status
firefox-esr115 --- unaffected
firefox119 --- unaffected
firefox120 --- unaffected
firefox121 --- wontfix
firefox122 --- wontfix

People

(Reporter: intermittent-bug-filer, Assigned: botond)

References

(Regression)

Details

(Keywords: intermittent-failure, intermittent-testcase, regression)

Attachments

(1 file, 1 obsolete file)

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=434652293&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/fobJWBfDR-OzbRbRWXjl-A/runs/0/artifacts/public/logs/live_backing.log


[task 2023-11-02T07:14:07.845Z] 07:14:07     INFO - TEST-START | /css/css-overscroll-behavior/overscroll-behavior.html
[task 2023-11-02T07:14:07.866Z] 07:14:07     INFO - Closing window 4e5d5879-8f7f-48a8-902a-ea3b4c9e432b
[task 2023-11-02T07:14:07.962Z] 07:14:07     INFO - {'actions': [{'type': 'none', 'actions': [{'type': 'pause', 'duration': 16}], 'id': '0'}, {'type': 'wheel', 'actions': [{'type': 'scroll', 'x': 200, 'y': 500, 'deltaX': 0, 'deltaY': -200, 'origin': 'viewport'}], 'id': '1'}]}
[task 2023-11-02T07:14:08.328Z] 07:14:08     INFO - {'actions': [{'type': 'none', 'actions': [{'type': 'pause', 'duration': 16}], 'id': '2'}, {'type': 'wheel', 'actions': [{'type': 'scroll', 'x': 200, 'y': 500, 'deltaX': -200, 'deltaY': 0, 'origin': 'viewport'}], 'id': '3'}]}
[task 2023-11-02T07:14:08.883Z] 07:14:08     INFO - 
[task 2023-11-02T07:14:08.883Z] 07:14:08     INFO - TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent scroll propagation on y axis. expected 0 but got 100
[task 2023-11-02T07:14:08.883Z] 07:14:08     INFO - verify_y_prevented_and_set_boundary_prevents_x/<@http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:106:18
[task 2023-11-02T07:14:08.884Z] 07:14:08     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2599:25
[task 2023-11-02T07:14:08.884Z] 07:14:08     INFO - verify_y_prevented_and_set_boundary_prevents_x@http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:104:8
[task 2023-11-02T07:14:08.884Z] 07:14:08     INFO - @http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:169:42
[task 2023-11-02T07:14:08.885Z] 07:14:08     INFO - TEST-OK | /css/css-overscroll-behavior/overscroll-behavior.html | took 1038ms
[task 2023-11-02T07:14:08.885Z] 07:14:08     INFO - Restarting browser for new test group
[task 2023-11-02T07:14:08.887Z] 07:14:08     INFO - PID 6056 | 1698909248883	Marionette	INFO	Stopped listening on port 55843
[task 2023-11-02T07:14:09.475Z] 07:14:09     INFO - PID 6056 | console.error: ({})
[task 2023-11-02T07:14:09.850Z] 07:14:09     INFO - Browser exited with return code 0
[task 2023-11-02T07:14:09.853Z] 07:14:09     INFO - Closing logging queue
[task 2023-11-02T07:14:09.853Z] 07:14:09     INFO - queue closed
[task 2023-11-02T07:14:09.896Z] 07:14:09     INFO - Application command: Z:\task_169890840657465\build\application\firefox\firefox.exe -marionette about:blank --wait-for-browser -profile C:\Users\task_169890840657465\AppData\Local\Temp\tmpoy6d5zc7
[task 2023-11-02T07:14:09.901Z] 07:14:09     INFO - PID 6000 | 1698909246474	Marionette	INFO	Marionette enabled
[task 2023-11-02T07:14:09.902Z] 07:14:09     INFO - PID 6000 | 1698909246541	Marionette	INFO	Listening on port 55857
[task 2023-11-02T07:14:09.902Z] 07:14:09     INFO - PID 6000 | JavaScript error: resource://gre/modules/XULStore.sys.mjs, line 60: Error: Can't find profile directory.
[task 2023-11-02T07:14:09.903Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.904Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.904Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.905Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.906Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.907Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.907Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.909Z] 07:14:09     INFO - PID 6000 | console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 399))
[task 2023-11-02T07:14:09.909Z] 07:14:09     INFO - Starting runner
[task 2023-11-02T07:14:11.035Z] 07:14:11     INFO - TEST-START | /css/css-position/parsing/bottom-computed.html

Set release status flags based on info from the regressing bug 1857130

:emilio, since you are the author of the regressor, bug 1857130, could you take a look?

For more information, please visit BugBot documentation.

I'm confused, my patch only added a test in the same directory. Hiro, Botond, any idea what might be going on? Might be something to do with the test.events.async.enabled:true pref?

Flags: needinfo?(hikezoe.birchill)
Flags: needinfo?(emilio)
Flags: needinfo?(botond)
See Also: → 1862496

It looks like overscroll-behavior-root.html runs before overscroll-behavior.html, so maybe some effect of overscroll-behavior-root.html "leaks" and affects the run of overscroll-behavior.html?

As you say, this might indicate an issue with test.events.async.enabled (and may be why we saw so many failures when we tried to turn test.events.async.enabled on across the whole test suite in bug 1852243).

In our APZ mochitests, we do a number of things to prevent tests from interfering with each other, such as running each one in a fresh tab, and calling utilities like waitUntilApzStable during test setup which does things like wait for any paints that might be pending from previous activity to be done.

Not sure how these strategies carry over to the WPT world...

Flags: needinfo?(botond)

(In reply to Botond Ballo [:botond] from comment #3)

In our APZ mochitests, we do a number of things to prevent tests from interfering with each other, such as running each one in a fresh tab, and calling utilities like waitUntilApzStable during test setup which does things like wait for any paints that might be pending from previous activity to be done.

Not sure how these strategies carry over to the WPT world...

James, do you know what would be the right place to do this?

Flags: needinfo?(james)

I checked all failures there (as of now). Every failure happened on the first test case, so I am ~100% sure that in the failure case at that moment APZ wasn't yet ready. overscroll-behavior.html has been flaky on Linux (with opt builds), so to me that's a good sign the test has started failing on other platforms because there should be no platform differences in the test (other than Android).

Also I am assuming the new test, overscroll-behavior-root.html doesn't fail even if APZ isn't ready, it will fail if APZ is ready and if our implementation were wrong.

That said, this is kinda annoying thing, as Botond told, we need waitUntilApzStable.

Flags: needinfo?(hikezoe.birchill)

Okay, now I am pretty sure we need to fix bug 1845646. I can easily see the failure locally with running css-overscroll-behavior/ directory, and I can no longer see the failure with the (not-right) workaround I commented in bug 1841305 comment 6.

Depends on: 1845646

In our APZ mochitests, we do a number of things to prevent tests from interfering with each other, such as running each one in a fresh tab,

We already do that.

And calling utilities like waitUntilApzStable during test setup which does things like wait for any paints that might be pending from previous activity to be done.

We can put code in https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py#870-874 to run before testharness tests (although that should probably be refactored so the setup code is passed in from the specific test class, rather than being in the low-level class, since it currently means we also run that code for print reftests (but not other reftests).

Flags: needinfo?(james)
Duplicate of this bug: 1837914

Update

There have been 33 total failures within the last 7 days:

  • 10 failures on Windows 11 x64 22H2 WebRender Shippable opt
  • 6 failures on Windows 11 x64 22H2 WebRender opt/debug
  • 1 failure on Windows 11 x64 22H2 CCov WebRender opt
  • 10 failures on Windows 11 x86 22H2 WebRender Shippable opt
  • 6 failures on Windows 11 x86 22H2 WebRender opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=436766309&repo=mozilla-central&lineNumber=3647

Whiteboard: [stockwell needswork]

The failure is believed to be caused by bug 1845646, and the failure rate
is too high to leave the test enabled until it is fixed.

Assignee: nobody → botond
Status: NEW → ASSIGNED

Looks like bug 1852340 touched the ini file for some reasons, the change is now on autoland (https://hg.mozilla.org/integration/autoland/file/73f60b8af6dadb15fb13d768baf6b4c3313d939b/testing/web-platform/meta/css/css-overscroll-behavior/overscroll-behavior.html.ini). For the reason, I can't land D194031 now.

Oops, I was wrong. it's https://hg.mozilla.org/integration/autoland/rev/1044018d08bd . Looks like we don't need to land any changes.

Set release status flags based on info from the regressing bug 1857130

Summary: Almost perma win 11 TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent → Intermittent win 11 TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent
Whiteboard: [stockwell disable-recommended]
Summary: Intermittent win 11 TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent scro → Intermittent TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-y: none should only prevent scro

Bug 1845646 was merged into m-c, but this test is still flaky on our CI servers for some reasons. Oddly I can't reproduce the failure locally on my Linux box so far.

Status: ASSIGNED → RESOLVED
Closed: 29 days ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Attachment #9364388 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: