Closed Bug 1864552 Opened 2 years ago Closed 2 years ago

[wpt-sync] Sync PR 43131 - Fix recent-input.html Flaky Test

Categories

(Testing :: web-platform-tests, task, P4)

Product:
Testing
Component:
web-platform-tests
Type:
task

Tracking

(firefox122 fixed)

Status:
RESOLVED FIXED
Milestone:
122 Branch
Tracking Flags:
Tracking Status
firefox122 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 43131 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/43131
Details from upstream follow.

Aoyuan Zuo <zuoaoyuan@chromium.org> wrote:

Fix recent-input.html Flaky Test

  • Fix the flakiness
  • Refactor/flatten the test for better readability
  • Rename clickHandler to pointerdownHandler

Previous Undesired Outcome:

  • assert_greater_than_equal(timeAfterClick, entry.lastInputTime)
    sometimes fail.

Root Cause:

  • The variable timeAfterClick is really a time after pointerdown and can
    possibly happen before the click event processingStart.
  • LastInputTime is the timestamp of the event comes in. And when a new
    event show up later, it will get refreshed.
  • Thus, if we try to compare pointdown's processingEnd with
    lastInputTime of (for example) click event, then there's no guarantee
    of one is greater the other.

This CL address it by:

  • Remove timeAfterClick, which is not a very meaningful timestamp, since
    we should never assume no more events coming in after.
  • Compare lastInputTime with the layout shift startTime instead, for
    which the sequence is always guaranteed.

Bug: 1441125
Change-Id: Id5fd368db78afd16df4b2a1e7b8924b32fa6a0f8
Reviewed-on: https://chromium-review.googlesource.com/5026199
WPT-Export-Revision: 16f7e2091c57be751754d9a7a2b448523bfe1956

CI Results

Ran 7 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 47 tests and 1 subtests

Status Summary

Firefox

OK : 43[Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] 44[Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview] 45[GitHub]
PASS : 29[Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, GitHub] 30[Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug]
FAIL : 13[Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] 14[Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt] 15[Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview] 16[GitHub]
TIMEOUT: 4
ERROR : 1

Chrome

OK : 46
PASS : 46
TIMEOUT: 1
ERROR : 1

Safari

OK : 47
FAIL : 47

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

Firefox-only Failures

  • /event-timing/first-input-interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub]
  • /event-timing/interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub]

New Tests That Don't Pass

  • /layout-instability/input-timestamp.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] (Chrome: OK, Safari: OK)
    • Layout shift right after user input is observable via PerformanceObserver.: FAIL (Chrome: PASS, Safari: FAIL)
  • /layout-instability/recent-input.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] (Chrome: OK, Safari: OK)
    • Layout shift right after user input is observable via PerformanceObserver.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/crossiframe.html [wpt.fyi]
    • Event Timing: entries should only be observable by its own frame.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/disconnect-target.html [wpt.fyi]
    • Event Timing: when target is disconnected, entry.target returns null.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/duration-with-target-low.html [wpt.fyi]
    • PerformanceObserver with durationThreshold of 300 and processingDelay of 0 doesn't see any entries in the observer: FAIL [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, GitHub], PASS [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] (Chrome: PASS, Safari: FAIL)
  • /event-timing/event-click-visibilitychange.html [wpt.fyi]: ERROR [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], OK [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug, GitHub] (Chrome: ERROR, Safari: OK)
  • /event-timing/event-retarget.html [wpt.fyi]
    • Event Timing: target reports the last Event Target, i.e. nothing from shadow DOM.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/first-input-interactionid-click.html [wpt.fyi]
    • The interactionId of the first input entry should match the same pointerdown entry of event timing when click.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/first-input-interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub] (Chrome: OK, Safari: OK)
    • The interactionId of the first input entry should match the same pointerdown entry of event timing when tap.: FAIL [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], TIMEOUT [GitHub] (Chrome: PASS, Safari: FAIL)
  • /event-timing/interaction-count-click.html [wpt.fyi]
    • EventTiming: verify interactionCount for click interaction: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interaction-count-press-key.html [wpt.fyi]
    • EventTiming: verify interactionCount for key interaction: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interaction-count-tap.html [wpt.fyi]
    • EventTiming: verify interactionCount for tap interaction: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interactionid-click.html [wpt.fyi]
    • Event Timing: compare event timing interactionId.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interactionid-press-key-as-input.html [wpt.fyi]
    • Event Timing: compare event timing interactionId for key press as input.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interactionid-press-key-no-effect.html [wpt.fyi]
    • Event Timing: compare event timing interactionId for key press with no effect.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub] (Chrome: OK, Safari: OK)
    • Event Timing: compare event timing interactionId for tap.: FAIL [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], TIMEOUT [GitHub] (Chrome: PASS, Safari: FAIL)
  • /event-timing/only-observe-firstInput.html [wpt.fyi]
    • Event Timing: check first-input for a PerformanceObserver observing only first-input.: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/programmatic-click-not-observed.html [wpt.fyi]
    • Event Timing: events from programmatic click are not observed: FAIL (Chrome: PASS, Safari: FAIL)
  • /event-timing/retrieve-firstInput.html [wpt.fyi]
    • Event Timing: check first-input after onload, observer, click, click.: FAIL (Chrome: PASS, Safari: FAIL)

Tests Disabled in Gecko Infrastructure

  • /layout-instability/input-timestamp.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] (Chrome: OK, Safari: OK)
  • /layout-instability/recent-input.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug] (Chrome: OK, Safari: OK)
  • /event-timing/first-input-interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub] (Chrome: OK, Safari: OK)
  • /event-timing/interactionid-tap.html [wpt.fyi]: OK [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview], SKIP [Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-64-2009-qr-debug], TIMEOUT [GitHub] (Chrome: OK, Safari: OK)
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/152866049637 [wpt PR 43131] - Fix recent-input.html Flaky Test, a=testonly
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/0dcade851bcb [wpt PR 43131] - Fix recent-input.html Flaky Test, a=testonly

https://hg.mozilla.org/mozilla-central/rev/0dcade851bcb

Status: NEW → RESOLVED
Closed: 2 years ago
status-firefox122: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 122 Branch
You need to log in before you can comment on or make changes to this bug.