2036685 - [wpt-sync] Sync PR 59565 - [AI WPTs] Split WPT tests to avoid timeouts

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P4)

Description

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Sync web-platform-tests PR 59565 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/59565
Details from upstream follow.

Isaac Ahouma <[e-mail]> wrote:

[AI WPTs] Split WPT tests to avoid timeouts

This CL resolves timeout regressions in the chrome_ai_wpt_tests suite by splitting large test files.

Key changes:

1. Test Splitting: Split language-model-prompt, language-model-response-json-schema, and usage tests into smaller, focused files to prevent timeouts on slow x64 bots.
2. Flexible Assertions: Added assert_approx_equals (with a tolerance of 5) for contextUsage to support slight token count variations across different models.
3. Helpers: Created shared createColorGridCanvas helper in util.js and shared JSON schema helpers to avoid duplication.

Bug: 459864801, 500479741, 507881464
Change-Id: I42e4e372135472b7f00ce85256f6a37e34055856
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/7758933
Reviewed-by: Mike Wasserman \<[e-mail]>
Commit-Queue: Isaac Ahouma \<[e-mail]>
Cr-Commit-Position: refs/heads/main@{#1623675}

Comment 1

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=a87823818d3c3f3556fd5211160290bea50065e0

Comment 2

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=22974cf9a3f513fb07e37a4992d70494b3226c57

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Attachment: Notable wpt changes

CI Results

Ran 11 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 115 tests and 7 subtests

Status Summary

Firefox

OK : 108[GitHub] 110[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-debug, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-debug, Gecko-windows11-64-25h2-opt]
PASS : 1[Gecko-linux2204-64-wayland-debug, Gecko-linux2204-64-wayland-opt] 2[Gecko-windows11-32-25h2-debug, Gecko-windows11-64-25h2-debug, GitHub] 3[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-opt]
FAIL : 348[Gecko-windows11-32-25h2-debug] 363[Gecko-windows11-64-25h2-debug] 367[GitHub] 377[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-opt]
PRECONDITION_FAILED: 1
TIMEOUT : 1[Gecko-windows11-32-25h2-debug] 2[Gecko-windows11-64-25h2-debug] 3[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-opt, GitHub]
ERROR : 2[Gecko-windows11-32-25h2-debug] 3[Gecko-windows11-64-25h2-debug] 4[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-opt, GitHub]
NOTRUN : 7[Gecko-windows11-32-25h2-debug] 11[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-debug, Gecko-windows11-64-25h2-opt, GitHub]

Chrome

OK : 23
PASS : 85
FAIL : 16
PRECONDITION_FAILED: 10
TIMEOUT : 177
ERROR : 1
NOTRUN : 184

Safari

OK : 108
PASS : 2
FAIL : 367
PRECONDITION_FAILED: 1
TIMEOUT : 3
ERROR : 4
NOTRUN : 11

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

(See attachment for full changes)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/4bc3949b42b2
https://hg.mozilla.org/integration/autoland/rev/2a0e07d0c851
[wpt PR 59565] - [AI WPTs] Split WPT tests to avoid timeouts, a=testonly
https://github.com/mozilla-firefox/firefox/commit/bc67fb11c894
https://hg.mozilla.org/integration/autoland/rev/506f139cd119
[wpt PR 59565] - Update wpt metadata, a=testonly

Comment 5

[Silaghi Andreea]

https://hg.mozilla.org/mozilla-central/rev/2a0e07d0c851
https://hg.mozilla.org/mozilla-central/rev/506f139cd119