[wpt-sync] Sync PR 62614 - [AI WPTs] Assert string return type for null and undefined prompt inputs
Categories
(Testing :: web-platform-tests, task, P4)
Tracking
(firefox158 fixed)
| Tracking | Status | |
|---|---|---|
| firefox158 | --- | fixed |
People
(Reporter: wpt-sync, Unassigned)
References
()
Details
(Whiteboard: [wptsync downstream])
Sync web-platform-tests PR 62614 into mozilla-central (this bug is closed when the sync is complete).
PR: https://github.com/web-platform-tests/wpt/pull/62614
Details from upstream follow.
Isaac Ahouma <iahouma@google.com> wrote:
[AI WPTs] Assert string return type for null and undefined prompt inputs
Assert typeof string for null and undefined inputs rather than matching
literal regex substrings.Conversational models like Gemma 4 often respond to null or undefined
inputs by asking for clarification (e.g. "That is a very minimal
response. Could you please tell me what you would like me to do?")
rather than echoing the literal input text. Asserting typeof string
aligns with other empty-inputs tests and avoids intermittent failures.Remove obsolete failure expectations from the AI test manifests.
Bug: 547959960
Change-Id: Ia9a82c12283942d7c72ff546167d185cd03c4b3e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/8390497
Commit-Queue: Isaac Ahouma \<iahouma@google.com>
Reviewed-by: Mike Wasserman \<msw@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1695812}
| Assignee | ||
Comment 1•2 months ago
|
||
Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=fdbd9be54ed8b13bf4f9bdd2276af5556aa71f40
| Assignee | ||
Updated•2 months ago
|
| Assignee | ||
Updated•2 months ago
|
| Assignee | ||
Comment 2•2 months ago
|
||
CI Results
Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI
Total 7 tests and 1 subtests
Status Summary
Firefox
OK : 2[GitHub] 7[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-debug, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-debug, Gecko-windows11-64-25h2-opt]
FAIL : 2[GitHub] 7[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-25h2-debug, Gecko-windows11-32-25h2-opt, Gecko-windows11-64-25h2-debug, Gecko-windows11-64-25h2-opt]
Chrome
TIMEOUT: 4
Safari
OK : 2
FAIL : 2
Links
Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base
Details
New Tests That Don't Pass
- /ai/language-model/prompt/empty-inputs/empty-array-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows empty array input:
FAIL
- LanguageModel.prompt() allows empty array input:
- /ai/language-model/prompt/empty-inputs/empty-object-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows empty object input:
FAIL
- LanguageModel.prompt() allows empty object input:
- /ai/language-model/prompt/empty-inputs/empty-sequence-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows empty message sequence input:
FAIL
- LanguageModel.prompt() allows empty message sequence input:
- /ai/language-model/prompt/empty-inputs/empty-string-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows empty string input:
FAIL
- LanguageModel.prompt() allows empty string input:
- /ai/language-model/prompt/empty-inputs/null-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows null input:
FAIL(Chrome:TIMEOUT, Safari:FAIL)
- LanguageModel.prompt() allows null input:
- /ai/language-model/prompt/empty-inputs/sequence-with-empty-string-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows message sequence with empty string input:
FAIL
- LanguageModel.prompt() allows message sequence with empty string input:
- /ai/language-model/prompt/empty-inputs/undefined-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows undefined input:
FAIL(Chrome:TIMEOUT, Safari:FAIL)
- LanguageModel.prompt() allows undefined input:
Comment 4•2 months ago
|
||
| bugherder | ||
Description
•