Closed Bug 2029365 Opened 3 months ago Closed 2 months ago

[wpt-sync] Sync PR 58888 - Prompt API: Setup experimental feature for sampling parameters

Tracking

(firefox151 fixed)

Status:

RESOLVED FIXED

Milestone:

151 Branch

Tracking Flags:

Tracking

Status

firefox151

---

fixed

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Description

•

3 months ago

Sync web-platform-tests PR 58888 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/58888
Details from upstream follow.

Mike Wasserman <msw@chromium.org> wrote:

Prompt API: Setup experimental feature for sampling parameters

Adds a new runtime enabled feature "AIPromptAPIParams".
V0 restores legacy temperature and topK options and attributes.
Split handling of legacy params from other renamed identifiers.
Update IDL gating and comments. Consolidate deprecation notices.

Bug: 496663356
Change-Id: I6d3cb2701493582c8e6e7ccd24473d4bcd698b03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/7708992
Reviewed-by: Jingyun Liu \<jingyun@google.com>
Auto-Submit: Mike Wasserman \<msw@chromium.org>
Reviewed-by: Dave Tapuska \<dtapuska@chromium.org>
Reviewed-by: Philip Rogers \<pdr@chromium.org>
Commit-Queue: Mike Wasserman \<msw@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1608540}

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Updated

•

3 months ago

Status: NEW → RESOLVED

Closed: 3 months ago

Resolution: --- → INVALID

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Updated

•

3 months ago

Status: RESOLVED → REOPENED

Resolution: INVALID → ---

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Comment 1

•

3 months ago

Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=031ba00d6a54c4507ea9227ad2161b92944dd807

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Comment 2

•

3 months ago

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 29 tests and 42 subtests

Status Summary

Firefox

OK : 3[GitHub] 28[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-24h2-debug, Gecko-windows11-32-24h2-opt, Gecko-windows11-64-24h2-debug, Gecko-windows11-64-24h2-opt]
FAIL : 10[GitHub] 129[Gecko-windows11-64-24h2-debug] 132[Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-24h2-debug, Gecko-windows11-32-24h2-opt, Gecko-windows11-64-24h2-opt]
TIMEOUT: 1
ERROR : 1
NOTRUN : 7

Chrome

PASS : 1
FAIL : 2
TIMEOUT: 8
NOTRUN : 11

Safari

OK : 3
FAIL : 10
TIMEOUT: 1
ERROR : 1
NOTRUN : 7

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

/ai/language-model/language-model-abort.tentative.https.window.html [wpt.fyi]
- Aborting LanguageModel.create().: FAIL
- Aborting LanguageModel.clone().: FAIL
- Aborting LanguageModel.prompt().: FAIL
- Aborting LanguageModel.promptStreaming().: FAIL
/ai/language-model/language-model-append.tentative.https.window.html [wpt.fyi]
- Simple LanguageModel.append() call: FAIL
- Check contextUsage increases from a simple LanguageModel.append() call: FAIL
- Check empty Object input for LanguageModel.append(): FAIL
- Test that append input exceeding the total context window rejects: FAIL
/ai/language-model/language-model-availability-available-multimodal.tentative.https.window.html [wpt.fyi]
- LanguageModel.availability() is available with multimodal audio option: FAIL
- LanguageModel.availability() is available with multimodal image option: FAIL
- LanguageModel.availability() returns available with supported multimodal options: FAIL
- LanguageModel.availability() returns unavailable with unsupported multimodal options: FAIL
/ai/language-model/language-model-availability-available.tentative.https.window.html [wpt.fyi]
- LanguageModel.availability() is available with no options: FAIL
- LanguageModel.availability() returns available with supported options: FAIL
- LanguageModel.availability() returns unavailable and create() rejects with unsupported options: FAIL
- LanguageModel.availability() rejects with invalid options: FAIL
/ai/language-model/language-model-availability.tentative.https.window.html [wpt.fyi]
- LanguageModel.availability() is defined: FAIL
- LanguageModel.availability() returns a valid value with no options: FAIL
- LanguageModel.availability() rejects when given invalid language tags: FAIL
- LanguageModel.availability() returns a valid value with plausible options: FAIL
/ai/language-model/language-model-clone.tentative.https.window.html [wpt.fyi]
- Language Model Clone: FAIL (Chrome: TIMEOUT, Safari: FAIL)
/ai/language-model/language-model-create-multimodal.tentative.https.window.html [wpt.fyi]
- LanguageModel.create() succeeds with supported multimodal type and language options: FAIL
- LanguageModel.create() succeeds with supported multimodal initialPrompts: FAIL
- LanguageModel.create() fails with unsupported multimodal initialPrompts: FAIL
/ai/language-model/language-model-create-user-activation.tentative.https.window.html [wpt.fyi]
- Create requires sticky user activation when availability is "downloadable": FAIL
/ai/language-model/language-model-create.tentative.https.window.html [wpt.fyi]: ERROR [Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-24h2-debug, Gecko-windows11-32-24h2-opt, Gecko-windows11-64-24h2-opt, GitHub], SKIP [Gecko-windows11-64-24h2-debug] (Chrome: TIMEOUT, Safari: ERROR)
- Ensure sessions can be created: FAIL (Chrome: PASS, Safari: FAIL)
- LanguageModel.create() returns a valid object with default options: FAIL (Chrome: TIMEOUT, Safari: FAIL)
- LanguageModel.create() notifies its monitor on downloadprogress: FAIL (Chrome: NOTRUN, Safari: FAIL)
- Progress events are not emitted after aborted.: TIMEOUT (Chrome: NOTRUN, Safari: TIMEOUT)
- Create with initialPrompts: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- Create with empty initialPrompts: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- Create with initialPrompts without system role: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- Create with system role not ordered first should fail: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- Create multiple system role entries should fail: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- LanguageModel.create() rejects when given invalid language tags: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
- LanguageModel.create() canonicalizes language tags: NOTRUN (Chrome: NOTRUN, Safari: NOTRUN)
/ai/language-model/language-model-destroy.tentative.https.window.html [wpt.fyi]
- Language Model Destroy: FAIL (Chrome: TIMEOUT, Safari: FAIL)
/ai/language-model/language-model-from-detached-iframe.tentative.https.window.html [wpt.fyi]
- Language Model From Detached Iframe: FAIL
/ai/language-model/language-model-iframe.tentative.https.html [wpt.fyi]
- Throw a 'NotAllowedError' when creating LanguageModel within cross-origin iframe: FAIL
- LanguageModel can be created within cross-origin iframe with permission policy: FAIL
- LanguageModel can be used within same-origin iframe: FAIL
- LanguageModel is unavailable within cross-origin iframe: FAIL
- LanguageModel is available within cross-origin iframe with permission policy: FAIL
- LanguageModel is available within same-origin iframe: FAIL
/ai/language-model/language-model-measure-context-usage.tentative.https.window.html [wpt.fyi]
- Language Model Measure Input Usage: FAIL
/ai/language-model/language-model-params.tentative.https.window.html [wpt.fyi]
- LanguageModel.params static accessor exists: FAIL (Chrome: FAIL, Safari: FAIL)
- LanguageModel.params() returns valid parameters: FAIL (Chrome: FAIL, Safari: FAIL)
- Default session has topK and temperature as numbers: FAIL (Chrome: TIMEOUT, Safari: FAIL)
- Create with topK and temperature returns a session with those values set: FAIL (Chrome: NOTRUN, Safari: FAIL)
- Clone preserves topK and temperature accessors: FAIL (Chrome: NOTRUN, Safari: FAIL)
/ai/language-model/language-model-prompt-context-destroyed.tentative.https.window.html [wpt.fyi]
- Detaching iframe while running prompt() should not cause memory leak: FAIL
/ai/language-model/language-model-prompt-empty-input.tentative.https.window.html [wpt.fyi]
- LanguageModel.prompt() allows empty or coerced inputs: FAIL
- LanguageModel.append() allows empty or coerced inputs: FAIL
- LanguageModel.promptStreaming() allows empty or coerced inputs: FAIL
/ai/language-model/language-model-prompt-gc.tentative.https.window.html [wpt.fyi]
- Prompt API must continue even after GC has been performed.: FAIL
/ai/language-model/language-model-prompt-monitor-callback-exception.tentative.https.window.html [wpt.fyi]
- Language Model Prompt Monitor Callback Exception: FAIL
/ai/language-model/language-model-prompt-multimodal-audio.tentative.https.window.html [wpt.fyi]
- Prompt audio without audio expectedInput: FAIL
- Test Audio initialPrompt: FAIL
- Prompt with Blob audio content: FAIL
- Prompt audio with blob containing invalid audio data.: FAIL
- Prompt with AudioBuffer: FAIL
- Prompt with BufferSource - ArrayBuffer: FAIL
/ai/language-model/language-model-prompt-multimodal-image.tentative.https.window.html [wpt.fyi]
- Prompt image without image expectedInput: FAIL
- Test Image initialPrompt: FAIL
- Prompt with Blob image content: FAIL
- Prompt with ImageBitmap image content: FAIL
- Prompt with OffscreenCanvas image content: FAIL
- Prompt with ImageData image content: FAIL
- Prompt with HTMLImageElement image content: FAIL
- Prompt with HTMLCanvasElement image content: FAIL
- Prompt with ArrayBuffer image content: FAIL
- Prompt with ArrayBufferView image content: FAIL
- Prompt with ArrayBufferView image content with an offset.: FAIL
- Prompt with HTMLImageElement image content (with SVG): FAIL
- Prompt with SVGImageElement image content: FAIL
/ai/language-model/language-model-prompt-multimodal-video.tentative.https.window.html [wpt.fyi]
- Prompt with VideoFrame image content: FAIL
- Prompt with HTMLVideoElement image content: FAIL
/ai/language-model/language-model-prompt-multimodal.tentative.https.window.html [wpt.fyi]
- Prompt with type:"text" and image content should reject: FAIL
- Prompt with assistant role should reject with multimodal input: FAIL
/ai/language-model/language-model-prompt-streaming-gc.tentative.https.window.html [wpt.fyi]
- Prompt Streaming API must continue even after GC has been performed.: FAIL
/ai/language-model/language-model-prompt-streaming.tentative.https.window.html [wpt.fyi]
- Language Model Prompt Streaming: FAIL
/ai/language-model/language-model-prompt.tentative.https.window.html [wpt.fyi]
- Simple LanguageModel.prompt() call: FAIL
- Check contextUsage increases from a simple LanguageModel.prompt() call: FAIL
- Check empty Object input: FAIL
- Check capital of France: FAIL
- Test that initialPrompt counts towards session contextUsage: FAIL
- The contextoverflow event is fired when overall usage exceeds the context window: FAIL
- Test that prompt input exceeding the total context window rejects: FAIL
/ai/language-model/language-model-quota-exceeded.tentative.https.window.html [wpt.fyi]
- QuotaExceededError is thrown when initial prompts are too large.: FAIL
/ai/language-model/language-model-response-json-schema.tentative.https.window.html [wpt.fyi]
- Prompt should reject response schemas with circular references: FAIL
- Prompt should reject response schemas with invalid types: FAIL
- Prompt should work when a valid response json schema is provided.: FAIL
- Prompt should work when a valid response json schema and matching prefix is provided.: FAIL
- Prompt should reject if the prefix deviates from the json schema constraint.: FAIL
- Prompt should omit response schema from input.: FAIL
/ai/language-model/language-model-response-regex.tentative.https.window.html [wpt.fyi]
- Prompt should work with a valid regex constraint.: FAIL
- Prompt should work with a valid regex constraint and matching prefix.: FAIL
- Prompt should reject if the prefix deviates from the regex constraint.: FAIL
/ai/language-model/language-model-tool-use.tentative.https.window.html [wpt.fyi]
- createLanguageModel should reject when tool has null inputSchema.: FAIL
- createLanguageModel should reject when tool inputSchema is empty object without type property.: FAIL
- createLanguageModel should reject when tool inputSchema is not an object.: FAIL
- createLanguageModel should reject when tool inputSchema has no type property.: FAIL
- createLanguageModel should reject when tool inputSchema type is not object.: FAIL
- createLanguageModel should reject when tool inputSchema properties is not an object.: FAIL
- createLanguageModel should reject when tool inputSchema required is not an array.: FAIL
- createLanguageModel should propagate exception when inputSchema getter throws.: FAIL
- createLanguageModel should propagate exception when inputSchema type getter throws.: FAIL
- createLanguageModel should reject when tool has empty name.: FAIL
- createLanguageModel should reject when tool has empty description.: FAIL
- createLanguageModel should reject when tools array contains duplicate tool names.: FAIL
- createLanguageModel should reject when tool has no inputSchema.: FAIL
- createLanguageModel should reject when tool has circular reference in inputSchema: FAIL
- createLanguageModel should reject when all tools have circular references: FAIL
- createLanguageModel should propagate V8 exception from custom toJSON(): FAIL
- createLanguageModel should propagate V8 exception from custom getter during serialization: FAIL
- createLanguageModel should fail-fast when one tool among many is invalid: FAIL
- createLanguageModel with schema containing invalid-JSON-like text (trailing commas, comments) as string content succeeds because JSON.stringify escapes properly: FAIL
- createLanguageModel should succeed with empty tools array.: FAIL
- createLanguageModel should succeed with no tools property.: FAIL
- createLanguageModel should reject when tools provided but expectedOutputs is missing.: FAIL
- createLanguageModel should reject when tools provided but expectedOutputs does not include tool-call.: FAIL
- createLanguageModel should succeed when tools provided with tool-call in expectedOutputs.: FAIL
- createLanguageModel should succeed with tool-call in expectedOutputs but no tools.: FAIL
- prompt() should return structured tool call messages in open-loop pattern: FAIL
- Open-loop pattern - send tool response via follow-up prompt: FAIL
- Tool response with DOM object (ImageBitmap) labeled as type object should reject: FAIL
- Multimodal tool response with ImageBitmap throws NotSupportedError: FAIL
- Multimodal tool response with AudioBuffer throws NotSupportedError: FAIL
- promptStreaming() should stream tool call messages: FAIL
- Tool with no arguments should have empty arguments object: FAIL
- Multiple tools can be declared and called: FAIL
- Tool response can include error field: FAIL
- Tool response with circular reference should reject with DataError: FAIL
- Tool response with function value should reject with DataError: FAIL
- Tool response with BigInt value should reject with DataError: FAIL
- Tool response with valid serializable values should succeed: FAIL
- Cloned model should preserve tools: FAIL
- prompt() returns both text and tool call in correct order when model outputs mixed response: FAIL
- prompt() should handle multiple batches of tool calls from model: FAIL
- promptStreaming() should handle multiple batches of tool calls from model: FAIL

Tests Disabled in Gecko Infrastructure

/ai/language-model/language-model-create.tentative.https.window.html [wpt.fyi]: ERROR [Gecko-android-em-14-x86_64-debug-geckoview, Gecko-android-em-14-x86_64-lite-opt-geckoview, Gecko-android-em-14-x86_64-opt-geckoview, Gecko-linux2404-64-debug, Gecko-linux2404-64-opt, Gecko-windows11-32-24h2-debug, Gecko-windows11-32-24h2-opt, Gecko-windows11-64-24h2-opt, GitHub], SKIP [Gecko-windows11-64-24h2-debug] (Chrome: TIMEOUT, Safari: ERROR)

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Updated

•

3 months ago

Depends on: 2030679

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Updated

•

3 months ago

Depends on: 2031228

Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)

Assignee

Updated

•

3 months ago

Depends on: 2031248

Pulsebot

Comment 3

•

2 months ago

Pushed by wptsync@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/578c4035c357 https://hg.mozilla.org/integration/autoland/rev/e23aba19dd8e [wpt PR 58888] - Prompt API: Setup experimental feature for sampling parameters, a=testonly https://github.com/mozilla-firefox/firefox/commit/af14957a5027 https://hg.mozilla.org/integration/autoland/rev/5d94786facc7 [wpt PR 58888] - Update wpt metadata, a=testonly

rperta

Comment 4

•

2 months ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/e23aba19dd8e
https://hg.mozilla.org/mozilla-central/rev/5d94786facc7

Status: REOPENED → RESOLVED

Closed: 3 months ago → 2 months ago

status-firefox151: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → 151 Branch

You need to log in before you can comment on or make changes to this bug.

Bugzilla

[wpt-sync] Sync PR 58888 - Prompt API: Setup experimental feature for sampling parameters

Categories

(Testing :: web-platform-tests, task, P4)

Tracking

(firefox151 fixed)

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Crash Data

Security

(public)

User Story

Description

Updated

Updated

Comment 1

Comment 2

CI Results

Status Summary

Firefox

Chrome

Safari

Links

Details

New Tests That Don't Pass

Tests Disabled in Gecko Infrastructure

Updated

Updated

Updated

Comment 3

Comment 4