Closed Bug 1790021 Opened 2 years ago Closed 2 years ago

Reassign all upstream wpt.fyi failures with valid BMO references

Tracking

(Not tracked)

Status:

RESOLVED FIXED

People

(Reporter: mozilla.org, Assigned: whimboo)

References

Details

(Whiteboard: [webdriver:m5])

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Reporter

Description

•

2 years ago

Syncing wpt PR 35577 found new untriaged test failures in CI

Tests Affected

New Tests That Don't Pass

/webdriver/tests/bidi/browsing_context/context_created/context_created.py [wpt.fyi]
- test_evaluate_window_open_without_url: FAIL (Chrome: PASS)
- test_evaluate_window_open_with_url: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/await_promise.py [wpt.fyi]
- test_await_promise_delayed[False]: FAIL (Chrome: PASS)
- test_await_promise_async_arrow[False]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/call_function.py [wpt.fyi]
- test_default_this: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/exception_details.py [wpt.fyi]
- test_invalid_function: FAIL (Chrome: PASS)
- test_exception_details[(Symbol('foo'))-expected13-True]: FAIL (Chrome: PASS)
- test_exception_details[(Symbol('foo'))-expected13-False]: FAIL (Chrome: PASS)
- test_exception_details[(()=>{})-expected16-True]: FAIL (Chrome: PASS)
- test_exception_details[(()=>{})-expected16-False]: FAIL (Chrome: PASS)
- test_exception_details[(function(){})-expected17-True]: FAIL (Chrome: PASS)
- test_exception_details[(function(){})-expected17-False]: FAIL (Chrome: PASS)
- test_exception_details[(async ()=>{})-expected18-True]: FAIL (Chrome: PASS)
- test_exception_details[(async ()=>{})-expected18-False]: FAIL (Chrome: PASS)
- test_exception_details[(async function(){})-expected19-True]: FAIL (Chrome: PASS)
- test_exception_details[(async function(){})-expected19-False]: FAIL (Chrome: PASS)
- test_exception_details[new WeakMap()-expected24-True]: FAIL (Chrome: PASS)
- test_exception_details[new WeakMap()-expected24-False]: FAIL (Chrome: PASS)
- test_exception_details[new WeakSet()-expected25-True]: FAIL (Chrome: PASS)
- test_exception_details[new WeakSet()-expected25-False]: FAIL (Chrome: PASS)
- test_exception_details[new Error('SOME_ERROR_TEXT')-expected26-True]: FAIL (Chrome: PASS)
- test_exception_details[new Error('SOME_ERROR_TEXT')-expected26-False]: FAIL (Chrome: PASS)
- test_exception_details[Promise.resolve()-expected27-True]: FAIL (Chrome: PASS)
- test_exception_details[Promise.resolve()-expected27-False]: FAIL (Chrome: PASS)
- test_exception_details[new Int32Array()-expected28-True]: FAIL (Chrome: PASS)
- test_exception_details[new Int32Array()-expected28-False]: FAIL (Chrome: PASS)
- test_exception_details[new ArrayBuffer()-expected29-True]: FAIL (Chrome: PASS)
- test_exception_details[new ArrayBuffer()-expected29-False]: FAIL (Chrome: PASS)
- test_exception_details[document.createElement('div')-expected30-True]: FAIL (Chrome: PASS)
- test_exception_details[document.createElement('div')-expected30-False]: FAIL (Chrome: PASS)
- test_exception_details[window-expected31-True]: FAIL (Chrome: PASS)
- test_exception_details[window-expected31-False]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/invalid.py [wpt.fyi]
- test_params_realm_unknown: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/result.py [wpt.fyi]
- test_remote_values[(Symbol('foo'))-expected0-True]: FAIL (Chrome: PASS)
- test_remote_values[(Symbol('foo'))-expected0-False]: FAIL (Chrome: PASS)
- test_remote_values[(()=>{})-expected3-True]: FAIL (Chrome: PASS)
- test_remote_values[(()=>{})-expected3-False]: FAIL (Chrome: PASS)
- test_remote_values[(function(){})-expected4-True]: FAIL (Chrome: PASS)
- test_remote_values[(function(){})-expected4-False]: FAIL (Chrome: PASS)
- test_remote_values[(async ()=>{})-expected5-True]: FAIL (Chrome: PASS)
- test_remote_values[(async ()=>{})-expected5-False]: FAIL (Chrome: PASS)
- test_remote_values[(async function(){})-expected6-True]: FAIL (Chrome: PASS)
- test_remote_values[(async function(){})-expected6-False]: FAIL (Chrome: PASS)
- test_remote_values[new WeakMap()-expected11-True]: FAIL (Chrome: PASS)
- test_remote_values[new WeakMap()-expected11-False]: FAIL (Chrome: PASS)
- test_remote_values[new WeakSet()-expected12-True]: FAIL (Chrome: PASS)
- test_remote_values[new WeakSet()-expected12-False]: FAIL (Chrome: PASS)
- test_remote_values[new Error('SOME_ERROR_TEXT')-expected13-True]: FAIL (Chrome: PASS)
- test_remote_values[new Error('SOME_ERROR_TEXT')-expected13-False]: FAIL (Chrome: PASS)
- test_remote_values[new Int32Array()-expected14-True]: FAIL (Chrome: PASS)
- test_remote_values[new Int32Array()-expected14-False]: FAIL (Chrome: PASS)
- test_remote_values[new ArrayBuffer()-expected15-True]: FAIL (Chrome: PASS)
- test_remote_values[new ArrayBuffer()-expected15-False]: FAIL (Chrome: PASS)
- test_remote_values[document.createElement('div')-expected16-True]: FAIL (Chrome: PASS)
- test_remote_values[document.createElement('div')-expected16-False]: FAIL (Chrome: PASS)
- test_remote_values[window-expected17-True]: FAIL (Chrome: PASS)
- test_remote_values[window-expected17-False]: FAIL (Chrome: PASS)
- test_remote_value_promise_no_await: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/result_ownership.py [wpt.fyi]
- test_invalid_script[root-True]: FAIL (Chrome: PASS)
- test_invalid_script[none-False]: FAIL (Chrome: PASS)
- test_invalid_script[None-False]: FAIL (Chrome: PASS)
- test_return_value[root-True-False]: FAIL (Chrome: PASS)
- test_return_value[none-False-False]: FAIL (Chrome: PASS)
- test_return_value[None-False-False]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/call_function/strict_mode.py [wpt.fyi]
- test_strict_mode: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/await_promise.py [wpt.fyi]
- test_no_await_promise_rejected: FAIL (Chrome: PASS)
- test_no_await_promise_resolved: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/exception_details.py [wpt.fyi]
- test_invalid_script: FAIL (Chrome: PASS)
- test_exception_details[(Symbol('foo'))-expected13-True]: FAIL (Chrome: PASS)
- test_exception_details[(Symbol('foo'))-expected13-False]: FAIL (Chrome: PASS)
- test_exception_details[(()=>{})-expected16-True]: FAIL (Chrome: PASS)
- test_exception_details[(()=>{})-expected16-False]: FAIL (Chrome: PASS)
- test_exception_details[(function(){})-expected17-True]: FAIL (Chrome: PASS)
- test_exception_details[(function(){})-expected17-False]: FAIL (Chrome: PASS)
- test_exception_details[(async ()=>{})-expected18-True]: FAIL (Chrome: PASS)
- test_exception_details[(async ()=>{})-expected18-False]: FAIL (Chrome: PASS)
- test_exception_details[(async function(){})-expected19-True]: FAIL (Chrome: PASS)
- test_exception_details[(async function(){})-expected19-False]: FAIL (Chrome: PASS)
- test_exception_details[new WeakMap()-expected24-True]: FAIL (Chrome: PASS)
- test_exception_details[new WeakMap()-expected24-False]: FAIL (Chrome: PASS)
- test_exception_details[new WeakSet()-expected25-True]: FAIL (Chrome: PASS)
- test_exception_details[new WeakSet()-expected25-False]: FAIL (Chrome: PASS)
- test_exception_details[new Error('SOME_ERROR_TEXT')-expected26-True]: FAIL (Chrome: PASS)
- test_exception_details[new Error('SOME_ERROR_TEXT')-expected26-False]: FAIL (Chrome: PASS)
- test_exception_details[Promise.resolve()-expected27-True]: FAIL (Chrome: PASS)
- test_exception_details[Promise.resolve()-expected27-False]: FAIL (Chrome: PASS)
- test_exception_details[new Int32Array()-expected28-True]: FAIL (Chrome: PASS)
- test_exception_details[new Int32Array()-expected28-False]: FAIL (Chrome: PASS)
- test_exception_details[new ArrayBuffer()-expected29-True]: FAIL (Chrome: PASS)
- test_exception_details[new ArrayBuffer()-expected29-False]: FAIL (Chrome: PASS)
- test_exception_details[document.createElement('div')-expected30-True]: FAIL (Chrome: PASS)
- test_exception_details[document.createElement('div')-expected30-False]: FAIL (Chrome: PASS)
- test_exception_details[window-expected31-True]: FAIL (Chrome: PASS)
- test_exception_details[window-expected31-False]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/invalid.py [wpt.fyi]
- test_params_realm_unknown: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/result.py [wpt.fyi]
- test_remote_values[(Symbol('foo'))-expected0]: FAIL (Chrome: PASS)
- test_remote_values[(()=>{})-expected3]: FAIL (Chrome: PASS)
- test_remote_values[(function(){})-expected4]: FAIL (Chrome: PASS)
- test_remote_values[(async ()=>{})-expected5]: FAIL (Chrome: PASS)
- test_remote_values[(async function(){})-expected6]: FAIL (Chrome: PASS)
- test_remote_values[new WeakMap()-expected11]: FAIL (Chrome: PASS)
- test_remote_values[new WeakSet()-expected12]: FAIL (Chrome: PASS)
- test_remote_values[new Error('SOME_ERROR_TEXT')-expected13]: FAIL (Chrome: PASS)
- test_remote_values[Promise.resolve()-expected14]: FAIL (Chrome: PASS)
- test_remote_values[new Int32Array()-expected15]: FAIL (Chrome: PASS)
- test_remote_values[new ArrayBuffer()-expected16]: FAIL (Chrome: PASS)
- test_remote_values[document.createElement('div')-expected17]: FAIL (Chrome: PASS)
- test_remote_values[window-expected18]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/result_ownership.py [wpt.fyi]
- test_invalid_script[root-True]: FAIL (Chrome: PASS)
- test_invalid_script[none-False]: FAIL (Chrome: PASS)
- test_invalid_script[None-False]: FAIL (Chrome: PASS)
- test_return_value[root-True-False]: FAIL (Chrome: PASS)
- test_return_value[none-False-False]: FAIL (Chrome: PASS)
- test_return_value[None-False-False]: FAIL (Chrome: PASS)
/webdriver/tests/bidi/script/evaluate/strict_mode.py [wpt.fyi]
- test_strict_mode: FAIL (Chrome: PASS)
/webdriver/tests/get_element_property/get.py [wpt.fyi]
- test_web_reference[shadowRoot-ShadowRoot]: FAIL (Chrome: PASS)

CI Results

Gecko CI (Treeherder)
GitHub PR Head

Notes

These updates will be on mozilla-central once bug 1786541 lands.

Note: this bug is for tracking fixing the issues and is not
owned by the wpt sync bot.

This bug is linked to the relevant tests by an annotation in
https://github.com/web-platform-tests/wpt-metadata. These annotations
can be edited using the wpt interop dashboard
https://jgraham.github.io/wptdash/

If this bug is split into multiple bugs, please also update the
annotations, otherwise we are unable to track which wpt issues are
already triaged. Resolving as duplicate or closing this issue should
be cause the bot to automatically update or remove the annotation.

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Reporter

Updated

•

2 years ago

Depends on: 1786541

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Updated

•

2 years ago

Flags: needinfo?(hskupin)

Priority: -- → P5

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Comment 1

•

2 years ago

•

Edited

I've updated all the Bugzilla references on wpt.fyi to point to actual valid bugs and if none were present I filed new bugs. Here are the upstream metadata PRs:

Given that this was quite some work to go through I would propose to track it for M5 with 2 points.

Assignee: nobody → hskupin

Blocks: webdriver:m5

Status: NEW → RESOLVED

Points: --- → 2

Closed: 2 years ago

Flags: needinfo?(hskupin)

Priority: P5 → P1

Resolution: --- → FIXED

Summary: New wpt failures in /webdriver/tests/ [bidi/browsing_context/context_created/context_created.py, bidi/script/call_function/await_promise.py, bidi/script/call_function/call_function.py, and 12 others] → Reassign all upstream wpt.fyi failures with valid BMO references

Whiteboard: [wpt] → [webdriver:m5]

You need to log in before you can comment on or make changes to this bug.

Bugzilla

Quick Search

Reassign all upstream wpt.fyi failures with valid BMO references

Categories

(Testing :: geckodriver, defect, P1)

Tracking

(Not tracked)

People

(Reporter: mozilla.org, Assigned: whimboo)

References

Details

(Whiteboard: [webdriver:m5])

Crash Data

Security

(public)

User Story

Description

Tests Affected

New Tests That Don't Pass

CI Results

Notes

Updated

Updated

Comment 1