1838683 - [wpt-sync] Sync PR 40570 - [FedCM] Actually run the multi-IDP tests

Status: RESOLVED FIXED

(Core :: DOM: Credential Management, task, P4)

Description

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Sync web-platform-tests PR 40570 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/40570
Details from upstream follow.

Christian Biesinger <[e-mail]> wrote:

[FedCM] Actually run the multi-IDP tests

Due to an incorrect merge conflict resolution in
https://chromium-review.googlesource.com/c/chromium/src/+/3924487/56..58
these tests never actually ran.

The problem is that they are skipped in the base test suite (as
expected), but the virtual test suite inherits the Skip expectation
and the line that changes it to Pass has been removed by accident.

Bug: 1348262
Change-Id: I2711c3607f727fa5f92e32833c7b645e31ba2f66
Reviewed-on: https://chromium-review.googlesource.com/4617005
WPT-Export-Revision: b24bb88087e42fb8a1cbb9961b8d8c8082eb1c10

Comment 1

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

PR 40570 applied with additional changes from upstream: aca430c872c68bbbdf10f45b94f7d947001f9d95

Comment 2

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=c889226609041476333229c97c8c03c06c3206de

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 9 tests and 1 subtests

Status Summary

Firefox

OK : 1
FAIL : 6
TIMEOUT: 3
ERROR : 8

Chrome

OK : 1
PASS : 1
FAIL : 3
TIMEOUT: 5
ERROR : 8

Safari

OK : 3
FAIL : 6
TIMEOUT: 4
ERROR : 5

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

• /credential-management/fedcm-multi-idp/abort-multiple-gets-through-first-idp.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
  • Test abort signal for a multi IDP request by aborting the first IDP: FAIL (Chrome: PASS, Safari: FAIL)
• /credential-management/fedcm-multi-idp/abort-multiple-gets-through-second-idp.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
  • Test abort signal for a multi IDP request by aborting the second IDP: FAIL (Chrome: TIMEOUT, Safari: FAIL)
• /credential-management/fedcm-multi-idp/get-before-and-after-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: OK)
  • When there's a get call before onload, a get call which occurs after onload but before the first get call resolves, should be rejected.: FAIL (Chrome: TIMEOUT, Safari: FAIL)
• /credential-management/fedcm-multi-idp/get-before-and-during-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
  • A get call before onload and a get call during onload should be combined.: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
• /credential-management/fedcm-multi-idp/get-before-onload-and-during-dom-content-loaded.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
  • A get call before onload and a get call during DOMContentLoaded event should combine despite being called from different tasks.: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
• /credential-management/fedcm-multi-idp/multiple-gets-after-abort.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
  • Multiple gets after aborting a multi IDP request should work: FAIL (Chrome: FAIL, Safari: FAIL)
• /credential-management/fedcm-multi-idp/multiple-gets-after-onload.https.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • No get calls before or during onload, multiple get calls after onload in the same task are allowed.: FAIL (Chrome: FAIL, Safari: FAIL)
• /credential-management/fedcm-multi-idp/multiple-gets-before-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: OK)
  • Multiple get calls before window onload are allowed.: FAIL (Chrome: FAIL, Safari: FAIL)
• /credential-management/fedcm-multi-idp/multiple-gets-during-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: TIMEOUT)
  • No get calls before onload, multiple get calls during onload are allowed.: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)

Tests Disabled in Gecko Infrastructure

• /credential-management/fedcm-multi-idp/abort-multiple-gets-through-first-idp.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
• /credential-management/fedcm-multi-idp/abort-multiple-gets-through-second-idp.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
• /credential-management/fedcm-multi-idp/get-before-and-after-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: OK)
• /credential-management/fedcm-multi-idp/get-before-and-during-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
• /credential-management/fedcm-multi-idp/get-before-onload-and-during-dom-content-loaded.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
• /credential-management/fedcm-multi-idp/multiple-gets-after-abort.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: ERROR)
• /credential-management/fedcm-multi-idp/multiple-gets-after-onload.https.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
• /credential-management/fedcm-multi-idp/multiple-gets-before-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: OK)
• /credential-management/fedcm-multi-idp/multiple-gets-during-onload.https.html [wpt.fyi]: ERROR [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: ERROR, Safari: TIMEOUT)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b6ddbab050ba
[wpt PR 40570] - [FedCM] Actually run the multi-IDP tests, a=testonly

Comment 5

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/b6ddbab050ba