Closed Bug 1904020 Opened 5 months ago Closed 5 months ago

[wpt-sync] Sync PR 46860 - [WPT] [CSP] Fix flakiness of CSPEE tests

Categories

(Core :: DOM: Security, task, P4)

Product:
Core
Component:
DOM: Security
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
129 Branch
Tracking Flags:
Tracking Status
firefox129 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 46860 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/46860
Details from upstream follow.

Antonio Sartori <antoniosartori@chromium.org> wrote:

[WPT] [CSP] Fix flakiness of CSPEE tests

This CL rewrites part of the core logic of the tests inside
external/wpt/content-security-policy/embedded-enforcement/ to make
them more resilient and less flaky.

One likely reason of flakiness before was that the tests where waiting
for several async conditions to be checked before succeeding, but they
were depending on the non-deterministic order of some of those
conditions. This CL fixes that by using one promise per each condition
and waiting for all promises to resolve.

Another problem with the tests is that they were checking for a loaded
iframe to be cross-origin by asserting a SecurityError exception when
accessing the iframe after exactly 500ms. That is now replaced with a
t.step_wait_func which retries for 5s every 500ms.

Bug: 40250883
Change-Id: I36d0bb3fabd4b0612fc70047cb995292131fb7a8
Reviewed-on: https://chromium-review.googlesource.com/5637037
WPT-Export-Revision: f4035bc82a3cb6d2f2b75791fa5c7d4b78bdb0f7

Component: web-platform-tests → DOM: Security
Product: Testing → Core

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 20 tests and 8 subtests

Status Summary

Firefox

OK : 19
PASS : 147
FAIL : 134
TIMEOUT: 3

Chrome

OK : 20
PASS : 283

Safari

OK : 19
PASS : 147
FAIL : 134
TIMEOUT: 3

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

  • /content-security-policy/embedded-enforcement/allow_csp_from-header.html [wpt.fyi]: SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt], TIMEOUT [GitHub] (Chrome: OK, Safari: TIMEOUT)
    • Same origin iframes with an empty Allow-CSP-From header get blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Same origin iframes without Allow-CSP-From header gets blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Same origin iframes are blocked if Allow-CSP-From does not match origin.: FAIL (Chrome: PASS, Safari: FAIL)
    • Cross origin iframe with an empty Allow-CSP-From header gets blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Cross origin iframe without Allow-CSP-From header gets blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Iframe with improper Allow-CSP-From header gets blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Star Allow-CSP-From header enforces EmbeddingCSP.: TIMEOUT (Chrome: PASS, Safari: TIMEOUT)
    • Allow-CSP-From header enforces EmbeddingCSP.: TIMEOUT (Chrome: PASS, Safari: TIMEOUT)
  • /content-security-policy/embedded-enforcement/blocked-iframe-are-cross-origin.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Two same-origin iframes must appear as cross-origin when one is blocked: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/required-csp-header-cascade.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Test same origin: Test same policy for both iframes: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test more restrictive policy on second iframe: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test less restrictive policy on second iframe: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test no policy on second iframe: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test no policy on first iframe: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test invalid policy on first iframe (bad directive name): FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test invalid policy on first iframe (report directive): FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test invalid policy on second iframe (bad directive name): FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Test invalid policy on second iframe (report directive): FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/required_csp-header-crlf.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/required_csp-header.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Test Required-CSP value on csp change: Sec-Required-CSP is not sent if csp attribute is not set on <iframe>.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Send Sec-Required-CSP when csp attribute of <iframe> is not empty.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Send Sec-Required-CSP when csp attribute of <iframe> is not empty.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Send Sec-Required-CSP when csp attribute of <iframe> is not empty.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Send Sec-Required-CSP when csp attribute of <iframe> is not empty.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Send Sec-Required-CSP when csp attribute of <iframe> is not empty.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Send Sec-Required-CSP Header on change of src attribute on iframe.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Send Sec-Required-CSP Header on change of src attribute on iframe.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Send Sec-Required-CSP Header on change of src attribute on iframe.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Send Sec-Required-CSP Header on change of src attribute on iframe.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Send Sec-Required-CSP Header on change of src attribute on iframe.: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - gibberish csp: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - gibberish csp: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - gibberish csp: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - gibberish csp: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - gibberish csp: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name in multiple directives: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name in multiple directives: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name in multiple directives: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name in multiple directives: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - unknown policy name in multiple directives: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - misspeled 'none': FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - misspeled 'none': FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - misspeled 'none': FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - misspeled 'none': FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - misspeled 'none': FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - query values in path: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - query values in path: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - query values in path: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - query values in path: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - query values in path: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - missing semicolon: FAIL (Chrome: PASS, Safari: FAIL)
    • Test same origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - missing semicolon: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - missing semicolon: FAIL (Chrome: PASS, Safari: FAIL)
    • Test cross origin redirect of cross origin iframe: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - missing semicolon: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong but allowed value of csp should still trigger sending Sec-Required-CSP Header - missing semicolon: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong and dangerous value of csp should not trigger sending Sec-Required-CSP Header - comma separated: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong and dangerous value of csp should not trigger sending Sec-Required-CSP Header - invalid characters in directive names: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong and dangerous value of csp should not trigger sending Sec-Required-CSP Header - invalid character in directive name: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong and dangerous value of csp should not trigger sending Sec-Required-CSP Header - report-uri present: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Wrong and dangerous value of csp should not trigger sending Sec-Required-CSP Header - report-to present: FAIL (Chrome: PASS, Safari: FAIL)
    • Test Required-CSP value on csp change: Sec-Required-CSP is not sent if csp attribute is longer than 4096 bytes: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-general.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Iframe with empty returned CSP should be blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Iframe with less restricting CSP should be blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Iframe with a different CSP should be blocked.: FAIL (Chrome: PASS, Safari: FAIL)
    • Host wildcard *.a.com does not match a.com: FAIL (Chrome: PASS, Safari: FAIL)
    • Iframe should block if intersection allows sources which are not in required_csp.: FAIL (Chrome: PASS, Safari: FAIL)
    • Iframe should block if intersection allows sources which are not in required_csp (other ordering).: FAIL (Chrome: PASS, Safari: FAIL)
    • Removed plugin-types directive should be ignored 3.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-hashes.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Returned should not include hashes not present in required csp.: FAIL (Chrome: PASS, Safari: FAIL)
    • Hashes do not have to be present in returned csp but must not allow all inline behavior.: FAIL (Chrome: PASS, Safari: FAIL)
    • Other expressions have to be subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp must allow 'sha256-abc123'.: FAIL (Chrome: PASS, Safari: FAIL)
    • Effective policy is properly found where 'sha256-abc123' is not subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
    • 'sha256-abc123' is not subsumed by 'sha256-abc456'.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-hosts.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Host must match.: FAIL (Chrome: PASS, Safari: FAIL)
    • Hosts without wildcards must match.: FAIL (Chrome: PASS, Safari: FAIL)
    • More specific subdomain should not match.: FAIL (Chrome: PASS, Safari: FAIL)
    • Specified host should not match a wildcard host.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-paths.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Returned CSP must specify a path.: FAIL (Chrome: PASS, Safari: FAIL)
    • Empty path is not subsumed by specified paths.: FAIL (Chrome: PASS, Safari: FAIL)
    • That should not be true when required csp specifies a specific page.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-ports.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Specified ports must match.: FAIL (Chrome: PASS, Safari: FAIL)
    • Returned CSP should be subsumed if the port is specified but is not default for a more secure scheme.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard port should not be subsumed by a default port.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard port should not be subsumed by a spcified port.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-protocols.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • https is more restrictive than http.: FAIL (Chrome: PASS, Safari: FAIL)
    • http: does not subsume other protocols.: FAIL (Chrome: PASS, Safari: FAIL)
    • If scheme source is present in returned csp, it must be specified in required csp too.: FAIL (Chrome: PASS, Safari: FAIL)
    • All scheme sources must be subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-nonces.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • A nonce has to be returned if required by the embedder.: FAIL (Chrome: PASS, Safari: FAIL)
    • Nonce intersection is still done on exact match - matching nonces.: FAIL (Chrome: PASS, Safari: FAIL)
    • Other expressions still have to be subsumed - negative test: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-none.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Required policy that allows none does not subsume empty list of policies.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp with effective none does not subsume a host source expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp with none does not subsume a host source expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp with effective none does not subsume none of another directive.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp with none does not subsume none of another directive.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp with none does not subsume none of different directives.: FAIL (Chrome: PASS, Safari: FAIL)
    • Both required and returned csp are none for only one directive.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-self.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Returned CSP must not allow 'self' if required CSP does not.: FAIL (Chrome: PASS, Safari: FAIL)
    • Returned 'self' should not be subsumed by a more secure version of origin's url.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-source_list-wildcards.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Wildcard does not subsume empty list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Empty source list does not subsume a wildcard source list.: FAIL (Chrome: PASS, Safari: FAIL)
    • 'none' does not subsume a wildcard source list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard source list does not subsume data: scheme source expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard source list does not subsume blob: scheme source expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • Source expressions do not subsume effective nonce expressions.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard source list is not subsumed by a host expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard list with keywords is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard list with 'unsafe-hashes' is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard list with 'unsafe-inline' is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard list with 'unsafe-eval' is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
    • Wildcard list with 'unsafe-eval' is not subsumed by list with a single expression.: FAIL (Chrome: PASS, Safari: FAIL)
    • The same as above but for 'unsafe-inline'.: FAIL (Chrome: PASS, Safari: FAIL)
    • data: is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
    • blob: is not subsumed by a wildcard list.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-strict_dynamic.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • 'strict-dynamic' is effective only for script-src.: FAIL (Chrome: PASS, Safari: FAIL)
    • 'strict-dynamic' is properly handled for finding effective policy.: FAIL (Chrome: PASS, Safari: FAIL)
    • 'strict-dynamic' has to be allowed by required csp if it is present in returned csp.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_eval.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • No other keyword has the same effect as 'unsafe-eval'.: FAIL (Chrome: PASS, Safari: FAIL)
    • Other expressions have to be subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp must allow 'unsafe-eval'.: FAIL (Chrome: PASS, Safari: FAIL)
    • Effective policy is properly found where 'unsafe-eval' is not subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_hashes.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • No other keyword has the same effect as 'unsafe-hashes'.: FAIL (Chrome: PASS, Safari: FAIL)
    • Other expressions have to be subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp must allow 'unsafe-hashes'.: FAIL (Chrome: PASS, Safari: FAIL)
    • Effective policy is properly found where 'unsafe-hashes' is not subsumed.: FAIL (Chrome: PASS, Safari: FAIL)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_inline.html?1-8 [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_inline.html?9-last [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • Required csp allows strict-dynamic, but retuned csp does.: FAIL (Chrome: PASS, Safari: FAIL)
    • Required csp does not allow unsafe-inline, but retuned csp does.: FAIL (Chrome: PASS, Safari: FAIL)
    • Returned csp allows a nonce.: FAIL (Chrome: PASS, Safari: FAIL)
    • Returned csp allows a hash.: FAIL (Chrome: PASS, Safari: FAIL)
    • Effective returned csp allows 'unsafe-inline': FAIL (Chrome: PASS, Safari: FAIL)

Tests Disabled in Gecko Infrastructure

  • /content-security-policy/embedded-enforcement/allow_csp_from-header.html [wpt.fyi]: SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt], TIMEOUT [GitHub] (Chrome: OK, Safari: TIMEOUT)
  • /content-security-policy/embedded-enforcement/blocked-iframe-are-cross-origin.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/required-csp-header-cascade.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/required_csp-header-crlf.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/required_csp-header.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-general.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-hashes.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-hosts.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-paths.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-ports.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-host_sources-protocols.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-nonces.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-none.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-self.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-source_list-wildcards.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-strict_dynamic.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_eval.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_hashes.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_inline.html?1-8 [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /content-security-policy/embedded-enforcement/subsumption_algorithm-unsafe_inline.html?9-last [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/cc3669171a10 [wpt PR 46860] - [WPT] [CSP] Fix flakiness of CSPEE tests, a=testonly

https://hg.mozilla.org/mozilla-central/rev/cc3669171a10

Status: NEW → RESOLVED
Closed: 5 months ago
status-firefox129: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 129 Branch
You need to log in before you can comment on or make changes to this bug.