Closed Bug 1661296 Opened 4 years ago Closed 2 years ago

[wpt-sync] Sync PR 25247 - [fetch] Introduce test generation tool & add more tests for metadata request headers

Categories

(Core :: DOM: Networking, task, P4)

task

Tracking

()

RESOLVED INVALID

People

(Reporter: mozilla.org, Unassigned)

References

()

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 25247 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/25247
Details from upstream follow.

Mike Pennisi <mike@mikepennisi.com> wrote:

[fetch] Introduce test generation tool & add more tests for metadata request headers

This patch is the latest in a research project we've been running since the beginning of the year. The README.md file included in this patch summarizes the practical usage aspects and the high-level design goals.

If you're interested in learning more about the design process, you can check out:

This patch replaces tests that were manually written with tests that have been procedurally generated by the new tool. I've verified that there are no regressions in coverage. To promote transparency on that detail, I've organized each replacement as a distinct commit in this pull request. If/when this is accepted for WPT, I recommend using a merge commit in order to preserve this information.

There are still more tests to be written (both to replace existing tests and to further improve coverage). I will be pushing additional commits to this feature branch over the next few days, though I do not expect to reach completeness. I'll be careful not to rewrite history to avoid disrupting the review process.

Component: web-platform-tests → DOM: Networking
Product: Testing → Core
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → INVALID
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Status: REOPENED → RESOLVED
Closed: 4 years ago2 years ago
Resolution: --- → INVALID
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]

CI Results

Ran 0 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 103 tests and 19 subtests

Status Summary

Firefox

OK : 99
PASS : 1855
FAIL : 94
TIMEOUT : 44
NOTRUN : 4

Chrome

OK : 100
PASS : 1877
FAIL : 109
PRECONDITION_FAILED: 20
TIMEOUT : 23
NOTRUN : 3

Safari

OK : 76
PASS : 745
FAIL : 820
PRECONDITION_FAILED: 2
TIMEOUT : 82
ERROR : 2
NOTRUN : 216

Links

GitHub PR Head
GitHub PR Base

Details

Firefox-only Failures

New Tests That Don't Pass

  • /fetch/api/redirect/redirect-to-dataurl.any.html [wpt.fyi]
    • Testing data URL loading after same-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after same-origin redirection (same-origin mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after cross-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
  • /fetch/api/redirect/redirect-to-dataurl.any.worker.html [wpt.fyi]
    • Testing data URL loading after same-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after same-origin redirection (same-origin mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after cross-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
  • /fetch/api/redirect/redirect-to-dataurl.any.sharedworker.html [wpt.fyi]
    • Testing data URL loading after same-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after same-origin redirection (same-origin mode): FAIL (Chrome: PASS, Safari: PASS)
    • Testing data URL loading after cross-origin redirection (no-cors mode): FAIL (Chrome: PASS, Safari: PASS)
  • /fetch/images/canvas-remote-read-remote-image-redirect.html [wpt.fyi]
    • Load a no-cors image from a same-origin URL that redirects to a cross-origin URL that redirects to the initial origin: FAIL (Chrome: PASS, Safari: PASS)
  • /fetch/metadata/embed.https.sub.tentative.html [wpt.fyi]: TIMEOUT (Chrome: OK, Safari: TIMEOUT)
    • Wrapper: Navigate to same-origin embed: TIMEOUT (Chrome: PASS, Safari: NOTRUN)
    • Wrapper: same-site embed: NOTRUN (Chrome: PASS, Safari: NOTRUN)
    • Wrapper: Navigate to same-site embed: NOTRUN (Chrome: PASS, Safari: NOTRUN)
    • Wrapper: cross-site embed: NOTRUN (Chrome: PASS, Safari: NOTRUN)
    • Wrapper: Navigate to cross-site embed: NOTRUN (Chrome: PASS, Safari: NOTRUN)
  • /fetch/metadata/generated/appcache-manifest.https.sub.html [wpt.fyi]: TIMEOUT (Chrome: OK, Safari: TIMEOUT)
    • sec-fetch-site - Same origin: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Cross-site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - HTTPS downgrade (header not sent): TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - HTTPS upgrade: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - HTTPS downgrade-upgrade: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Origin -> Cross-Site -> Same-Origin redirect: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Origin -> Same-Site -> Same-Origin redirect: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Cross-Site -> Same Origin: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Cross-Site -> Same-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Cross-Site -> Cross-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Origin -> Same Origin: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Origin -> Same-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Origin -> Cross-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Site -> Same Origin: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Site -> Same-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-site - Same-Site -> Cross-Site: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-mode: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-dest: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
    • sec-fetch-user: TIMEOUT (Chrome: PRECONDITION_FAILED, Safari: TIMEOUT)
  • /fetch/metadata/generated/css-images.https.sub.tentative.html [wpt.fyi]: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same origin: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Cross-site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Origin -> Cross-Site -> Same-Origin redirect: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Origin -> Same-Site -> Same-Origin redirect: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Cross-Site -> Same Origin: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Cross-Site -> Same-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Cross-Site -> Cross-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Origin -> Same Origin: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Origin -> Same-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Origin -> Cross-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Site -> Same Origin: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Site -> Same-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - Same-Site -> Cross-Site: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - HTTPS downgrade-upgrade: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-mode: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-dest: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
  • /fetch/metadata/generated/css-images.sub.tentative.html [wpt.fyi]: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - HTTPS upgrade: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
    • background-image sec-fetch-site - HTTPS downgrade-upgrade: TIMEOUT (Chrome: TIMEOUT, Safari: TIMEOUT)
  • /fetch/metadata/generated/element-a.https.sub.html [wpt.fyi]
    • sec-fetch-mode - attributes: download: FAIL (Chrome: FAIL)
    • sec-fetch-dest - attributes: download: FAIL (Chrome: PASS)
    • sec-fetch-user - no attributes with user activation: FAIL (Chrome: FAIL)
  • /fetch/metadata/generated/element-area.https.sub.html [wpt.fyi]
    • sec-fetch-mode - attributes: download: FAIL (Chrome: FAIL)
    • sec-fetch-dest - attributes: download: FAIL (Chrome: PASS)
    • sec-fetch-user - no attributes with user activation: FAIL (Chrome: FAIL)
  • /fetch/metadata/generated/element-embed.https.sub.html [wpt.fyi]
    • sec-fetch-mode: FAIL (Chrome: FAIL, Safari: NOTRUN)
  • /fetch/metadata/generated/element-frame.https.sub.html [wpt.fyi]
    • sec-fetch-user with user activation: FAIL (Chrome: PASS, Safari: FAIL)
  • /fetch/metadata/generated/element-iframe.https.sub.html [wpt.fyi]
    • sec-fetch-user with user activation: FAIL (Chrome: PASS, Safari: FAIL)
  • /fetch/metadata/generated/element-link-icon.https.sub.html [wpt.fyi]
    • sec-fetch-dest no attributes: FAIL (Chrome: FAIL, Safari: NOTRUN)
  • /fetch/metadata/generated/element-link-prefetch.https.optional.sub.html [wpt.fyi]
    • sec-fetch-dest attributes: as=audio: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=document: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=embed: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=fetch: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=font: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=image: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=object: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=script: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=style: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=track: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=video: FAIL (Chrome: FAIL)
    • sec-fetch-dest attributes: as=worker: FAIL (Chrome: FAIL)
  • /fetch/metadata/generated/form-submission.https.sub.html [wpt.fyi]
    • sec-fetch-user - POST: FAIL (Chrome: PASS, Safari: NOTRUN)
  • /fetch/metadata/generated/header-link.https.sub.html [wpt.fyi]
    • sec-fetch-site rel=icon - Same origin: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Cross-site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Origin -> Cross-Site -> Same-Origin redirect: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Origin -> Same-Site -> Same-Origin redirect: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Cross-Site -> Same Origin: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Cross-Site -> Same-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Cross-Site -> Cross-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Origin -> Same Origin: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Origin -> Same-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Origin -> Cross-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Site -> Same Origin: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Site -> Same-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Same-Site -> Cross-Site: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode rel=icon: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest rel=icon: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user rel=icon: FAIL (Chrome: FAIL, Safari: FAIL)
  • /fetch/metadata/generated/header-link.sub.html [wpt.fyi]
    • sec-fetch-site rel=icon - Not sent to non-trustworthy same-origin destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Not sent to non-trustworthy same-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - Not sent to non-trustworthy cross-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode rel=icon - Not sent to non-trustworthy same-origin destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode rel=icon - Not sent to non-trustworthy same-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode rel=icon - Not sent to non-trustworthy cross-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest rel=icon - Not sent to non-trustworthy same-origin destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest rel=icon - Not sent to non-trustworthy same-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest rel=icon - Not sent to non-trustworthy cross-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user rel=icon - Not sent to non-trustworthy same-origin destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user rel=icon - Not sent to non-trustworthy same-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user rel=icon - Not sent to non-trustworthy cross-site destination: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - HTTPS downgrade (header not sent): FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - HTTPS upgrade: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site rel=icon - HTTPS downgrade-upgrade: FAIL (Chrome: FAIL, Safari: FAIL)
  • /fetch/metadata/generated/svg-image.https.sub.html [wpt.fyi]
    • sec-fetch-mode attributes: crossorigin: FAIL (Chrome: PASS, Safari: FAIL)
    • sec-fetch-mode attributes: crossorigin=anonymous: FAIL (Chrome: PASS, Safari: FAIL)
    • sec-fetch-mode attributes: crossorigin=use-credentials: FAIL (Chrome: PASS, Safari: FAIL)
    • sec-fetch-dest no attributes: FAIL (Chrome: FAIL, Safari: FAIL)
  • /fetch/metadata/generated/worker-dedicated-constructor.sub.html [wpt.fyi]
    • sec-fetch-site - Not sent to non-trustworthy same-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-site - Not sent to non-trustworthy cross-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode - Not sent to non-trustworthy same-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-mode - Not sent to non-trustworthy cross-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest - Not sent to non-trustworthy same-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-dest - Not sent to non-trustworthy cross-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user - Not sent to non-trustworthy same-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
    • sec-fetch-user - Not sent to non-trustworthy cross-site destination, no options: FAIL (Chrome: FAIL, Safari: FAIL)
  • /fetch/metadata/paint-worklet.https.html [wpt.fyi]
    • The fetch metadata for paint worklet: FAIL (Chrome: PASS, Safari: FAIL)
  • /fetch/metadata/preload.https.sub.html [wpt.fyi]
  • /fetch/redirects/data.window.html [wpt.fyi]
    • <img> fetch that redirects to data: URL: FAIL (Chrome: PASS, Safari: FAIL)
  • /html/semantics/scripting-1/the-script-element/muted-errors.sub.html [wpt.fyi]
    • Errors for cross-origin script should be muted: FAIL (Chrome: PASS, Safari: PASS)
    • Errors for cross-origin script should be muted even if the script is once loaded as same-origin: FAIL (Chrome: PASS, Safari: PASS)
    • Errors for same-origin scripts redirected to a cross-origin url and redirected back to same-origin should be muted: FAIL (Chrome: PASS, Safari: PASS)
    • Errors for cross-origin scripts redirected to a same-origin url should be muted: FAIL (Chrome: PASS, Safari: PASS)
    • Non-synthetic errors for same-origin scripts redirected to a cross-origin URL and redirected back to same-origin should be muted: FAIL (Chrome: PASS, Safari: PASS)
    • Syntax error for same-origin script redirected to a cross-origin URL and redirected back to same-origin should be muted: FAIL (Chrome: PASS, Safari: PASS)
  • /webaudio/the-audio-api/the-mediaelementaudiosourcenode-interface/no-cors.https.html [wpt.fyi]
    • X Recorded channel #0: Expected 0 for all values but found 38591 unexpected values:
      Index Actual
      [5505] 0.015655517578125
      [5506] 0.031280517578125
      [5507] 0.0467529296875
      [5508] 0.06201171875
      ...and 38587 more errors.: FAIL
    • X Recorded channel #1: Expected 0 for all values but found 38593 unexpected values:
      Index Actual
      [5505] 0.015655517578125
      [5506] 0.031280517578125
      [5507] 0.046722412109375
      [5508] 0.06201171875
      ...and 38589 more errors.: FAIL
    • X Recorded channel #2: Expected 0 for all values but found 38592 unexpected values:
      Index Actual
      [5505] 0.015655517578125
      [5506] 0.031280517578125
      [5507] 0.046722412109375
      [5508] 0.062042236328125
      ...and 38588 more errors.: FAIL
    • X Recorded channel #3: Expected 0 for all values but found 38592 unexpected values:
      Index Actual
      [5505] 0.015655517578125
      [5506] 0.031280517578125
      [5507] 0.046722412109375
      [5508] 0.06207275390625
      ...and 38588 more errors.: FAIL
    • < [start-playback-and-capture] 4 out of 4 assertions were failed.: FAIL
    • AUDIT TASK RUNNER FINISHED: 1 out of 2 tasks were failed.: FAIL

Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f18c5ef0bf1c
[wpt PR 25247] - [fetch] Introduce test generation tool & add more tests for metadata request headers, a=testonly
https://hg.mozilla.org/integration/autoland/rev/492992cfed30
[wpt PR 25247] - Update wpt metadata, a=testonly
You need to log in before you can comment on or make changes to this bug.