Closed Bug 1587660 Opened 1 year ago Closed 10 months ago

[wpt-sync] Sync PR 19612 - [testharness.js] Don't complete after allowed exc.

Categories

(Testing :: web-platform-tests, task, P4)

task

Tracking

(firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

()

Details

(Whiteboard: [wptsync downstream])

Attachments

(3 files)

Sync web-platform-tests PR 19612 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/19612
Details from upstream follow.

Mike Pennisi <mike@mikepennisi.com> wrote:

[testharness.js] Don't complete after allowed exc.

The following tests intentionally produce an uncaught exception and then define
subtests:

  • html/semantics/scripting-1/the-script-element/module/error-and-slow-dependency.html
  • html/webappapis/scripting/processing-model-2/window-onerror-parse-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error-throw.html

testharness.js immediately transitions to "complete" and ignores the
subsequent subtests.


Today, the tests referenced in the commit message are reported as passing single-page tests. Once we've completed the implementation of WPT RFC 28, they will instead be reported as test harness errors.

As an alternative to this patch, we could modify each of the tests. If a subtest is defined before the exception is thrown, then all the subtests are reported as expected. (We could, for example, declare the existing synchronous subtests as asynchronous subtests instead.)

Changing the harness is definitely riskier, but it also seems more correct to me.

This change could interfere with single-page tests that use allow_uncaught_exceptions. My experiments don't indicate that any such tests exist, but it's difficult to demonstrate or summarize that research here. A more blunt heuristic might do. Only three tests use allow_uncaught_exception without invoking functions named test, promise_test, or async_test:

 $ git grep -l allow_uncaught_exception | xargs grep -LE '\b(test|promise_test|async_test)\s*\('
 IndexedDB/fire-error-event-exception.html
 IndexedDB/fire-success-event-exception.html
 custom-elements/upgrading/upgrading-enqueue-reactions.html

All three define subtests indirectly through helper functions.

Result changes from PR not available.
Result changes from PR not available.

Automatic update from web-platform-tests
[testharness.js] Don't complete after allowed exc. (#19612)

The following tests intentionally produce an uncaught exception and then define
subtests:

  • html/semantics/scripting-1/the-script-element/module/error-and-slow-dependency.html
  • html/webappapis/scripting/processing-model-2/window-onerror-parse-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error-throw.html

testharness.js immediately transitions to "complete" and ignores the
subsequent subtests.

--

wpt-commits: e9465a52d29ca76d30e327db675bb66139f27b8c
wpt-pr: 19612

Depends on D53418

Automatic update from web-platform-tests
[testharness.js] Don't complete after allowed exc. (#19612)

The following tests intentionally produce an uncaught exception and then define
subtests:

  • html/semantics/scripting-1/the-script-element/module/error-and-slow-dependency.html
  • html/webappapis/scripting/processing-model-2/window-onerror-parse-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error-throw.html

testharness.js immediately transitions to "complete" and ignores the
subsequent subtests.

--

wpt-commits: e9465a52d29ca76d30e327db675bb66139f27b8c
wpt-pr: 19612

Depends on D53428

Automatic update from web-platform-tests
[testharness.js] Don't complete after allowed exc. (#19612)

The following tests intentionally produce an uncaught exception and then define
subtests:

  • html/semantics/scripting-1/the-script-element/module/error-and-slow-dependency.html
  • html/webappapis/scripting/processing-model-2/window-onerror-parse-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error.html
  • html/webappapis/scripting/processing-model-2/window-onerror-runtime-error-throw.html

testharness.js immediately transitions to "complete" and ignores the
subsequent subtests.

--

wpt-commits: e9465a52d29ca76d30e327db675bb66139f27b8c
wpt-pr: 19612

Result changes from PR not available.
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ff22aa46bca0
[wpt PR 19612] - [testharness.js] Don't complete after allowed exc., a=testonly
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
You need to log in before you can comment on or make changes to this bug.