Closed Bug 1489591 Opened Last year Closed Last year

[wpt-sync] Sync PR 12898 - [testharness.js] Reject non-thenable values

Categories

(Testing :: web-platform-tests, enhancement, P4)

enhancement

Tracking

(firefox64 fixed)

RESOLVED FIXED
mozilla64
Tracking Status
firefox64 --- fixed

People

(Reporter: wptsync, Unassigned)

References

()

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 12898 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/12898
Details from upstream follow.

Mike Pennisi <mike@mikepennisi.com> wrote:
>  [testharness.js] Reject non-thenable values
>  
>  [testharness.js] Reject non-thenable values
>      
>  Previously, any test defined via the `promise_test` API would fail
>  immediately if its body returned the value `undefined`. The test would
>  *not* fail if it returned any other value. Because the motivation for
>  using `promise_test` is to track resolution with a "thenable" value
>  (i.e. an object with a "then" method), this was overly permissive and
>  allowed contributors to write tests which spuriously passed [1].
>      
>  Update testharness.js to enforce more stringent criteria on the value
>  return by `promise_test` bodies: that it is a "thenable" value. This
>  change is incompatible with an exiting functional test, but it does not
>  effect any tests in WPT (as verified by a survey using both the Chrome
>  and Firefox browsers). Update the functional test accordingly.
>      
>  [1] cca6b6845678d9b5f792b886f3f7045d1d2cf0a7
>  
>  ---
>  
>  As noted in the commit message, the current leniency recently allowed test bugs that could have been avoided.
>  
>  Because there is no use case for a `promise_test` that does not return a thenable, I've also tried to verify that no other tests in WPT exhibit this problem. [I modified testharness.js and wptserve to log those cases, introduced an intentional infraction](54c71baa55408df584d12086b49acc5b8c7161bd), and [ran those changes through Chrome dev and Firefox Nightly on Taskcluster](https://tools.taskcluster.net/groups/RNMSCl4pQLa856S4zlKHoA). I used the new `tc-download` feature to inspect the result:
>  
>      $ ./wpt tc-download --repo-name bocoup/wpt --ref suspicious-promise --out-dir koo --artifact-name suspicious.txt
>      $ ls koo
>      wpt-chrome-dev-testharness-4-0-suspicious.txt  wpt-firefox-nightly-testharness-4-0-suspicious.txt
>      $ cat koo/*
>      http://web-platform.test:8000/dom/events/CustomEvent.html - CustomEvent
>      http://web-platform.test:8000/dom/events/CustomEvent.html - CustomEvent
>  
>  Since Firefox and Chrome reported the intentional infraction, I think the process is sound. Since they reported no other infractions, I don't think their are any previously-existing errors.
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c2ff4ef0946b
[wpt PR 12898] - [testharness.js] Reject non-thenable values, a=testonly
Result changes from PR not available.
https://hg.mozilla.org/mozilla-central/rev/c2ff4ef0946b
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
You need to log in before you can comment on or make changes to this bug.