Closed Bug 1458341 Opened 6 years ago Closed 6 years ago

[wpt-sync] Sync PR 10742 - [wptrunner] Re-enable return code

Categories

(Testing :: web-platform-tests, enhancement, P4)

Product:
Testing
Component:
web-platform-tests
Type:
enhancement

Tracking

(firefox62 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla62
Tracking Flags:
Tracking Status
firefox62 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream]) 

Sync web-platform-tests PR 10742 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/w3c/web-platform-tests/pull/10742
Details from upstream follow.

Mike Pennisi <mike@mikepennisi.com> wrote:
>  [wptrunner] Re-enable return code
>  
>  In a previous commit [1] the WPT CLI was modified to ignore the return
>  value of the `wptrunner.start` method. Although this behavior is
>  convenient in cases where there is no expectation data, it obscures
>  errors that are relevant in any context.
>  
>  Specifically, the `start` method returns a non-zero value when no tests
>  are run, but this is a reliable indicator of an erroneous configuration.
>  Because the `wpt run` command is used to validate the infrastructure of
>  the web-platform-tests project, ignoring this case allows
>  mis-configurations to go unnoticed [2].
>  
>  Re-enable observance of the value returned by `wptrunner.start` and
>  update the code generated for the TaskCluster service to ignore this
>  value.
>  
>  [1] 5a1b0365267b540c7f4feac3e0a85e9146997fcd
>  [2] https://github.com/w3c/web-platform-tests/pull/10721
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Ran 1 tests and 1 subtests
OK     : 1
FAIL   : 1
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f693364bbfbc
[wpt PR 10742] - [wptrunner] Re-enable return code, a=testonly
https://hg.mozilla.org/mozilla-central/rev/f693364bbfbc
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox62: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.