Closed Bug 1401684 Opened 2 years ago Closed 2 years ago

Add option to disable unstable tests with wpt-update

Categories

(Testing :: web-platform-tests, enhancement)

Version 3
enhancement
Not set

Tracking

(firefox58 fixed)

RESOLVED FIXED
mozilla57
Tracking Status
firefox58 --- fixed

People

(Reporter: maja_zf, Assigned: maja_zf)

References

Details

Attachments

(1 file)

Given a set of log files from multiple runs of the same tests, wpt-update should update metadata to disable tests that have inconsistent results.
Comment on attachment 8910439 [details]
Bug 1401684 - Add option to disable unstable tests with wpt-update;

https://reviewboard.mozilla.org/r/181874/#review187554

::: testing/web-platform/tests/tools/wptrunner/wptrunner/manifestupdate.py:249
(Diff revision 1)
>                          self.new_expected,
>                          property_order=self.root.property_order,
>                          boolean_properties=self.root.boolean_properties)
> -                except ConditionError:
> +                except ConditionError as e:
> +                    if stability is not None:
> +                       self.set("disabled", stability or "unstable", e.cond.children[0])

Would be good to get a list of disabled tests here and output it somehow so that we can tell which tests were disabled.
Attachment #8910439 - Flags: review?(james) → review+
Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/3f6b16e6955a
Add option to disable unstable tests with wpt-update; r=jgraham
We're sorry - something has gone wrong while rewriting or rebasing your commits. The commits being pushed no longer match what was requested. Please file a bug.
https://hg.mozilla.org/mozilla-central/rev/3f6b16e6955a
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in before you can comment on or make changes to this bug.