Closed Bug 1558951 Opened 1 year ago Closed 1 year ago

Add support to wptrunner for allowed known_intermittent statuses


(Testing :: web-platform-tests, enhancement)

Version 3
Not set


(firefox69 fixed)

Tracking Status
firefox69 --- fixed


(Reporter: nikkis, Assigned: nikkis, Mentored)


(Blocks 1 open bug)



(1 file)

Mozlog supports an additional field for statuses that are a known_intermittent. Similar functionality would be beneficial to web-platform-tests, rather than skipping or disabling tests that are intermittent. To do so, initial support for the known_intermittent field should be added to the following:

  • - TestNode may need to be updated
  • - potentially add a KnownIntermittentUpdate class, similar to expected, or include support in ExpectedUpdate class for a known_intermittent field. - This can be part of Bug 1558954
  • - the ExpectedUpdater class needs support added to multiple class methods to ensure a known_intermittent is recorded as expected.
  • - TestRunnerManager.test_ended() needs updating to reflect new known_intermittent field

This patch adds initial support for multiple allowed expected statuses in the
test metadata. A known_intermittent property has been add to the ExpectedManifest,
TestNode and SubtestNode objects. A known_intermittent(), similar to the expected()
has been added to the Test class. This defines an expected status as the first status
in the list if expected value is a list, simply the value of expected if it is a
string. A test has been written for to reflect these changes. Where mozlog
is used, known_intermittent has been included in the expected log.

Blocks: 1558954
Depends on: 1561421
Pushed by
Add support to wptrunner for allowed known_intermittent statuses r=webdriver-reviewers,maja_zf
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
Created web-platform-tests PR for changes under testing/web-platform/tests
You need to log in before you can comment on or make changes to this bug.