[meta] Remove outdated/redundant "implementation-status: backlog"
Categories
(Testing :: web-platform-tests, task)
Tracking
(Not tracked)
People
(Reporter: saschanaz, Unassigned)
References
Details
(Keywords: meta)
It downgrades tests from tier 1 to tier 2 if the corresponding feature is not implemented. If that changed, it should go away.
Reporter | ||
Comment 1•24 days ago
|
||
Would it make sense for wpt-sync to automatically remove the status for tests with non-zero success rate (or at least some percent of success rate)? Would it be doable?
Comment 2•23 days ago
|
||
I agree this is unfortunate :/
Non-zero success rate wouldn't work; suites often have some percentage of tests that pass when they aren't implemented (typically checking the feature doesn't change some existing behaviour). That can be relatively high; I don't think 25% is unrealistic.
I think in theory the sync could remove implementation-status
annotations when we reach some higher threshold e.g. 50%, but I think it's probably not quite trivial to implement. First we'd need to find everything marked as implementation-status: backlog
and then record the pass rate for those tests. But at the moment the update code assumes that all the information it needs to decide whether to update something is in the log files i.e. we're not using the metadata files as input to decide whether to update something in the first place (and as a result get to skip reading them except when we already determined that something needs to be updated). Since AFAIK we don't have backlog status in the log files, (except in some very hacky way like via the job name), that doesn't quite work here.
Description
•