Since some web-platform tests are known to perma fail or time out, and this is recorded in the expectation manifest files, it may be advantageous to skip tests which are known to fail to reduce wasted effort and to improve load and run times. I did a [base line try](https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=7372432e9fac0cbcbf34215c011a27a4ee29b84a) with mach try fuzzy --query web-platform and compared it to [skip failure try](https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=795f20453a0599b330733154ba0672523ea1734f) total taskcluster run time | baseline | skip failures | |-----------|---------------| | 97,469.98 | 79,196.79 | This certainly seems like a good move. 19% reduction
Bug 1594796 Comment 0 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Since some web-platform tests are known to perma fail or time out, and this is recorded in the expectation manifest files, it may be advantageous to skip tests which are known to fail to reduce wasted effort and to improve load and run times. I did a [base line try](https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=7372432e9fac0cbcbf34215c011a27a4ee29b84a) with mach try fuzzy --query web-platform and compared it to [skip failure try](https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=795f20453a0599b330733154ba0672523ea1734f) total taskcluster run time | baseline | skip failures | |-----------|---------------| | 97,469.98 | 79,196.79 | 19% reduction This certainly seems like a good move.