Open
Bug 1418375
Opened 6 years ago
Updated 2 years ago
test-verify-wpt improvements for mass file changes / imports
Categories
(Testing :: General, enhancement, P3)
Testing
General
Tracking
(Not tracked)
NEW
People
(Reporter: gbrown, Unassigned)
References
(Blocks 1 open bug)
Details
One of the challenges of test verification is time: Try to strenuously verify too many tests -- or even just a few long-running tests -- and you find the wall clock has advanced quite a ways. Too long of a run keeps people waiting for results, and risks timeouts (and relaxation of timeouts needs to be weighed against the probability of hangs). The initial implementation of test-verify favored speed over completeness: - if a single test is taking more than about 10 minutes to verify, give up - if many tests files change, only verify the first 10 - if a test verification job is taking longer than an hour, give up The initial implementation of test-verify-wpt followed the same model. However, web-platform tests are often updated en-masse on mozilla-central, to incorporate upstream modifications, and need to undergo a similar verification process. It would be nice if test-verify-wpt could handle those import scenarios. See https://bugzilla.mozilla.org/show_bug.cgi?id=1411660#c7 particularly. Specific enhancements: - a way to relax max-time and max-tests, perhaps from try - more efficient mozharness calls into the test harness (not one run per affected test) - possibly less logging (see --verify-log-full use)
Reporter | ||
Updated•6 years ago
|
Priority: -- → P3
Reporter | ||
Updated•5 years ago
|
Type: defect → enhancement
Updated•2 years ago
|
Severity: normal → S3
You need to log in
before you can comment on or make changes to this bug.
Description
•