Closed Bug 1504357 Opened Last year Closed 11 months ago

`./mach wpt path/to/single/disabled.test --repeat-until-unexpected --no-pause-after-test` results in an instant infinite loop

Categories

(Testing :: web-platform-tests, defect)

Version 3
defect
Not set

Tracking

(firefox65 fixed)

RESOLVED FIXED
mozilla65
Tracking Status
firefox65 --- fixed

People

(Reporter: KWierso, Assigned: KWierso)

References

Details

Attachments

(1 file)

I'm going through disabling wpt tests that are failing on Fennec to get the rest of the tests running in CI (or in preparation for getting these tests up and running for future Geckoview-based browsers).

As part of this, I use `./mach wpt path/to/single/test.file` with the --repeat-until-unexpected and --no-pause-after-test flags passed in to help quickly re-run tests that only intermittently fail. 

I accidentally did this with a test that I had already disabled, and I ended up in an infinite loop of the test runner printing out things like this:

 0:29.56 INFO Repetition 1594
 0:29.57 SUITE_START: web-platform-test - running 1 tests
 0:29.57 INFO Running reftest tests
 0:29.57 ERROR Unsupported test type reftest for product fennec
 0:29.57 INFO Running wdspec tests
 0:29.57 ERROR Unsupported test type wdspec for product fennec
 0:29.57 INFO Running testharness tests
 0:29.57 TEST_START: /dom/events/Event-dispatch-on-disabled-elements.html
 0:29.57 TEST_END: SKIP
 0:29.57 INFO No testharness tests to run
 0:29.57 INFO Got 0 unexpected results
 0:29.57 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 1594 checks (1594 tests)
Expected results: 0
Skipped: 1594 tests
OK




I feel like --repeat-until-unexpected should bail out once it realizes that there are zero tests being run.
If zero tests are selected to run while --repeat-until-unexpected is specified, wptrunner can get into a fast infinite loop of running nothing until killed by the user.

This patch should stop that from happening by breaking the loop after the first iteration of nothing finishes.
Assignee: nobody → wkocher
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/f988af9437f5
Make sure --repeat-until-unexpected doesn't loop over zero tests r=jgraham
https://hg.mozilla.org/mozilla-central/rev/f988af9437f5
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/14303 for changes under testing/web-platform/tests
Upstream PR merged
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/0f33ddfa45f6
[wpt PR 14303] - [Gecko Bug 1504357] Make sure --repeat-until-unexpected doesn't loop over zero tests, a=testonly
Depends on: 1528735
You need to log in before you can comment on or make changes to this bug.