Allow mochitest --run-until-failure to run forever

NEW
Assigned to

Status

enhancement
6 years ago
6 years ago

People

(Reporter: Mook, Assigned: Mook)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

With mochitest, there's no easy way to get --run-until-failure to run until the test actually fails; by default, it runs 30 times.

Workaround (untested):
  mach mochitest* --run-until-failure --repeat 9007199254740995
(This relies on floating point rounding problems, since --(magic) results in (magic) + 1)

The attached patch will default --run-until-failure to repeat forever; to get existing behaviour with the patch applied, use --run-until-failure --repeat 30

I'm also open to using magic values for this, e.g. --run-until-failure --repeat -1 or something.

Of course, maybe having an infinite-looping default is a bad idea and this should be wontfix.  In that case, at least I'd know :)
Attachment #785353 - Flags: review?(jmaher)
Comment on attachment 785353 [details] [diff] [review]
0001-mochitest-change-default-cap-of-run-until-failure-to.patch

Review of attachment 785353 [details] [diff] [review]:
-----------------------------------------------------------------

run-until-failure implies it will run forever until it finds a failure.  We should make a reasonable upper limit and encourage people to override this with --repeat.  Right now the limit is 30 cycles.  Should we make it 1000 by default?  While I don't like the --repeat=-1 as infinite, that is a common way programs operate.
Attachment #785353 - Flags: review?(jmaher) → review-
This is a little uglier in the code, but adds `--repeat forever` instead; it's only valid in conjunction with --run-until-failure.  The error output is a bit wrong in the invalid parameter case (it reports the right error, but _also_ reports no tests to run).
Attachment #785353 - Attachment is obsolete: true
Attachment #786143 - Flags: review?(jhammel)
Summary: Default mochitest --run-until-failure to run forever → Allow mochitest --run-until-failure to run forever
Comment on attachment 786143 [details] [diff] [review]
use --repeat forever

Ugh. Sorry, wrong reviewer.  I, umm, blame the j and the m.
Attachment #786143 - Flags: review?(jhammel) → review?(jmaher)
Comment on attachment 786143 [details] [diff] [review]
use --repeat forever

Review of attachment 786143 [details] [diff] [review]:
-----------------------------------------------------------------

this seems to be only for browser chrome, what about chrome and mochitest-plain?
Attachment #786143 - Flags: review?(jmaher) → review+
You need to log in before you can comment on or make changes to this bug.