901201 - Allow mochitest --run-until-failure to run forever

Status: NEW

(Testing :: Mochitest, enhancement)

Description

[:Mook]

Attachment: 0001-mochitest-change-default-cap-of-run-until-failure-to.patch

With mochitest, there's no easy way to get --run-until-failure to run until the test actually fails; by default, it runs 30 times.

Workaround (untested):
  mach mochitest* --run-until-failure --repeat 9007199254740995
(This relies on floating point rounding problems, since --(magic) results in (magic) + 1)

The attached patch will default --run-until-failure to repeat forever; to get existing behaviour with the patch applied, use --run-until-failure --repeat 30

I'm also open to using magic values for this, e.g. --run-until-failure --repeat -1 or something.

Of course, maybe having an infinite-looping default is a bad idea and this should be wontfix.  In that case, at least I'd know :)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 785353 [details] [diff] [review]
0001-mochitest-change-default-cap-of-run-until-failure-to.patch

Review of attachment 785353 [details] [diff] [review]:
-----------------------------------------------------------------

run-until-failure implies it will run forever until it finds a failure.  We should make a reasonable upper limit and encourage people to override this with --repeat.  Right now the limit is 30 cycles.  Should we make it 1000 by default?  While I don't like the --repeat=-1 as infinite, that is a common way programs operate.

Comment 2

[:Mook]

Attachment: use --repeat forever

This is a little uglier in the code, but adds `--repeat forever` instead; it's only valid in conjunction with --run-until-failure.  The error output is a bit wrong in the invalid parameter case (it reports the right error, but _also_ reports no tests to run).

Comment 3

[:Mook]

Comment on attachment 786143 [details] [diff] [review]
use --repeat forever

Ugh. Sorry, wrong reviewer.  I, umm, blame the j and the m.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 786143 [details] [diff] [review]
use --repeat forever

Review of attachment 786143 [details] [diff] [review]:
-----------------------------------------------------------------

this seems to be only for browser chrome, what about chrome and mochitest-plain?

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.