Closed Bug 479518 Opened 11 years ago Closed 10 years ago

Increase the reftest timeout, at least for tests we know are slow

Categories

(Testing :: Reftest, enhancement)

x86
macOS
enhancement
Not set

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 471579
mozilla1.9.2a1

People

(Reporter: jruderman, Unassigned)

References

Details

Attachments

(1 obsolete file)

<ted> i think we need to declare war on intermittent tests
<Jesse> i think we should increase the reftest timeout
<Jesse> to like 10 minutes
<ted> just because you like to write long-running tests? :)
<Jesse> so we can deal with slow tests out of band, not by having them fail intermittently
<ted> what if we let reftests specify their own timeouts, or something?
<ted> or at least say "i'm a long-running test, give me more time"
<Jesse> i could go for that

Allowing a test to be slow is more efficient than splitting it into multiple smaller tests, each of which has its own overhead.
(Not to mention things like bug 473911 really *can't* be split up!)
Whiteboard: [orange]
Intermittent orange is not the right way to avoid having tests that are too slow.
Attachment #380560 - Flags: review?
Attachment #380560 - Flags: review? → review?(dbaron)
See also bug 471579, which has a patch to make the timeout vary by platform.
From the patch I have in bug 471579, you can adjust the timeout in the command line to runreftest.py.  Unless there is a need for this being hardcoded to 10 minutes, we should stick with a flexible solution.
Flexibility is great, but we also need a sane default, especially since Tinderbox uses the default.
how about I adjust my patch in bug 471579:
https://bug471579.bugzilla.mozilla.org/attachment.cgi?id=380409

to change:
gTimeoutValue = 10000; //ms

to be similar to your value:
gTimeoutValue = 10 * 60 * 1000; //10 minutes


We would also have to adjust my code (in the same patch) in runreftest.py where it accepts --timeout and currently has a default of 10000.  

Ted, could you offer some insight here.
Ten minutes seems a bit long; I can't believe two or three minutes wouldn't be more than adequate.
Maybe five minutes would be better; the difference between five and ten could be noticeable in fixing tinderbox bustage.  Other than that, it really shouldn't matter.
Attachment #380560 - Flags: review?(dbaron) → review+
Comment on attachment 380560 [details] [diff] [review]
increase reftest timeout from 10 seconds to 10 minutes
[Superseded by bug 471579]

r=dbaron
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → DUPLICATE
Target Milestone: --- → mozilla1.9.2a1
Version: unspecified → Trunk
Duplicate of bug: 471579
Attachment #380560 - Attachment description: increase reftest timeout from 10 seconds to 10 minutes → increase reftest timeout from 10 seconds to 10 minutes [Superseded by bug 471579]
Attachment #380560 - Attachment is obsolete: true
Whiteboard: [orange]
You need to log in before you can comment on or make changes to this bug.