If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Intermittent failure in js1_5/Regress/regress-211590.js | Math.random should be random

RESOLVED FIXED

Status

()

Core
JavaScript Engine
RESOLVED FIXED
8 years ago
5 years ago

People

(Reporter: philor, Unassigned)

Tracking

({intermittent-failure})

Trunk
x86
Linux
intermittent-failure
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

8 years ago
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1269642466.1269644287.24637.gz
Linux mozilla-central debug test jsreftest on 2010/03/26 15:27:46
s: mv-moz2-linux-ix-slave07

begin test: js1_5/Regress/regress-211590.js
BUGNUMBER: 211590
STATUS: Math.random should be random
 FAILED! Math.random should be random : Expected value 'between 48% and 52%', Actual value ' is 47.980'
 FAILED! Math.random should be random : Expected value 'between 48% and 52%', Actual value ' is 47.980'
REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/mozilla-central-linux-debug-unittest-jsreftest/build/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-211590.js | Math.random should be random Expected value 'between 48% and 52%', Actual value ' is 47.980'  item 1
REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/mozilla-central-linux-debug-unittest-jsreftest/build/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-211590.js | Math.random should be random Expected value 'between 48% and 52%', Actual value ' is 47.980'  item 2
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Comment 4

7 years ago
sayrer, jorendorff: any objection to just marking this test as random? or is of so little value that it should be disabled?

/me considers the irony of marking a test for randomness as random.
Comment hidden (Treeherder Robot)
(In reply to comment #4)
> sayrer, jorendorff: any objection to just marking this test as random? or is of
> so little value that it should be disabled?

Fine with me. But note that this is likely a real bug!

The probability that any given run of this test will fail, *if* the implementation of Math.random is really correct, is about P=0.000061. I'm not sure how many times tinderbox runs this test every day, but the failures seem a little too frequent, don't they?
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Based on
http://oduinn.com/blog/2010/09/09/infrastructure-load-for-august-2010/ let's say we're doing about 1000 changesets per month for which we'd notice this failure (since most try failures wouldn't get logged here).  We run this test 10 times per changeset (5 platforms * (opt, debug)).

If P=.000061 (per comment 6), we'd expect about 0.61 failures per month given those (very rough) numbers.  (Or, if that's P=.000061 for each of the two tests that the test does, then we'd expect about 1.22 failures per month.)

We've had seven failures over a period of unknown duration lasting at least 7 months.

So I don't think that implies anything is wrong.


Can we change the allowed range to be within 47.5 to 52.5 ?

Comment 10

7 years ago
Only six of the failures were on mozilla-central.  One was on electrolysis.
Comment hidden (Treeherder Robot)

Comment 12

7 years ago
Created attachment 488278 [details] [diff] [review]
widen the allowed range
Attachment #488278 - Flags: review?(jorendorff)
Comment on attachment 488278 [details] [diff] [review]
widen the allowed range

Fine with me.
Attachment #488278 - Flags: review?(jorendorff) → review+

Comment 14

7 years ago
http://hg.mozilla.org/mozilla-central/rev/bd9113cacfc5
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Keywords: intermittent-failure
Whiteboard: [orange]
You need to log in before you can comment on or make changes to this bug.