Closed Bug 555393 Opened 14 years ago Closed 14 years ago

Intermittent failure in js1_5/Regress/regress-211590.js | Math.random should be random

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: philor, Unassigned)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

widen the allowed range
1.15 KB, patch
jorendorff
: review+
Details | Diff | Splinter Review 
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1269642466.1269644287.24637.gz
Linux mozilla-central debug test jsreftest on 2010/03/26 15:27:46
s: mv-moz2-linux-ix-slave07

begin test: js1_5/Regress/regress-211590.js
BUGNUMBER: 211590
STATUS: Math.random should be random
 FAILED! Math.random should be random : Expected value 'between 48% and 52%', Actual value ' is 47.980'
 FAILED! Math.random should be random : Expected value 'between 48% and 52%', Actual value ' is 47.980'
REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/mozilla-central-linux-debug-unittest-jsreftest/build/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-211590.js | Math.random should be random Expected value 'between 48% and 52%', Actual value ' is 47.980'  item 1
REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/mozilla-central-linux-debug-unittest-jsreftest/build/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-211590.js | Math.random should be random Expected value 'between 48% and 52%', Actual value ' is 47.980'  item 2
sayrer, jorendorff: any objection to just marking this test as random? or is of so little value that it should be disabled?

/me considers the irony of marking a test for randomness as random.
(In reply to comment #4)
> sayrer, jorendorff: any objection to just marking this test as random? or is of
> so little value that it should be disabled?

Fine with me. But note that this is likely a real bug!

The probability that any given run of this test will fail, *if* the implementation of Math.random is really correct, is about P=0.000061. I'm not sure how many times tinderbox runs this test every day, but the failures seem a little too frequent, don't they?
Based on
http://oduinn.com/blog/2010/09/09/infrastructure-load-for-august-2010/ let's say we're doing about 1000 changesets per month for which we'd notice this failure (since most try failures wouldn't get logged here).  We run this test 10 times per changeset (5 platforms * (opt, debug)).

If P=.000061 (per comment 6), we'd expect about 0.61 failures per month given those (very rough) numbers.  (Or, if that's P=.000061 for each of the two tests that the test does, then we'd expect about 1.22 failures per month.)

We've had seven failures over a period of unknown duration lasting at least 7 months.

So I don't think that implies anything is wrong.


Can we change the allowed range to be within 47.5 to 52.5 ?
Only six of the failures were on mozilla-central.  One was on electrolysis.

        Attachment #488278 -
        Flags: review?(jorendorff)

    
    

    
  
Comment on attachment 488278 [details] [diff] [review]
widen the allowed range

Fine with me.

        Attachment #488278 -
        Flags: review?(jorendorff) → review+

    
    

    
  
http://hg.mozilla.org/mozilla-central/rev/bd9113cacfc5
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Whiteboard: [orange]

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: