Benchmark.js patterned high margin of error

UNCONFIRMED
Unassigned

Status

()

P3
normal
UNCONFIRMED
2 years ago
a year ago

People

(Reporter: timolawl, Unassigned)

Tracking

({triage-deferred})

54 Branch
triage-deferred
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

2 years ago
Created attachment 8875346 [details]
firefox variance.PNG

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170605125726

Steps to reproduce:

Spawn a webworker to run benchmark.js and compare the performance of two test cases.


Actual results:

Benchmark.js test case margin of error is extremely high for the first test if both test cases are the same. If the tests differ (i.e., the first test is slower), then the high variance seems to shift to the second test.


Expected results:

Variance should be roughly the same at low values. This is the result in Firefox 53. In the attached image, Firefox 54 is on the main window on the left, and Firefox 53 is the result on the right.
Component: Untriaged → JavaScript Engine
Product: Firefox → Core
Keywords: triage-deferred
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.