Closed
Bug 849558
Opened 11 years ago
Closed 11 years ago
Allow easier debug of talos tests outside of the talos environment
Categories
(Datazilla Graveyard :: Metrics, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: avih, Assigned: avih)
Details
Attachments
(1 file, 1 obsolete file)
7.18 KB,
patch
|
jmaher
:
review+
|
Details | Diff | Splinter Review |
As suggested at bug 845943 comment #7, attached is talos-debug.js. Just include it from any talos test, run the test in a browser, and tpRecordTime will display the summary. Here's an excerpt from the file itself: /****************************************************************** * window.talosDebug provides some statistical functions * (sum, average, median, stddev) and a tpRecordTime method which * reports some statistics about the data set, including detected * stability point of the data (detecting first few noisy elements). * If tpRecordTime doesn't exist globally (i.e. running outside of * talos, e.g. in a browser), then it points it to window.talosDebug.tpRecordTime * Can be controlled by few properties (disable display, hardcoded * stability point, etc) *****************************************************************/ The warmup-preiod detection was fine tuned with the noise patterns of the tscroll tests (in various browsers), and while completely unscientific, provides surprisingly useful and consistent results (e.g. usually shows a value of 1-10 for the tscroll tests, which I manually confirmed as valid). We might need to improve it if this proves useful for tests other than tscroll, but for now, it can serve as a starting point.
Attachment #723117 -
Flags: review?(jmaher)
Assignee | ||
Comment 1•11 years ago
|
||
Also, here's a sample output of the SVG scroll test on my system. Notice how after (automatically) skipping 2 items (of 44), stddev goes from 10.8% down to 2.7%: ------------------- Count: 44 Average: 16.38 Median: 16.88 StdDev: 1.77 (10.83% of average) Warmup auto-detected: 2 After ignoring first 2 items: Count: 42 Average: 16.70 Median: 16.89 StdDev: 0.47 (2.79% of average) Recorded: 5.68, 13.59, [warmed-up:], 17.26, 16.93, 15.95, 17.05, 16.98, 16.08, 16.94, 16.91, 16.39, 16.72, 17.03, 16.03, 16.96, 17.72, 16.03, 17.04, 17.07, 15.92, 16.99, 17.09, 15.94, 16.99, 16.99, 16.01, 16.99, 17.08, 15.96, 16.84, 17.21, 16.11, 16.86, 16.92, 16.21, 16.86, 16.88, 16.19, 16.88, 16.97, 16.21, 16.74, 17.11, 16.21 Stddev from item NN to last: 1.77, 0.66, [warmed-up:], 0.47, 0.46, 0.47, 0.46, 0.46, 0.46, 0.46, 0.46, 0.47, 0.47, 0.48, 0.49, 0.48, 0.49, 0.45, 0.44, 0.44, 0.45, 0.43, 0.43, 0.43, 0.41, 0.42, 0.42, 0.40, 0.41, 0.41, 0.38, 0.39, 0.37, 0.35, 0.37, 0.37, 0.37, 0.38, 0.40, 0.39, 0.42, 0.44, 0.45, 0.64, 0.00
Attachment #723117 -
Attachment is obsolete: true
Attachment #723117 -
Flags: review?(jmaher)
Attachment #723203 -
Flags: review?(jmaher)
Comment 2•11 years ago
|
||
Comment on attachment 723203 [details] [diff] [review] v2 - minor fix: also handle single, non-string reported value Review of attachment 723203 [details] [diff] [review]: ----------------------------------------------------------------- I would add a license header to this.
Attachment #723203 -
Flags: review?(jmaher) → review+
Assignee | ||
Comment 3•11 years ago
|
||
Landed talos-debug.js: https://hg.mozilla.org/build/talos/rev/053f69f5d92d Added an MPL clause: https://hg.mozilla.org/build/talos/rev/f443093a8ce3
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•