Invert tcheckerboard (and tcheckerboard2) output and make it cumulative instead of average

RESOLVED FIXED in Firefox 15

Status

()

Firefox for Android
General
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: kats, Assigned: kats)

Tracking

unspecified
Firefox 15
All
Android
Points:
---

Firefox Tracking Flags

(firefox15 fixed, blocking-fennec1.0 -)

Details

Attachments

(1 attachment, 1 obsolete attachment)

Right now the output means 1.0 is no checkerboarding and 0.0 is all checkerboarding. We should flip this so that when viewing the graphs on the graph server the y-axis scale will get more detailed as we drive down checkerboarding and we can better see smaller regressions.

Also it just seems more reasonable for a test called "checkerboarding" to measure checkerboarding as opposed to not-checkerboarding.
good call.  We should be able to change the test cases themselves and ignore the rest of the automation and buildbot scripts for this.
blocking-fennec1.0: ? → -
We might also want to change the scale from [0,1] to [0,100] instead. Right now when I click on a data point on the graph server it always reports the score as "0", which I assume is because it doesn't like fractional values. Scaling the values up would avoid this.
Created attachment 617007 [details] [diff] [review]
Patch
Attachment #617007 - Flags: review?(jmaher)
Comment on attachment 617007 [details] [diff] [review]
Patch

Review of attachment 617007 [details] [diff] [review]:
-----------------------------------------------------------------

this looks fine!  I assume this will work for both tcheckerboard tests?
Attachment #617007 - Flags: review?(jmaher) → review+
Yeah I pushed it to try at https://tbpl.mozilla.org/?tree=Try&rev=2050322f4a9a and it looked fine.
Assignee: nobody → bugmail.mozilla
Blocks: 748149
Duplicate of this bug: 748174
Blocks: 748736
No longer blocks: 748149
Summary: Invert tcheckerboard (and tcheckerboard2) output → Invert tcheckerboard (and tcheckerboard2) output and make it cumulative instead of average
Duplicate of this bug: 748817
Created attachment 618351 [details] [diff] [review]
Patch (v2)

New patch that also reports the sum instead of average. The "inversion" still applies, so that lower numbers are better.
Attachment #617007 - Attachment is obsolete: true
Attachment #618351 - Flags: review?(jmaher)
Comment on attachment 618351 [details] [diff] [review]
Patch (v2)

Review of attachment 618351 [details] [diff] [review]:
-----------------------------------------------------------------

looks good, thanks for pushing these patches through.
Attachment #618351 - Flags: review?(jmaher) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/c978a9f2d1d8
status-firefox15: --- → fixed
Target Milestone: --- → Firefox 15
https://hg.mozilla.org/mozilla-central/rev/c978a9f2d1d8
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.