904642 - Talos Tp5 Responsiveness results are rounded to the nearest millisecond, causing aliasing

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Matt Brubeck (:mbrubeck)]

The Talos Tp Responsiveness measurement (bug 631571) is always rounded to integer milliseconds.  This causes aliasing, which can lead to false regression alarms or hide real regressions, as discussed in bug 755633:

http://graphs.mozilla.org/graph.html#tests=[[275,52,22]]&sel=1372099976003.843,1374081573273.4675,37.296086316668124,47.35195223845582&displayrange=90&datatype=running

It would be better to report these measurements with microsecond precesion when possible.  (They could still be in units of ms for compatibility with existing data, but the output should be floating point instead of integer.)

Comment 1

[(not currently active) Ted Mielczarek]

Currently the underlying measurements are being printed as integer ms:
http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/EventTracer.cpp#142

But it wouldn't be hard to change that (it's just calling TimeDuration::ToSecondsSigDigits and truncating the result). We'd need to change the Talos code that parses the output to deal with this first.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: fix talos to support floating point responsiveness data (1.0)

a small, but necessary change

Comment 3

[(not currently active) Ted Mielczarek]

Comment on attachment 793659 [details] [diff] [review]
fix talos to support floating point responsiveness data (1.0)

Review of attachment 793659 [details] [diff] [review]:
-----------------------------------------------------------------

Close, but looks like you need to fix the usage of int() here as well:
http://hg.mozilla.org/build/talos/file/2a32cc64e7ff/talos/output.py#l99

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: fix talos to support floating point responsiveness data (1.1)

updated with review feedback, tested on try and simulated the floating point values coming from the log file.

Comment 5

[(not currently active) Ted Mielczarek]

Once you get this rolled out to production we should fix the underlying code to print floating point measurements.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

http://hg.mozilla.org/build/talos/rev/a559a437b340

marking this as leave-open so we can fix this in the platform.  Who could pick up that work?  or more specifically where could I look to fix that?

Comment 7

[(not currently active) Ted Mielczarek]

I have a patch in bug 927355 that fixes the platform. Sample output looks like:
MOZ_EVENT_TRACE sample 1381922313420 143.015346