799197 - throw an error from talos if we do not collect all the counters in xperf that we intend to collect

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

oh errors, we need more of ye.

We found a changeset that didn't collect any xperf data, but ran through the test just fine.  We know what counters we want to collect, so we need to enforce this collection and throw an error if we did not collect our predefined set.

Maybe we shrink our set so we can be green.

Comment 1

[Jeff Hammel]

We currently don't throw any errors if we don't collect counter data that we expect to collect:

http://hg.mozilla.org/build/talos/file/e351f3aa6ecc/talos/output.py#l207

Do we want to change this across the board?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we probably should, otherwise what is the point of collecting it?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: remove unused counters and throw an error if we do not report a value for one we are trying to collect (1.0)

nice and simple, green on try.

Comment 4

[Jeff Hammel]

Comment on attachment 670507 [details] [diff] [review]
remove unused counters and throw an error if we do not report a value for one we are trying to collect (1.0)

looks good to me :)

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

http://hg.mozilla.org/build/talos/rev/524c6ff1736b