972440 - Provide average memory measurements

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect, P2)

Description

[Staś Małolepszy :stas]

Attachment: Memory average

Let's provide an average of all the memory measurements gathered by test-perf.

Comment 1

[Hubert Figuiere [:hub]]

Do you mind making a pull request on Github? This makes it easy for us.

Then all you need is to add attachment and past the Pull Request URL as content.

Thanks !

Comment 2

[Hubert Figuiere [:hub]]

Comment on attachment 8375605 [details] [diff] [review]
Memory average

Review of attachment 8375605 [details] [diff] [review]:
-----------------------------------------------------------------

Love it otherwise.

::: tests/reporters/jsonmozperf.js
@@ +105,5 @@
>  }
>  
> +function averageObjects(arr) {
> +  if (arr.length === 0) {
> +    return 0;

I think we should return null here instead of zero.

Comment 3

[Staś Małolepszy :stas]

Attachment: Link to pull request

Thanks for the review, Hubert!  Here's the link to the pull request.  I also addressed your comment about changing 0 to null.  I absent-mindedly copied the line from the average function above it.

Comment 4

[Hubert Figuiere [:hub]]

Comment on attachment 8375911 [details] [review]
Link to pull request

r=me

Comment 5

[Staś Małolepszy :stas]

Commit: https://github.com/mozilla-b2g/gaia/commit/9b34f79019404c355575c3a78583196a4e3cec92
Merge: https://github.com/mozilla-b2g/gaia/commit/1c571fed331873e6a8778aa02689f2d302b195aa

Thanks for the review!

Comment 6

[Dietrich Ayala (:dietrich)]

Average is not ideal - hides the outliers and lets them bias the interpreted result.

We should look to how talos handles things like this. IIRC it used to drop highest and lowest, and then provide avg, std deviation and median.

Comment 7

[Dietrich Ayala (:dietrich)]

Where will these results show up - in Datazilla? How can i access them as a developer locally?

Comment 8

[Hubert Figuiere [:hub]]

we still don't report to Datazilla. See bug 962040