Closed Bug 972440 Opened 7 years ago Closed 7 years ago

Provide average memory measurements

Categories

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect, P2)

x86_64
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED
1.4 S1 (14feb)

People

(Reporter: stas, Assigned: stas)

References

Details

(Keywords: perf, Whiteboard: [c=automation p= s= u=])

Attachments

(1 file, 1 obsolete file)

Attached patch Memory average (obsolete) — Splinter Review
Let's provide an average of all the memory measurements gathered by test-perf.
Attachment #8375605 - Flags: review?(hub)
Keywords: perf
Whiteboard: [c=automation p= s= u=]
Do you mind making a pull request on Github? This makes it easy for us.

Then all you need is to add attachment and past the Pull Request URL as content.

Thanks !
Comment on attachment 8375605 [details] [diff] [review]
Memory average

Review of attachment 8375605 [details] [diff] [review]:
-----------------------------------------------------------------

Love it otherwise.

::: tests/reporters/jsonmozperf.js
@@ +105,5 @@
>  }
>  
> +function averageObjects(arr) {
> +  if (arr.length === 0) {
> +    return 0;

I think we should return null here instead of zero.
Attachment #8375605 - Flags: review?(hub) → review-
Attached file Link to pull request
Thanks for the review, Hubert!  Here's the link to the pull request.  I also addressed your comment about changing 0 to null.  I absent-mindedly copied the line from the average function above it.
Assignee: nobody → stas
Attachment #8375605 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Comment on attachment 8375911 [details] [review]
Link to pull request

r=me
Attachment #8375911 - Flags: review+
Commit: https://github.com/mozilla-b2g/gaia/commit/9b34f79019404c355575c3a78583196a4e3cec92
Merge: https://github.com/mozilla-b2g/gaia/commit/1c571fed331873e6a8778aa02689f2d302b195aa

Thanks for the review!
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Priority: -- → P2
Target Milestone: --- → 1.4 S1 (14feb)
Average is not ideal - hides the outliers and lets them bias the interpreted result.

We should look to how talos handles things like this. IIRC it used to drop highest and lowest, and then provide avg, std deviation and median.
Where will these results show up - in Datazilla? How can i access them as a developer locally?
we still don't report to Datazilla. See bug 962040
You need to log in before you can comment on or make changes to this bug.