697128 - Windows NT 6.1 results results are low, as before introducing system graphics details

Status: VERIFIED INVALID

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Mihaela Velimiroviciu (:mihaelav)]

According to Bug 693247 comment #18, introducing the system graphics details to the endurance test reports caused increases in the resident memory for endurance tests; 
Windows NT 6.1 resuls were missing since the graphics results were added until Oct 17 and when they appeared again, they were still low, as before introducing the graphics details (average 74 MB). They should have been higher, comparable with other Windows versions.

Mozmill results reports:
http://mozmill-release.brasstacks.mozilla.com/#/endurance/reports?branch=10.0&platform=Windows NT&from=2011-10-18&to=2011-10-25

Note: For other Win systems, the values are higher (average 115 MB vs 74 MB for Win NT 5.1) since the mentioned environment change.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We're raising bugs when memory usage drops now? :)

While I think it's worth highlighting these improvements in memory usage, I don't think they should be in the form of a bug. Perhaps we should add them to the highlights section of the scrumpad?

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I've also just noticed that you're comparing Windows NT 5.1 with 6.1, as 6.1 was unavailable for some time. In fact, the period of unavailability of 6.1 matches the increase in memory usage.

I would propose that we don't substitute any other platforms when gathering data as it introduces too much variance. We should report unavailable platforms, and simply not log them for the duration.

Comment 3

[Maniac Vlad Florin (:vladmaniac)]

This is invalid then based on comment 1 - please change if otherwise