1126567 - [Raptor] Prime application in setup of cold launch phase

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect)

Description

[:Eli Perelman]

The purpose of cold launch testing is to measure the launch performance of an application as a new process. By running the cold launch tests from a fresh flash scenario, we include latency from an app's initial launch that may skew the time to be much longer than normal, whether this be from DB initialization or other IO operations. These measurements should be accounted for in the "first-time" suites, but not the cold launch ones.

To achieve this, we should change the setup of the cold launch suite to launch an app and then close it before starting any of the actual test runs. This should eliminate the bulk of extra latency between first-time and cold-launch numbers.

Comment 1

[Robert Wood [:rwood]]

Excellent - yeah I was thinking we would need to 'throw out' the first value because of this; this is an even better solution.

Comment 2

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/9

Comment 3

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

FWIW, if you need a simple term for this, I've heard it called "priming" in relationship to website caches. I've been calling our issue the same. But yeah, this is a much better solution than chucking the dirty result.

Comment 4

[:Eli Perelman]

Me likely. I'll rename the method to `prime` before merging.

Comment 5

[Robert Wood [:rwood]]

Comment on attachment 8557219 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/9

Awesome. Tested by running the launch test on flame and the b2g emulator. Verified the app is primed/pre-launched successfully, and the data from the pre-launch is not included in the raptor.log and visualization, as expected.

Comment 6

[:Eli Perelman]

Merged. https://github.com/mozilla-b2g/raptor/commit/dbcba0a3f866d5fd06a6089654ecf7a2bcc52af4