The purpose of cold launch testing is to measure the launch performance of an application as a new process. By running the cold launch tests from a fresh flash scenario, we include latency from an app's initial launch that may skew the time to be much longer than normal, whether this be from DB initialization or other IO operations. These measurements should be accounted for in the "first-time" suites, but not the cold launch ones. To achieve this, we should change the setup of the cold launch suite to launch an app and then close it before starting any of the actual test runs. This should eliminate the bulk of extra latency between first-time and cold-launch numbers.
Excellent - yeah I was thinking we would need to 'throw out' the first value because of this; this is an even better solution.
Created attachment 8557219 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/9
Attachment #8557219 - Flags: review?(rwood)
FWIW, if you need a simple term for this, I've heard it called "priming" in relationship to website caches. I've been calling our issue the same. But yeah, this is a much better solution than chucking the dirty result.
Me likely. I'll rename the method to `prime` before merging.
Summary: [Raptor] Pre-launch and close application in setup of cold launch phase → [Raptor] Prime application in setup of cold launch phase
Comment on attachment 8557219 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/9 Awesome. Tested by running the launch test on flame and the b2g emulator. Verified the app is primed/pre-launched successfully, and the data from the pre-launch is not included in the raptor.log and visualization, as expected.
Attachment #8557219 - Flags: review?(rwood) → review+
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.