2032007 - Profile full runs of Speedometer 3 on Android

Status: ASSIGNED

(Testing :: Performance, task, P3)

Description

[Abhishek Nimalan [:animalan]]

Profile entire runs of Speedometer 3 on Android with Simpleperf. In Bug 1978586, we collect per-suite profiles (individual profiles for Editor-TipTap, Perf-Dashboard, etc.). We should also have a way of profiling the entire Speedometer 3 benchmark with Simpleperf. The most powerful and flexible way to do this is by profiling the entire Browsertime instance that runs Speedometer 3. At a high level, profling 5 full Sp3 runs would occur like this:

Start simpleperf profiler

   Start Browsertime

       Start browser (cycle 1/5) 
           Open page tab
               Start sp3 benchmark
               Stop sp3 benchmark
           Close page tab
       Stop browser  (cycle 1/5)
    
       Start browser (cycle 2/5) 
           Open page tab
               Start sp3 benchmark
               Stop sp3 benchmark
           Close page tab
       Stop browser  (cycle 2/5)

                    .
                    .
                    .  x 3

   Stop Browsertime

Stop Simpleperf profiler

This major benefit of this approach (over profiling within Browsertime as is done in Bug 1978586) is we are able to capture multiple browser cycles in a single profile. We should be using the Simpleperf binary directly for this (rather than app_profiler.py) as it will allow us to profile all processes on the system (as well as symbolicate kernel frames).

One drawback to this approach is profiles can get large very quickly. Since these profiles need to be processed and symbolicated, we can end up running into memory issues in CI (more so with our BitBar devices than LambdaTest) when capturing multiple Speedometer 3 runs. We'll need to investigate how to deal with this.