943109 - Make the performance testing framework report app startup times to datazilla

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

In bug 915156, gaia's performance testing framework (ala 'make test-perf') is being ported to the new js-marionette runner.

Once that's done, tests should be added to it to report app startup times, the same that are currently reported by b2gperf.  We'd like to do this so we can retire b2gperf; there's not a good reason to have two separate gaia performance harnesses in use.

We don't need the fps metrics that b2gperf is reporting; Eideticker will supply those.  For developer testing, engineers can turn on the fps counter.

Comment 1

[Julien Wajsberg [:julienw]]

This is actually already done :)

see [1] and [2];

[1] is essentially the same as the existing b2gperf.
[2] is doing more: by using a standardized event "startup-path-done", we record all events sent by the app from the launch to this event, and record all timestamps in the process, and report them all separately.

[1] https://github.com/mozilla-b2g/gaia/blob/master/tests/performance/startup_test.js
[2] https://github.com/mozilla-b2g/gaia/blob/master/tests/performance/startup_events_test.js

Comment 2

[Jonathan Griffin (:jgriffin)]

Great, then we should be able to turn off b2gperf after the new perf system is reporting to datazilla reliably.