1088131 - Introduce new performance test engine "raptor" in Gaia

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect)

Description

[:Eli Perelman]

The performance tests of make test-perf rely on a mix of atom scripts, helper scripts, and event listeners in order to collect performance data. We should refactor the performance testing engine to allow us to collect much more efficiently, by gathering markers instead of binding events and doing in-test calculations.

Comment 1

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/80

Adding the gaia-raptor module for new performance testing.

Comment 2

[Kevin Grandon :kgrandon]

Comment on attachment 8539382 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/80

It looks like the raptor code has already been reviewed in bug 1090299, so I'm fine with the manifest bump. Thanks!

Comment 3

[:Eli Perelman]

Comment on attachment 8539382 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/80

In gaia_node_modules master: https://github.com/mozilla-b2g/gaia-node-modules/commit/c2c0768525fc59c7b358a25b4152a21c37537cad

Comment 4

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Alexandre, could you review my changes for profile generation for our new performance tests?

Rob, could you review my changes for the addition of new raptor tests?

Comment 5

[Robert Wood [:rwood]]

Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

See comment in PR.

Comment 6

[:Eli Perelman]

Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Thanks for catching that. r? again.

Comment 7

[Robert Wood [:rwood]]

Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Awesome, was able to generate the raptor gaia profile, and run the tests successfully on Flame. Noticed the update to gaia_node_modules.revision was removed in your updated PR, so R+ on assumption that was intentional and not required (?).

Comment 8

[:Eli Perelman]

(In reply to Robert Wood [:rwood] from comment #7)
> Noticed the update to gaia_node_modules.revision was removed in your updated PR

Yeah, turns out gaia-node-modules was bumped again before this patch landed, so the revision Gaia is using already contains gaia-raptor now.

Comment 9

[Robert Wood [:rwood]]

(In reply to :Eli Perelman from comment #8)
> (In reply to Robert Wood [:rwood] from comment #7)
> > Noticed the update to gaia_node_modules.revision was removed in your updated PR
> 
> Yeah, turns out gaia-node-modules was bumped again before this patch landed,
> so the revision Gaia is using already contains gaia-raptor now.

Thanks Eli.

Update: The boot_test actually times out on my Flame, see comment/log the PR. Sorry I didn't spot that earlier. Ran it three times, same result. Thanks.

Comment 10

[:Eli Perelman]

Try run: https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=ee6c169cd9f4
In master: https://github.com/mozilla-b2g/gaia/commit/ab41c35eed35404b632a4841714b81b56364ba32