Introduce new performance test engine "raptor" in Gaia

RESOLVED FIXED

Status

Firefox OS
Gaia::PerformanceTest
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: Eli, Assigned: Eli)

Tracking

({perf})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [systemsfe])

Attachments

(2 attachments)

(Assignee)

Description

4 years ago
The performance tests of make test-perf rely on a mix of atom scripts, helper scripts, and event listeners in order to collect performance data. We should refactor the performance testing engine to allow us to collect much more efficiently, by gathering markers instead of binding events and doing in-test calculations.
Whiteboard: [systemsfe]
(Assignee)

Updated

4 years ago
Summary: Refactor Gaia performance test to enable capturing of metrics generated via User Timing markers → Introduce new performance test engine "raptor" to capture of metrics generated via User Timing markers
(Assignee)

Comment 1

4 years ago
Created attachment 8539382 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/80

Adding the gaia-raptor module for new performance testing.
Attachment #8539382 - Flags: review?(kgrandon)
Comment on attachment 8539382 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/80

It looks like the raptor code has already been reviewed in bug 1090299, so I'm fine with the manifest bump. Thanks!
Attachment #8539382 - Flags: review?(kgrandon) → review+
(Assignee)

Comment 4

4 years ago
Created attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Alexandre, could you review my changes for profile generation for our new performance tests?

Rob, could you review my changes for the addition of new raptor tests?
Attachment #8539418 - Flags: review?(rwood)
Attachment #8539418 - Flags: review?(poirot.alex)
(Assignee)

Updated

4 years ago
Summary: Introduce new performance test engine "raptor" to capture of metrics generated via User Timing markers → Introduce new performance test engine "raptor" in Gaia
Attachment #8539418 - Flags: review?(poirot.alex) → review+

Comment 5

4 years ago
Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

See comment in PR.
Attachment #8539418 - Flags: review?(rwood) → review-
(Assignee)

Comment 6

4 years ago
Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Thanks for catching that. r? again.
Attachment #8539418 - Flags: review- → review?(rwood)

Comment 7

4 years ago
Comment on attachment 8539418 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26918

Awesome, was able to generate the raptor gaia profile, and run the tests successfully on Flame. Noticed the update to gaia_node_modules.revision was removed in your updated PR, so R+ on assumption that was intentional and not required (?).
Attachment #8539418 - Flags: review?(rwood) → review+
(Assignee)

Comment 8

4 years ago
(In reply to Robert Wood [:rwood] from comment #7)
> Noticed the update to gaia_node_modules.revision was removed in your updated PR

Yeah, turns out gaia-node-modules was bumped again before this patch landed, so the revision Gaia is using already contains gaia-raptor now.

Comment 9

4 years ago
(In reply to :Eli Perelman from comment #8)
> (In reply to Robert Wood [:rwood] from comment #7)
> > Noticed the update to gaia_node_modules.revision was removed in your updated PR
> 
> Yeah, turns out gaia-node-modules was bumped again before this patch landed,
> so the revision Gaia is using already contains gaia-raptor now.

Thanks Eli.

Update: The boot_test actually times out on my Flame, see comment/log the PR. Sorry I didn't spot that earlier. Ran it three times, same result. Thanks.
(Assignee)

Comment 10

4 years ago
Try run: https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=ee6c169cd9f4
In master: https://github.com/mozilla-b2g/gaia/commit/ab41c35eed35404b632a4841714b81b56364ba32
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.