Closed Bug 1210704 Opened 4 years ago Closed 4 years ago

Add service worker benchmark for Flame and Aries

Categories

(Firefox OS Graveyard :: Performance, defect, P2)

ARM
Gonk (Firefox OS)
defect

Tracking

(tracking-b2g:backlog)

RESOLVED FIXED
tracking-b2g backlog

People

(Reporter: bobby.chien+bugzilla, Assigned: azasypkin)

References

()

Details

Attachments

(1 file)

As gaia development, I would like to know what is performance of service worker. Also, I want to track performance regression.

Goal:
Integrate service worker benchmark with raptor, and perform benchmark per daily based.

Devices:
- Flame
- Aries
ni? myself to do the following:

* Remove manual "Install Service Worker" step from benchmarking app [1];

* Find out where in Gaia tree we can place this app. Looks like "dev_apps" folder is going to die (see bug 1133711), so we need long term plan.

[1] https://github.com/nga-crash-tests/service-worker-benchmarks
Flags: needinfo?(azasypkin)
[Tracking Requested - why for this release]:
(In reply to Oleg Zasypkin [:azasypkin][⏰UTC+1] from comment #1)
> * Find out where in Gaia tree we can place this app. Looks like "dev_apps"
> folder is going to die (see bug 1133711), so we need long term plan.

Discussing in bug 1133711.
See Also: → 1133711
Depends on: 1219176
Trying to make dynamically preloaded apps work with Raptor in bug 1219176.
Flags: needinfo?(azasypkin)
bug 1219176 is in review now, so I can start preparing the test app itself here.
Assignee: nobody → azasypkin
Status: NEW → ASSIGNED
Blocks: 1158848
See Also: → 1178756
Depends on: 1225117
Depends on: 1225121
Waiting for bug 1225121 to be resolved.
Status: ASSIGNED → NEW
Comment on attachment 8686576 [details] [review]
[gaia] azasypkin:bug-1210704-sw-benchmarks > mozilla-b2g:master

Okay, all dependencies are resolved and it seems working fine for me locally.

Hey Eli, how does it look to you? Could you please check if it works fine for the Raptor Dashboard as well?

I use the following commands to run 3 benchmarking apps that are included into the package:

* raptor test coldlaunch --app fxos-performance.github.io --entry-point service-worker-baseline --runs 30

* raptor test coldlaunch --app fxos-performance.github.io --entry-point service-worker-empty --runs 30

* raptor test coldlaunch --app fxos-performance.github.io --entry-point service-worker-simple-fetch --runs 30

Thanks!
Attachment #8686576 - Flags: review?(eperelman)
Comment on attachment 8686576 [details] [review]
[gaia] azasypkin:bug-1210704-sw-benchmarks > mozilla-b2g:master

Looks good to me. I'll try and get this up in automation next week some time.
Attachment #8686576 - Flags: review?(eperelman) → review+
ni to add these apps to coldlaunch automation.
Status: NEW → ASSIGNED
Flags: needinfo?(eperelman)
(In reply to :Eli Perelman from comment #9)
> Comment on attachment 8686576 [details] [review]
> [gaia] azasypkin:bug-1210704-sw-benchmarks > mozilla-b2g:master
> 
> Looks good to me. I'll try and get this up in automation next week some time.

Thanks! So let's see if it sticks.

Master: https://github.com/mozilla-b2g/gaia/commit/d2ff5923a622b398d4f55b4f3d93b7728028cc51

Not marking this bug as FIXED until we have automation support for this.
Benchmark apps have been added to Jenkins automation configuration for:

flame-kk-1024.b2g-inbound.tinderbox.perf.coldlaunch.v2
flame-kk-512.b2g-inbound.tinderbox.perf.coldlaunch.v2

Apps have also been added to the dashboard configuration for raptor.mozilla.org. Awaiting builds to ensure tests and running and reporting successfully.
Flags: needinfo?(eperelman)
(In reply to :Eli Perelman from comment #12)
> Benchmark apps have been added to Jenkins automation configuration for:
> 
> flame-kk-1024.b2g-inbound.tinderbox.perf.coldlaunch.v2
> flame-kk-512.b2g-inbound.tinderbox.perf.coldlaunch.v2
> 
> Apps have also been added to the dashboard configuration for
> raptor.mozilla.org. Awaiting builds to ensure tests and running and
> reporting successfully.

Sweet, thanks a lot for your help, Eli!
I see at least two datapoints at Dashboard so marking bug as resolved :)
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Hey Mike,

Could you please include apps from comment 8 to the daily Raptor runs on Aries as well?

We have had them for Flame for some time already and didn't have any noticeable issues so far.

Thanks!
Flags: needinfo?(mlien)
See Also: → 1196205
just add three tests from comment 8, leave ni to see if results are posted to dashboard correctly
Now Aries has three service worker cases and records
Flags: needinfo?(mlien)
(In reply to Mike Lien[:mlien] from comment #17)
> Now Aries has three service worker cases and records

Nice, thanks a lot!
You need to log in before you can comment on or make changes to this bug.