1178756 - [ServiceWorkers] Create performance test plan

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[Francisco Jordano [:arcturus] [:francisco]]

We have a meta bug for tracking performance in SW:

https://bugzilla.mozilla.org/show_bug.cgi?id=1158848

With some bugs assigned. We also need to create a series of performance test to measure and share this data with the DOM content team.

Comment 1

[Salvador de la Puente González [:salva]]

Taking to ensure we talk about this before the end of the spring (i.e. today)

Comment 2

[Salvador de la Puente González [:salva]]

# Service Worker & ServiceWorkerWare performance measurements

Notice these measurements demonstrate only the local increase in time. This
should be multiplied by the average time for applications using this technology.

## Related to service worker impact on the main loop

The intention is to determine the cost for the main loop into create the
service worker and see if there is an impact on the max FPS (60fps).

 1. Measure the time to send an intercepted XHR.

These tests should be done for single-core and dual-core devices. Curves for
multicore would be nice-to-have for the remaining measurements.

## Related to client requests for resources

We are going to measure a complete request which includes waking the worker up
overhead but this should be the average use case. So we should ensure that
each request awakes the worker.

 1. Measure image simple request (non cached).
 2. Measure image simple request (cached in http cache / pinned cache).
 3. Measure image intercepted request (stored as base64 in the sw).
 4. Measure image intercepted request (in an offline cache).
 5. Measure image intercepted request (via sww, base64).
 6. Measure image intercepted request (via sww, offline cache).

Repeat the tests ensuring the sw in running so we can measure times without
creation overhead as well. Not counting this time is possible.

Same tests must be performed from XHR and fetch() instead of resources.

## Related to communications via post message

NOTE: Don't know if needed since
[there are performance measures](https://hacks.mozilla.org/2015/07/how-fast-are-web-workers/)
for this but related with workers in general.

Now we are going to measure communication between client and sw and viceversa.
Again, this measures should include the waking up overhead.

 1. Measure sending a small payload to the sw.
 2. Measure sending a big payload to the sw and see the available bandwith.
 3. Measure sending a small payload to the client.
 4. Measure sending a big payload to the client and see the available bandwith.

Repeat without taking into account waking up overhead.

## Related to client request and focused on SWW

 1. Model the time response based on the number of middleware plugged
in the worker.
 2. Characterize memory / time consumption of the different algorithms for SWW.

## Related to service workers

 1. Time to perform .clone()
 2. Time to store a response (can be measured from Content)
 3. Time to get a response (can be measured from Content)
 4. Measure the average life time of a sw

Comment 3

[Salvador de la Puente González [:salva]]

Comment #2 presents the overview of the performance test plan for Service Workers and Service Worker Ware, this overview will be translated to a metabug and specific bugs in the Sprint Planning next Monday, July, 10th.

Apart from the bugs derived from the planning. It is necessary to define a testing methodology to allow multiple developers to take care of the different bugs at the same time it's ensured we are measuring all the same way. This specific task will be open as another different bug next Monday too.

We have opted to not use Raptor because its maturity state, in an attempt to avoid introducing noise both in measurements and methodology.