1433916 - extend service worker telemetry probes

Status: RESOLVED FIXED

(Core :: DOM: Service Workers, enhancement, P2)

Description

[Ben Kelly [:bkelly, not reviewing]]

We got a notification that the following probes are expiring in FF60:

* SCALARS_SW.ALTERNATIVE_BODY_USED_COUNT expires in version 61.0a1 (watched by [e-mail], [e-mail]) - The count of number of synthesize response using alternative body.
* SCALARS_SW.CORS_RES_FOR_SO_REQ_COUNT expires in version 61.0a1 (watched by [e-mail], [e-mail]) - The count of number of synthesize response made by service workers and it's a cors type resposne for a same-origin mode request.
* SCALARS_SW.SYNTHESIZED_RES_COUNT expires in version 61.0a1 (watched by [e-mail], [e-mail]) - The count of number of synthesize response made by service workers.
* SERVICE_WORKER_FETCH_EVENT_CHANNEL_RESET_MS expires in version 61.0a1 (watched by [e-mail]) - Time (in ms) measured between when the fetch handler finished executing and when we reset the network channel.
* SERVICE_WORKER_FETCH_EVENT_DISPATCH_MS expires in version 61.0a1 (watched by [e-mail]) - Time (in ms) measured between when the fetch event is dispatched by the Service Worker and before we execute the event listeners. This effectively measures the SW backlong, in the future this should also cover the IPC steps required to dispatch events to the SW.
* SERVICE_WORKER_FETCH_EVENT_FINISH_SYNTHESIZED_RESPONSE_MS expires in version 61.0a1 (watched by [e-mail]) - Time (in ms) measured between when the respondWith promise resolves and when we provide the response through the intercepted channel (FinishSynthesizedResponse).
* SERVICE_WORKER_FETCH_INTERCEPTION_DURATION_MS expires in version 61.0a1 (watched by [e-mail]) - Time delta (ms) between when a network request is intercepted and the service worker provides a response. This includes: possible SW startup, worker thread dispatch, time until the respondWith() promise resolves, time to synthesize the response (including the body), worker thread to main thread dispatch. In the future this should include the IPC roundtrips involved in fetch interception.

Comment 1

[Ben Kelly [:bkelly, not reviewing]]

We probably want to extend all of these except SCALARS_SW.CORS_RES_FOR_SO_REQ_COUNT.  The spec has been changed to disallow that situation and this probe was used to help make that decision.  We don't need it any more.

Marion, can you add this to our list of things we need to get done before the next merge?  Thanks.

Also, you may want to figure out how to get on the sw-telemetry@m.c mailing list so you see these notifications as well.

Comment 2

[Marion Daly [:mdaly]]

Will do, I'll see what I can do make sure this is sorted. Thank you for bringing to our attention.

Comment 3

[Andrew Overholt [:overholt]]

(In reply to Ben Kelly [:bkelly] from comment #1)
> Also, you may want to figure out how to get on the sw-telemetry@m.c mailing
> list so you see these notifications as well.

Done (Marion is an owner of sw-telemetry@ now).

Comment 4

[Marion Daly [:mdaly]]

Attachment: P1 Update Scalars and Histogram for SW Tele Probes from 61 to 65

Comment 5

[Chris H-C :chutten]

Comment on attachment 8961003 [details] [diff] [review]
P1 Update Scalars and Histogram for SW Tele Probes from 61 to 65

Review of attachment 8961003 [details] [diff] [review]:
-----------------------------------------------------------------

Need a bug number and a reviewer on the commit message, too.

For Data Collection Review you need to fill out this form over here: https://github.com/mozilla/data-review/blob/master/request.md

Either paste it into a comment or add it as an attachment and answer the questions inline. Then flag me for review or needinfo or whatever.

::: toolkit/components/telemetry/Scalars.yaml
@@ +1454,4 @@
>        - 1423623
>      description: >
>        The count of number of synthesize response using alternative body.
> +    expires: "66"

Why 66? The others are 65.

(You can make them all 66, if you want. I'm just confused by this one being different)

Comment 6

[Marion Daly [:mdaly]]

(In reply to Chris H-C :chutten from comment #5)
> Comment on attachment 8961003 [details] [diff] [review]
> P1 Update Scalars and Histogram for SW Tele Probes from 61 to 65
> 
> Review of attachment 8961003 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Need a bug number and a reviewer on the commit message, too.
> 
> For Data Collection Review you need to fill out this form over here:
> https://github.com/mozilla/data-review/blob/master/request.md
> 
> Either paste it into a comment or add it as an attachment and answer the
> questions inline. Then flag me for review or needinfo or whatever.
> 
> ::: toolkit/components/telemetry/Scalars.yaml
> @@ +1454,4 @@
> >        - 1423623
> >      description: >
> >        The count of number of synthesize response using alternative body.
> > +    expires: "66"
> 
> Why 66? The others are 65.
> 
> (You can make them all 66, if you want. I'm just confused by this one being
> different)

My mistake, that's a typo on my end. I'll correct it and resubmit the patch

Comment 7

[Yaron Tausky [:ytausky]]

Attachment: Bug 1433916: Extend usage of ServiceWorker telemetry probes

Several useful probes are set to expire in version 61. This commit extends them
until version 68.

Comment 8

[Yaron Tausky [:ytausky]]

# Request for data collection review form

1) What questions will you answer with this data?

We will use the timing probes to monitor the latency associated with using service workers. This would help us improve the latency of user-initiated actions. The counters are used as health indicators, allowing us to ensure that certain optimizations are actually applied in real use cases.

2) Why does Mozilla need to answer these questions?  Are there benefits for users? Do we need this information to address product or business requirements? 

The latency associated with using service workers translates to latency in responding to user actions. A high latency implementation might adversely affect the adoption of the ServiceWorker API.

3) What alternative methods did you consider to answer these questions? Why were they not sufficient?

None, since real world data would provide the most insight, and none of it can be used to identify users.

4) Can current instrumentation answer these questions?

No, those are specific measurements that are not mirrored in other probes.

5) List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox [data collection categories](https://wiki.mozilla.org/Firefox/Data_Collection) on the Mozilla wiki.   

All measurements are in category 1.

6) How long will this data be collected? 

We would like to extend the collection up to version 68.

7) What populations will you measure?

Nightly, beta & release; no filters.

8) If this data collection is default on, what is the opt-out mechanism for users?

about:preferences#privacy

9) Please provide a general description of how you will analyze this data.

We will monitor the data over time for any unusual changes as the code is being worked on.

10) Where do you intend to share the results of your analysis?

Within the team and on Bugzilla, as necessary.

Comment 9

[Chris H-C :chutten]

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Standard Telemetry mechanisms apply.

    Is there a control mechanism that allows the user to turn the data collection on and off? 

Standard Telemetry mechanisms apply.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? **

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice? 

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data? 

Yes. :ytausky is responsible for filing a follow-up if necessary.

---
Result: datareview+

Comment 10

[Chris H-C :chutten]

Comment on attachment 8997964 [details]
Bug 1433916: Extend usage of ServiceWorker telemetry probes

Chris H-C :chutten has approved the revision.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/26ead8a6fe5e
Extend usage of ServiceWorker telemetry probes r=chutten

Comment 12

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/26ead8a6fe5e