1352992 - (webextensions-telemetry) [tracking] Create basic quality metrics, instrumentation and graphs for WebExtensions features

Status: NEW

(WebExtensions :: General, enhancement, P3)

Description

[Bob Silverberg [:bsilverberg]]

As discussed in a meeting about WebExtensions quality planning, we plan to instrument our code to gather metrics which can then be used to measure quality/performance. 

The initial set of high priority measurements we plan to implement includes:

- Time spent loading content script
- Memory consumed by content script
- Memory consumed by background pages
- Jank caused by the content script
- Popup and sidebar metrics
- Crashes from the extension process
- Startup time for WebExtensions
- Startup background page time

This tracking bug also covers creating a dashboard for all of this information.

Comment 1

[Bob Silverberg [:bsilverberg]]

Attachment: Analysis Plan

Requesting a data review for the WebExtensions Analysis Plan from rweiss.

Comment 2

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8863412 [details]
Analysis Plan

Requesting an implementation review for the WebExtensions Analysis Plan from bsmedberg.

Comment 3

[Benjamin Smedberg]

I apologize for the delay. I added a few early questions to the gdoc and then realized that I probably need a more detailed meeting to understand some things about what kind of regressions you're planning on identifying and how that will show up in long-term monitoring. Could you set up a 30-minute meeting with me/dzeber/rweiss (and anyone else necessary from the addons team) to discuss?

Comment 4

[Bob Silverberg [:bsilverberg]]

Thanks Ben, As you've already seen, I scheduled a meeting for tomorrow afternoon.

Comment 5

[Benjamin Smedberg]

Comment on attachment 8863412 [details]
Analysis Plan

Clearing this out. General summary:

This plan is going to start with an understanding/survey of the current state of things, and not specific regression measurements. To that end, we'll document the decisions that we're trying to make in each bug and that should be fine for exploratory 6-month data collection.

We'll regroup for a round 2+ to cover the following additional areas:
* regression detection of the Firefox code for webextensions
* performance and regression detection for webextensions

Comment 6

[Andy McKay]

Given that the majority of metrics we wanted for 57 have landed, dropping in priority.

Comment 7

[Bob Silverberg [:bsilverberg]]

This is a tracking bug so doesn't really need to be assigned to anyone.