[tracking] Create basic quality metrics, instrumentation and graphs for WebExtensions features

NEW
Unassigned

Status

()

Toolkit
WebExtensions: General
P2
normal
10 months ago
4 months ago

People

(Reporter: bsilverberg, Unassigned)

Tracking

(Depends on: 4 bugs, {meta})

51 Branch
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [metrics] triaged)

Attachments

(1 attachment)

(Reporter)

Description

10 months ago
As discussed in a meeting about WebExtensions quality planning, we plan to instrument our code to gather metrics which can then be used to measure quality/performance. 

The initial set of high priority measurements we plan to implement includes:

- Time spent loading content script
- Memory consumed by content script
- Memory consumed by background pages
- Jank caused by the content script
- Popup and sidebar metrics
- Crashes from the extension process
- Startup time for WebExtensions
- Startup background page time

This tracking bug also covers creating a dashboard for all of this information.
(Reporter)

Updated

10 months ago
Depends on: 1297167
(Reporter)

Updated

10 months ago
Depends on: 1320504
(Reporter)

Updated

10 months ago
Depends on: 1353162
(Reporter)

Updated

10 months ago
Depends on: 1353165
(Reporter)

Updated

10 months ago
Depends on: 1353168
(Reporter)

Updated

10 months ago
Depends on: 1353171
(Reporter)

Updated

10 months ago
Depends on: 1353172
(Reporter)

Updated

10 months ago
Depends on: 1353173
(Reporter)

Updated

9 months ago
Depends on: 1356323
(Reporter)

Comment 1

9 months ago
Created attachment 8863412 [details]
Analysis Plan

Requesting a data review for the WebExtensions Analysis Plan from rweiss.
Attachment #8863412 - Flags: review?(rweiss)
(Reporter)

Comment 2

9 months ago
Comment on attachment 8863412 [details]
Analysis Plan

Requesting an implementation review for the WebExtensions Analysis Plan from bsmedberg.
Attachment #8863412 - Flags: review?(benjamin)

Comment 3

9 months ago
I apologize for the delay. I added a few early questions to the gdoc and then realized that I probably need a more detailed meeting to understand some things about what kind of regressions you're planning on identifying and how that will show up in long-term monitoring. Could you set up a 30-minute meeting with me/dzeber/rweiss (and anyone else necessary from the addons team) to discuss?
Flags: needinfo?(bob.silverberg)
(Reporter)

Comment 4

9 months ago
Thanks Ben, As you've already seen, I scheduled a meeting for tomorrow afternoon.
Flags: needinfo?(bob.silverberg)

Comment 5

9 months ago
Comment on attachment 8863412 [details]
Analysis Plan

Clearing this out. General summary:

This plan is going to start with an understanding/survey of the current state of things, and not specific regression measurements. To that end, we'll document the decisions that we're trying to make in each bug and that should be fine for exploratory 6-month data collection.

We'll regroup for a round 2+ to cover the following additional areas:
* regression detection of the Firefox code for webextensions
* performance and regression detection for webextensions
Attachment #8863412 - Flags: review?(rweiss)
Attachment #8863412 - Flags: review?(benjamin)
(Reporter)

Updated

8 months ago
Depends on: 1371398

Comment 6

4 months ago
Given that the majority of metrics we wanted for 57 have landed, dropping in priority.
Keywords: meta

Updated

4 months ago
Priority: P1 → P2
(Reporter)

Comment 7

4 months ago
This is a tracking bug so doesn't really need to be assigned to anyone.
Assignee: bob.silverberg → nobody
You need to log in before you can comment on or make changes to this bug.