1281286 - (e10s-addon-telemetry) how to measure e10s-addons release criteria measurements collected via telemetry

Status: RESOLVED FIXED

(Firefox :: Extension Compatibility, defect)

Description

[:shell escalante]

much of the insight gathered for e10s release criteria came from telemetry.  who/how do we need to engage to collect similar info to measure addons impact.

Comment 1

[:shell escalante]

For experiment in bug 1282120 - who can pull the telemetry data collection and do analysis between the experiment and the cohort?  

took info from chris's wiki and put in spreadsheet so easier to mark what is telemetry versus other info https://docs.google.com/spreadsheets/d/1j1GnKGaMmdHDHccanst8EQC2wSK07pvRnHAWISv5FoA/edit#gid=0

BDS said crash data didn't come from telemetry?  where does that come from as it's likely highly important.

also who decides & defines what an acceptable variance is for the impact addons have on the Firefox results (between e10s/non-e10s populations with same addons)?  Is it same as what was set for e10s over non-e10s?  I saw some variance defined (+/- 10%) in the wiki - but not all.

Comment 2

[:shell escalante]

needing info to Kev who is leading data analysis interactions with various teams

Comment 3

[:shell escalante]

David Zeber took the release criteria [minus stability (in different graph) and talos (not covering with add-ons)] and organized it into python notebook.  the link changes based on release - but here is the link for beta 3-4 https://github.com/dzeber/e10s_analyses/blob/beta-49-addons/beta/49/week3/e10s_experiment.ipynb