1281284 - metrics/telemetry/analysis work for e10s-addon visibility

Status: RESOLVED WORKSFORME

(Firefox :: Extension Compatibility, defect)

Description

[:shell escalante]

Related to the release criteria that is specific for e10s-addons... but this is for metrics that we should start tracking for better visibility to make decisions before, during, and after experiments on the state of add-ons.

https://docs.google.com/document/d/1i8eEkt1s_ctK8q-ABUtKsJEWBL0wznVKuAzL7GaKEZs/edit?ts=5750c28b

Comment 1

[:shell escalante]

First target....

The data i want would identify top add-ons that should be e10s compatible.  We'd then test the most used ones and ask the authors to mark the add-on MPC=true.  Having a good set of MPC=true add-ons will be vital for the next beta and greatly improve our potential target audience.

The information would be based on CPOW (cross process object wrapper) usage.  CPOWs are also called shims often.  If an add-on doesn't use CPOWs, than it's a very good indicator that it should be compatible with e10s. 

http://arewee10syet.com/ site has links on the bottom to the python sheet that is looking at the telemetry probes
        there is data for Shims - yes or no
        there are CPOW counts
        if there are 0 CPOWs - shims should be "no" - so one source or the other is wrong. 
I need to be able to trust the CPOW count - as that data is actionable.

Comment 2

[:shell escalante]

arewee10syet has been adapt to show any add-ons with more than 2000 users (minimum required for Featured Add-on).  

talked with billm, zeber, andym about telemetry feeding arewee10syet.  going to make some adaptations - to select info only from latest release (to make sure the shim/cpow info is current).  Got all details on what the data reflects

Difference between shims and CPOWs
Shims intercept an add-on request that might not be e10s compatible and try to make it e10s compatible. 
Usually shims hand out CPOWs, although the add-on may not end up using the CPOW. CPOWs are cases where the e10s parent process makes a synchronous request of the e10s child process. Right now, if an add-on is marked as multiprocessCompatible, it does not get shims. 

Every time an add-on uses a shim, we record the add-on ID and the type of shim in telemetry. An add-on shows up as shim=yes if we ever see telemetry where the add-on used a shim.
TEST P1: 0 CPOW & no shim = almost guaranteed that these add-ons will work with MPC=true.
TEST P2: 0 CPOW & yes shim = worth testing.  It's possible that we could set MPC for these add-ons and they would still work, but there's no guarantee. Not all shims use CPOWs, so it might be that the add-on is using a shim that doesn't use CPOWs.

Comment 3

[:shell escalante]

closing out the e10s-add-ons specific telemetry tracker.  all metrics specific to that have been completed and moving on to just monitoring the e10s-multi + webextension info over next several releases.  no new metrics in this area (there is work in other areas).