1499498 - Devtools nWAU engaged user metric

Status: RESOLVED FIXED

(Data Science :: Investigation, task, P3)

Description

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Work related to understanding the engagement of Devtools users.

Counting the number of users who activate devtools on at least n days of any one-week period yields "nWAU."

nWAU proposal document: https://docs.google.com/document/d/1JaUQznD1LkQzeVFphyC1Uffs8xE4zpmdz_PcsgHZEuM/edit

Comment 1

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Some explorations so far:

* Early exploration https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/31325/command/31326
  - includes segmentation by channel, country
  - covariance with user-day toolbox active time, active_ticks, URI counts
* Historical counts of nWAU https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/33783/command/33784
* Devtools retention after n usage-days in a calendar week https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/36073/command/36096

Some pending explorations:

* language/locale vs country for nWAU
* default browser status, number of Firefox profiles for nWAU

Comment 2

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

cc Harald -- the data science team is trying out a Bugzilla workflow; there's no action needed but thought you might find it helpful to be tagged in here.

Comment 3

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Default browser status: https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/41069/command/41076 -- there's a weak, gradually increasing trend between number of devtools user-days in a sample week and the probability of having Firefox set as the default browser. National differences are larger; DE > US > IN > CN.

I opened a pull request to make the NUMBER_OF_PROFILES probe available in main_summary, but I can pull raw pings to take a look at that next week.

Comment 4

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I updated the notebook linked above [1] to also plot locale vs user-days; locale was mostly independent of the number of user-days. Almost all CN users have a zh-CN locale; most German users use DE; most India users use en-US but some are en-GB.

It's possible that that German users with a greater number of user-days are marginally more likely to have a en-US locale (vs DE), and India users with a greater number of user days may be marginally more likely to have a en-GB locale (vs en-US). Both are small effects that would need additional validation.

[1] https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/41069/command/41076

Comment 5

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I'll close this bug since there isn't any ongoing work to track; if we develop some criteria for a retrospective evaluation we can open a new bug.