Closed
Bug 1388648
Opened 8 years ago
Closed 7 years ago
[meta] [Onboarding] [Telemetry] Add telemetry probe and build dashboard for the Onboarding tour
Categories
(Firefox :: General, enhancement, P1)
Firefox
General
Tracking
()
RESOLVED
FIXED
Tracking | Status | |
---|---|---|
firefox57 | --- | wontfix |
People
(Reporter: chsiang, Unassigned)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
(Keywords: meta, Whiteboard: [photon-onboarding][onboarding-telemetry] )
User Story
The Firefox Product team has a need to understand how new users and update users are interacting with the Firefox onboarding tour and how these interactions influence retention of the product. Schedule: 57 Requirement doc https://docs.google.com/document/d/1hc6RLoWglQ58Uq4n9ofjMLhwTSWQcs7q9K4AAmjk-ow/edit#heading=h.lkhxjjvoeyop
This is the meta bug to track onboarding telemetry and dashboard using ping center.
Comment 1•8 years ago
|
||
After discuss with Activity Stream team, we'll focus on collect onboarding notification and overlay ping events within onboarding addon at first stage. Then explore other possibility when ping-centre module is integrated into firefox.
Current discussion and break down are cooked in google doc
https://docs.google.com/a/mozilla.com/document/d/1hc6RLoWglQ58Uq4n9ofjMLhwTSWQcs7q9K4AAmjk-ow/edit?usp=sharing
Will file related bugs when we have more clear sight on this.
Updated•8 years ago
|
Whiteboard: [photon-onboarding][triage]
Updated•8 years ago
|
Priority: -- → P3
Whiteboard: [photon-onboarding][triage] → [photon-onboarding]
Updated•7 years ago
|
Whiteboard: [photon-onboarding] → [photon-onboarding][onboarding-telemetry]
Comment 2•7 years ago
|
||
Based on the prototype board
https://sql.telemetry.mozilla.org/dashboard/onboarding
Here is the current check result
* Onboarding Daily Sessions
Onboarding session by date
- Should only count event=onboarding_session
* Onboarding Daily Events
number of users interacting with onboarding
- Should calc none repeated client id
* Onboarding_Notification Engagement
- current have Total engagement, can add New, Update number
* Overlay Engagement by Tour
- description: Performance, Default Browser, Sync CTA count are auto generate
- current have Total engagement, can add New, Update number
* Onboarding Interaction
- Add overlay-close-button-click & overlay-close-outside-click events once available
Per tour interaction
- Calc drop rate by (total Tour1 impression/total Tour1 cta click)
Comment 3•7 years ago
|
||
for `drop rate` I mean `daily engage rate` if it make sense
Comment 4•7 years ago
|
||
As discussed on slack, redash does not allow co-edit, so I'll provide unpublished query so you can integrate them on onboarding dashboard and do changes anytime you want.
Updated: onboarding sessions (only count how many tabs created)
https://sql.telemetry.mozilla.org/queries/48510
Updated: onboarding daily users that interact with any onboarding elements (overlay/notification) (with right user number)
https://sql.telemetry.mozilla.org/queries/48511
New: daily users that opened the overlay (with right user number)
https://sql.telemetry.mozilla.org/queries/48518
Update: engagement daily users that click the overlay or notification cta button (with right user number)
https://sql.telemetry.mozilla.org/queries/48519
Flags: needinfo?(chsiang)
Updated•7 years ago
|
Summary: [meta] [Onboarding] [Telemetry] Add telemetry probe and build dashboard for 57 onboarding tour → [meta] [Onboarding] [Telemetry] Add telemetry probe and build dashboard for the Onboarding tour
Comment 6•7 years ago
|
||
Hey Fischer,
Unfortunately, if there are any other bugs that have not yet landed, we cannot uplift them to 57. The last beta went to build yesterday. I worked with Fred to get Bug 1412164 out the door and he said that was the highest priority for telemetry. All others will have to wait till 58.
Updated•7 years ago
|
status-firefox57:
--- → wontfix
Comment 7•7 years ago
|
||
As discussed in last Friday with Cindy & engineers, we won't trace new telemetry columns (except Bug 1412164) for v57.
So the priority is not related to v57 but just related to our current tasks priority.
Comment 8•7 years ago
|
||
new/update user can be queried by adding `tour_type = 'new' or 'tour_type = update'` clause
Comment 9•7 years ago
|
||
New/Update Users who open tour overlay by day
https://sql.telemetry.mozilla.org/queries/48942/source#132220
Comment 10•7 years ago
|
||
Onboarding overlay that filtered out the experiment events
https://sql.telemetry.mozilla.org/queries/49127#132287
Comment 11•7 years ago
|
||
how many home/newtabs created daily (replace Onboarding Daily Sessions)
https://sql.telemetry.mozilla.org/queries/49128#132291
Comment 12•7 years ago
|
||
Engagement with Firefox logo when it’s active (colored) vs. when it’s timed out (watermark)
(filter overlay-session + tour_source = ‘default’/tour_source = ‘watermark’)
https://sql.telemetry.mozilla.org/queries/49129#132294
Comment 13•7 years ago
|
||
Daily user > onboarding user https://sql.telemetry.mozilla.org/queries/49174#132400
Can use to replace New Tab User by day_57 Release & Onboarding Tour User by day_57 Release
Comment 14•7 years ago
|
||
Onboarding overlay engagement https://sql.telemetry.mozilla.org/queries/49175#table
Can use to replace `Overlay Engagement by Tour`
Comment 15•7 years ago
|
||
After discussion, Cindy decides we are ok with the current dashboard state.
We will build new dashboard and send fx-metrics review with new telemetry data in bug 1419995
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•