1401043 - Build Activity Stream telemetry dashboards

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Activity Stream, enhancement, P1)

Description

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

We should model it after the desktop dashboard [1]. Note that we have existing dashboards that get us part of the way there: [2] [3].

Here's the data we want to display:

(In reply to Michael Comella (:mcomella) from bug 1377288 comment #12)
> I spoke with Maria. She emphasized that *accuracy* of the data is more
> important than the quantity of the data (i.e. how many probes we're
> collecting). Her priorities on the data we collect are:
> - Clicks to load urls, divided by top sites, pocket, and highlights (already
> in the dashboard)
> - Context menu interactions, divided by top sites, pocket, and highlights
> - The telemetry on the Pocket More button, bug 1400408.
> 
> Like the other dashboards, it'd be good to include engagement and retention.
> 
> Notes:
> - "engagement" = DAU/MAU, which is essentially an estimate on the percent of
> the MAU active per day.
> - I have to verify the probes we already have in place are accurate

---

Adding to the current sprint so we can track this work, but it doesn't need to land before the 57 beta merge.

[1]: https://sql.telemetry.mozilla.org/dashboard/activity-stream-system-addon-metrics-summary

[2]: https://sql.telemetry.mozilla.org/dashboard/android-activity-stream-proof-of-concept- (note: re-add the hyphen at the end when clicking this link!)
[3]: https://sql.telemetry.mozilla.org/dashboard/android-activity-stream-interactions

Comment 1

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Moved NI from bug 1377288. It'd be useful to so that we can compare our new data with the old top sites data.

(In reply to Michael Comella (:mcomella) from bug 1377288 comment #10)
> Barbara, are you aware of any existing dashboards for engagement-like
> queries on top sites, pre-Activity Stream?
> 
> From what I've found:
> - Android: UI Telemetry Summary [1]: clicking the "Event summary" chart
> causes the browser to warn me a page is slowing down my computer, but I
> think this a similar query to...
> - Android: UI Top Features [2]: this is a pivot table that shows the total
> number of clicks on various UI elements over the past 7 days
> - Android: DAU (High Level by features) [3]. Shows unique daily users
> loading sites from bookmarks, top sites, etc.
> 
> To find these, I searched "android" and "fennec" and skimmed the list of the
> query search results.
> 
> 
> [1]: https://sql.telemetry.mozilla.org/queries/59#633
> [2]: https://sql.telemetry.mozilla.org/queries/2917#5504
> [3]: https://sql.telemetry.mozilla.org/queries/169#272

Comment 2

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

(moved NI from bug 1377288) Marina, in bug 1377288 comment 1, Grisha mentions:

(In reply to :Grisha Kruglov from bug 1377288 comment #1)
> Another thing to note is that the dashboards have diverged, making direct
> comparisons harder. I'm wondering if there's value in committing to keeping
> up our telemetry dashboards for Desktop, Android and iOS in relative
> lockstep, particularly for the intersection of what we measure similarly
> across platforms. Android and iOS are inherently going to be closer
> neighbours due to how mobile AS Telemetry evolved.

Do you think there's value in keeping our dashboards together?

Comment 3

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

[triage] This doesn't require code or uplift. It'd be great to do this during Beta or just before merge to see the resulting telemetry.

Comment 4

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

All open Activity Stream bugs are moving from the whiteboard tag, "[mobileAS]", to the Firefox for Android component, "Activity Stream", so that I can keep better track of these bugs as the new triage owner; I will send out an email shortly with additional details, caveats, etc.

Comment 5

[Marina Samuel [:emtwo]]

After a chat with Maria, it sounds like she does want to see consistency across dashboards for iOS/Android. I'll be spending some time polishing the existing Android dashboard in the next couple of weeks.

Comment 6

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

FYI that the Activity Stream telemetry docs are slightly out-dated (mostly missing info): I have bug 1402026 to fix this and there is a patch waiting for review.

Comment 7

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

The docs are here btw: http://searchfox.org/mozilla-central/source/mobile/android/docs/activitystreamtelemetry.rst

Comment 8

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

I added some features Maria requested and this should be good enough for release.

There will be additional polish work in https://github.com/mozilla/ping-centre/issues/105.