Add telemetry for home panel customizations

RESOLVED FIXED in Firefox 47

Status

()

Firefox for Android
Awesomescreen
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: liuche, Assigned: liuche)

Tracking

Trunk
Firefox 47
ARM
Android
Points:
---

Firefox Tracking Flags

(firefox47 fixed)

Details

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Assignee)

Description

2 years ago
We should add a Simple Measures probe to see how many users have customized what home panels are displayed.
(Assignee)

Comment 1

2 years ago
Created attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

Review commit: https://reviewboard.mozilla.org/r/33351/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33351/
(Assignee)

Comment 2

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

I looked at how Desktop adds simple measures - I'm not entirely sure this will work because it's also calling into Java because the config is stored in SharedPreferences, and it has to wait for a callback.

Is there some way to trigger a telemetry collection?
Attachment #8715118 - Flags: feedback?(margaret.leibovic)
(In reply to Chenxia Liu [:liuche] from comment #2)
> Comment on attachment 8715118 [details]
> MozReview Request: Bug 1245368 - Add telemetry for home panel customizations.
> 
> I looked at how Desktop adds simple measures - I'm not entirely sure this
> will work because it's also calling into Java because the config is stored
> in SharedPreferences, and it has to wait for a callback.

Do you have a way to visualize the Simple Measures? Is there a dashboard somewhere for that?

> Is there some way to trigger a telemetry collection?

Putting Fennec into the background triggers a telemetry collection
We also have some UI Telemetry for hiding, showing, moving, and setting default. Look in Events for "panel.*.1" events. Looks like most activity on hiding panels or setting different defaults.

Comment 5

2 years ago
Why are we using UI telemtry? Can we just use a boolean histogram to know whether or not users have a stored panel config?

If I remember our conversation correctly, we talked about just wanting to understand what % of users would require a home config migration. To do that, I would just create a new histogram probe and do a check here to see if that customized home config shared preference exists: 
http://hg.mozilla.org/mozilla-central/file/tip/mobile/android/base/java/org/mozilla/gecko/BrowserApp.java#l1727

Updated

2 years ago
Assignee: nobody → liuche

Comment 6

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

Like mfinkle said, we don't have a way to visualize this, so I don't think we should do this. It also seems like too much overhead to add a new JSON message for the sake of gathering this data.
Attachment #8715118 - Flags: feedback?(margaret.leibovic) → feedback-
(Assignee)

Comment 7

2 years ago
(In reply to :Margaret Leibovic from comment #5)
> Why are we using UI telemtry? Can we just use a boolean histogram to know
> whether or not users have a stored panel config?
> 
Hm, I remember us talking about *not* using histograms since histograms are are suited to collecting a range of values (like startup times) rather than a single value (is pref X enabled). And with a histogram, we won't have the "end state," just how many times they've sent telemetry pings while in each state (which could still be helpful), whereas simple measures reflects a snapshot of the most recent state.

And I'm not sure about this but if the histograms are aggregated on the histograms dashboard, I don't think we'd be able to tell the _number_ of users who have each enabled, but would get the total number of responses we've collected, with multiple responses from each user. I suppose we could use this as a proxy for "number of people who have customized home panels", or switch our question to "what proportion of the time do people have customized home panels".

Comment 8

2 years ago
(In reply to Chenxia Liu [:liuche] from comment #7)
> (In reply to :Margaret Leibovic from comment #5)
> > Why are we using UI telemtry? Can we just use a boolean histogram to know
> > whether or not users have a stored panel config?
> > 
> Hm, I remember us talking about *not* using histograms since histograms are
> are suited to collecting a range of values (like startup times) rather than
> a single value (is pref X enabled). And with a histogram, we won't have the
> "end state," just how many times they've sent telemetry pings while in each
> state (which could still be helpful), whereas simple measures reflects a
> snapshot of the most recent state.

We already use histograms for the "is pref X enabled" state, if you look at the code I point to above.

> And I'm not sure about this but if the histograms are aggregated on the
> histograms dashboard, I don't think we'd be able to tell the _number_ of
> users who have each enabled, but would get the total number of responses
> we've collected, with multiple responses from each user. I suppose we could
> use this as a proxy for "number of people who have customized home panels",
> or switch our question to "what proportion of the time do people have
> customized home panels".

I'm fine with this proxy level, since it's something that we already do for the other prefs we measure.

Bottom line is that we need a way to visualize the telemetry that we include, and I think it would be too much work to make a new simple measurements dashboard just for this use case.
(Assignee)

Updated

2 years ago
Attachment #8715118 - Attachment description: MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. → MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret
Attachment #8715118 - Flags: feedback- → review?(margaret.leibovic)
(Assignee)

Comment 9

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/33351/diff/1-2/

Comment 10

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

https://reviewboard.mozilla.org/r/33351/#review30445

::: mobile/android/base/java/org/mozilla/gecko/BrowserApp.java:1752
(Diff revision 2)
> +            Telemetry.addToHistogram("FENNEC_HOMEPANELS_CUSTOM", hasCustomHomepanels ? 1 : 0);

This seems fine to me, but you'll need to define a new histogram in Histograms.json.
Attachment #8715118 - Flags: review?(margaret.leibovic)
(Assignee)

Comment 11

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/33351/diff/2-3/
Attachment #8715118 - Flags: review?(margaret.leibovic)

Updated

2 years ago
Attachment #8715118 - Flags: review?(margaret.leibovic) → review+

Comment 12

2 years ago
Comment on attachment 8715118 [details]
MozReview Request: Bug 1245368 - Add telemetry for home panel customizations. r=margaret

https://reviewboard.mozilla.org/r/33351/#review30675

Comment 13

2 years ago
https://hg.mozilla.org/integration/fx-team/rev/e342ec93ddcb

Comment 14

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/e342ec93ddcb
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox47: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 47
You need to log in before you can comment on or make changes to this bug.