1368629 - Add histogram for Leanplum usage

Status: RESOLVED INVALID

(Firefox for Android Graveyard :: Metrics, enhancement, P2)

Description

[Nevin Chen(Not active on Bugzilla)]

We need to add a histogram for "has leanplum enabled". Would be nice to get a rough estimate of how many users actually have it on.

Comment 1

[Nevin Chen(Not active on Bugzilla)]

Attachment: Bug 1368629 - Add histogram for Leanplum usage.

Review commit: https://reviewboard.mozilla.org/r/144240/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/144240/

Comment 2

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

https://reviewboard.mozilla.org/r/144240/#review148258

::: mobile/android/base/java/org/mozilla/gecko/mma/MmaLeanplumImp.java:89
(Diff revision 1)
>      }
>  
>      @Override
>      public void stop() {
>          Leanplum.stop();
> +        Telemetry.addToHistogram(HISTOGRAM_FENNEC_LEANPLUM_ENABLE, 0);

We are only calling STOP if the setting is changed right? So we wouldn't count users that just haven't leanplum enabled..? Could we instead add this histogram where we add all the other ones to count ALL users in the histogram to know how many of *all* users have leanplum?

Comment 3

[Nevin Chen(Not active on Bugzilla)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/144240/diff/1-2/

Comment 4

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

https://reviewboard.mozilla.org/r/144240/#review152890

::: mobile/android/base/java/org/mozilla/gecko/mma/MmaDelegate.java:67
(Diff revision 2)
>          PrefsHelper.PrefHandler handler = new PrefsHelper.PrefHandlerBase() {
>              @Override
>              public void prefValue(String pref, boolean value) {
>                  if (pref.equals(KEY_PREF_BOOLEAN_MMA_ENABLED)) {
>                      Log.d(TAG, "prefValue() called with: pref = [" + pref + "], value = [" + value + "]");
> +                    Telemetry.addToHistogram(HISTOGRAM_FENNEC_MMA_ENABLED, value ? 1 : 0);

This will not catch user for which the preference is disabled right?

I would like to see a count over *all* Fennec users ("We have Leanplum enabled for 10% of our users [with telemetry]"). Otherwise the percentages are misleading.

Comment 5

[Nevin Chen(Not active on Bugzilla)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

https://reviewboard.mozilla.org/r/144240/#review161048

::: mobile/android/base/java/org/mozilla/gecko/mma/MmaDelegate.java:67
(Diff revision 2)
>          PrefsHelper.PrefHandler handler = new PrefsHelper.PrefHandlerBase() {
>              @Override
>              public void prefValue(String pref, boolean value) {
>                  if (pref.equals(KEY_PREF_BOOLEAN_MMA_ENABLED)) {
>                      Log.d(TAG, "prefValue() called with: pref = [" + pref + "], value = [" + value + "]");
> +                    Telemetry.addToHistogram(HISTOGRAM_FENNEC_MMA_ENABLED, value ? 1 : 0);

This pref is not used by the user. It's only used for releng team to control Leanplum when building. Users will need to ue "Health Report" to disable Leanplum.

Comment 6

[Nevin Chen(Not active on Bugzilla)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/144240/diff/2-3/

Comment 7

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

https://reviewboard.mozilla.org/r/144240/#review163502

::: mobile/android/base/java/org/mozilla/gecko/mma/MmaDelegate.java:66
(Diff revision 3)
> +                final boolean healthReport = GeckoPreferences.getBooleanPref(activity, GeckoPreferences.PREFS_HEALTHREPORT_UPLOAD_ENABLED, true);
> +                Telemetry.addToHistogram(HISTOGRAM_FENNEC_MMA_ENABLED, value && healthReport ? 1 : 0);
> +

If leanplum is not enabled in switchboard then we won't get here, I guess? So this wouldn't cover *all* users?

In BrowserApp we handle the Telemetry:Gather event and add a bunch of histograms for all users. Would it be possible to know here whether leanplum is (or is going to be) enabled for this user and add the histogram here?

Comment 8

[Nevin Chen(Not active on Bugzilla)]

Comment on attachment 8872704 [details]
Bug 1368629 - Add histogram for Leanplum usage.

https://reviewboard.mozilla.org/r/144240/#review163502

> If leanplum is not enabled in switchboard then we won't get here, I guess? So this wouldn't cover *all* users?
> 
> In BrowserApp we handle the Telemetry:Gather event and add a bunch of histograms for all users. Would it be possible to know here whether leanplum is (or is going to be) enabled for this user and add the histogram here?

No, we won't. But I think for this bug, it's used to track those users "fall into Switchboard bucket" but "won't have Leanplum enabled becuase they turn off Health Report". I want this number cause I can double check it in Histogram with the number from Leanplum( which we will be charged accordingly)

I'm not sure If we check if Leanplum is enabled or not in Telemetry:Gather would get an accurate number. Cause at that time, switchboard may not completes its initialization. And since we are not going to enable Leanplum for every users, should we still need to colect *all* users?

Thanks for your patient and sorry for keep asking questions :)

Comment 9

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

We need someone to analyze this data to add probes so we can reopen if this is needed again.