1243387 - Add telemetry probe to understand how often users load reading list items when offline

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Reader View, defect)

Description

[:Margaret Leibovic]

I think we should be able to use the Tabs.useCache property to see whether or not the app is offline when we're trying to load a reader view page.

We can add the probe where we get the article here:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/Reader.js#328

Comment 1

[:Margaret Leibovic]

Note: this would depend on browser.tabs.useCache being set to true, so we couldn't uplift this probe to 45 beta (or if we want to, we'll need to modify it to use some of that offline detection logic).

Comment 3

[:Margaret Leibovic]

I'm debating the best way to go about writing this probe. Options I see:

1) Create a boolean histogram probe in Reader.js that tells us whether or not the user is online when reader view is loaded. This would be simple to implement, but it won't be specific to reader view pages loaded from the reading list. The data would likely skew "online", since it would include loading reader view after tapping the reader view button.

2) Modify the UI telemetry event for loading an item from the reading list to include "offline" or "online" in the extras. This would be trickier to implement because we would need to get the network state from Java (could we just use an Android API, or should we send a message to use the gecko network status?). This also doesn't necessarily correspond to all reader view page loads, since reading list items aren't guaranteed to be reader view-able.

I think the second option does a better job answering the question we have, which is whether or not people are trying to take advantage of the offline functionality of reading list. However, to be more scientific about it, I feel like we should also track the online/offline state for loading bookmarks to give us a baseline for "no expectation of offline".

Comment 4

[Barbara Bermes [:barbara] - NI please!]

"This also doesn't necessarily correspond to all reader view page loads, since reading list items aren't guaranteed to be reader view-able"

So 2) currently can distinguish between a user being offline and online when loading RL items but it can't distinguish if the item they viewed in offline mode was in RV?

Comment 5

[:Margaret Leibovic]

(In reply to Barbara Bermes [:barbara] from comment #4)
> "This also doesn't necessarily correspond to all reader view page loads,
> since reading list items aren't guaranteed to be reader view-able"
> 
> So 2) currently can distinguish between a user being offline and online when
> loading RL items but it can't distinguish if the item they viewed in offline
> mode was in RV?

Correct. If they're offline, the page may show in reader view, or we'll show an error page.

We don't have a way to tell if an item in reading list turned into a reader view page or not (without adding more logic, and there's not really an easy way to do that right now).

Comment 6

[Mark Finkle (:mfinkle) (use needinfo?)]

Why not add a boolean histogram to the ReadingList Java code? You could add a probe right under the LOADURL code we already have:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/home/ReadingListPanel.java#94

The Android code to checking for network connection looks like:

boolean isOnline() {
    ConnectivityManager connectivity = (ConnectivityManager) this.getSystemService(Context.CONNECTIVITY_SERVICE);
    NetworkInfo networkInfo = connectivity.getActiveNetworkInfo();
    if (networkInfo == null) {
      return false;
    }
    return networkInfo.isConnected();
}

Comment 7

[:Margaret Leibovic]

Attachment: MozReview Request: Bug 1243387 - Add telemetry probe to understand how often users load reading list items when offline. r=mfinkle

Review commit: https://reviewboard.mozilla.org/r/33789/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33789/

Comment 8

[:Margaret Leibovic]

Attachment: MozReview Request: Bug 1243387 - (Part 2) Update SuggestClient to use NetworkUtils. r=mfinkle

Review commit: https://reviewboard.mozilla.org/r/33791/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33791/

Comment 9

[:Margaret Leibovic]

Comment on attachment 8716307 [details]
MozReview Request: Bug 1243387 - (Part 2) Update SuggestClient to use NetworkUtils. r=mfinkle

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/33791/diff/1-2/

Comment 10

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 8716306 [details]
MozReview Request: Bug 1243387 - Add telemetry probe to understand how often users load reading list items when offline. r=mfinkle

https://reviewboard.mozilla.org/r/33789/#review30513

Comment 11

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 8716307 [details]
MozReview Request: Bug 1243387 - (Part 2) Update SuggestClient to use NetworkUtils. r=mfinkle

https://reviewboard.mozilla.org/r/33791/#review30515

Comment 12

[:Margaret Leibovic]

https://hg.mozilla.org/integration/fx-team/rev/67713b7ddcd1caeb9cef842af78e5e5d6c51fe89
Bug 1243387 - Add telemetry probe to understand how often users load reading list items when offline. r=mfinkle

https://hg.mozilla.org/integration/fx-team/rev/d81d3ca528ad5c96440312f9c4731ee6b46ca623
Bug 1243387 - (Part 2) Update SuggestClient to use NetworkUtils. r=mfinkle

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/67713b7ddcd1
https://hg.mozilla.org/mozilla-central/rev/d81d3ca528ad

Comment 14

[Barbara Bermes [:barbara] - NI please!]

- Where can I find this in our dashboards?
- Once we migrated the RL into the Bookmarks panel, will we be able to determine if a RV item was loaded offline from the bookmarks panel?

Comment 15

[:Margaret Leibovic]

(In reply to Barbara Bermes [:barbara] from comment #14)
> - Where can I find this in our dashboards?

This is a histogram probe: http://mzl.la/21kc01q

It looks like very few people are trying to load anything when they're offline.

> - Once we migrated the RL into the Bookmarks panel, will we be able to
> determine if a RV item was loaded offline from the bookmarks panel?

As currently implemented, no. RV items would just be lumped in with bookmarks, since they will be bookmarks in this case. We'll need to update the probe if we want to distinguish between RV and non-RV bookmark items. We can file a bug to do that as part of the migration work if we need this data.

Comment 16

[:Margaret Leibovic]

Comment on attachment 8716306 [details]
MozReview Request: Bug 1243387 - Add telemetry probe to understand how often users load reading list items when offline. r=mfinkle

Approval Request Comment
[Feature/regressing bug #]: Migrate reading list to bookmarks.

[User impact if declined]: We won't understand how people try to load bookmarks when they're offline.

[Describe test coverage new/current, TreeHerder]: No tests.

[Risks and why]: Low-risk, adds new telemetry probe.

[String/UUID change made/needed]: None.

Comment 17

[Barbara Bermes [:barbara] - NI please!]

(In reply to :Margaret Leibovic from comment #15)
> (In reply to Barbara Bermes [:barbara] from comment #14)
> > - Where can I find this in our dashboards?
> 
> This is a histogram probe: http://mzl.la/21kc01q
> 
> It looks like very few people are trying to load anything when they're
> offline.
> 
> > - Once we migrated the RL into the Bookmarks panel, will we be able to
> > determine if a RV item was loaded offline from the bookmarks panel?
> 
> As currently implemented, no. RV items would just be lumped in with
> bookmarks, since they will be bookmarks in this case. We'll need to update
> the probe if we want to distinguish between RV and non-RV bookmark items. We
> can file a bug to do that as part of the migration work if we need this data.

I've been mentioning this in other bugs before (https://bugzilla.mozilla.org/show_bug.cgi?id=1251058), knowing what type of content is being saved to, and loaded from the bookmarks panel is important to understand. 

I've created a bug for RV items saved/loaded: https://bugzilla.mozilla.org/show_bug.cgi?id=1251661

Comment 18

[Liz Henry (:lizzard) (relman/hg->git project)]

Marking affected for aurora

Comment 19

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8716306 [details]
MozReview Request: Bug 1243387 - Add telemetry probe to understand how often users load reading list items when offline. r=mfinkle

Sounds good to try to get this out to more users fast so we can tell how they work with new reading list features.

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/d3d8118a3267