935190 - Try to show cached pages when Fennec is offline

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Brian Nicholson (:bnicholson)]

There have been several situations where I lose network connectivity and am stuck staring at a Gecko error page, even though the browser has a cached version of the page I want.

Real life example: I look up the CalTrain timetable to see which train I want to catch. After riding the train a few minutes, I want to check the time I'll be arriving at my stop. But the train is going through a dead zone, and I can't use the Internets. Sadness ensues.

Second example: I'm in another country with poor (or expensive) signal. I use the hotel's wifi to look up things to do and come up with travel plans for the day. I leave the hotel, and forgot to leave all those tabs open. Oops.

To get a cached version of a page, perhaps the error page can notify the user if there's one available. There may be cases where a manual override might useful too, though; in the second example, if my phone has a poor 1X connection, I might want to force offline mode rather than wait for pages to load over the slow connection.

Comment 1

[Brian Nicholson (:bnicholson)]

Attachment: Add about:neterror button to enable offline mode (early WIP)

Very early WIP for "offline mode". To use it, go to about:neterror and click Go offline. Doing so will make all requests use the LOAD_FROM_CACHE flag for that tab.

Some issues I've noticed using this flag:
* Clicking links in Google doesn't fire an onLocationChange for clicked results. That is, if I search "mozilla" and click the first result, I get an onLocationChange only for [1], not [2]. Consequently, even though I end up on [2], [1] is what appears in the URL bar as the location, and a history entry is stored for [1] even though it should redirect to [2].
* Going to facebook.com while in offline mode results in an infinite redirect loop (this happens only if facebook.com is not already in the cache).

[1] http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CDoQtwIwAA&url=http%3A%2F%2Fwww.mozilla.org%2Fen-US%2F&ei=UDGBUu3PFqr8igK524CYDw&usg=AFQjCNHtBTDG4UfknyFIKF6lZkaFvokqGA&sig2=EioCg9XncD3nWQVbBHeAIw&bvm=bv.56146854,d.cGE
[2] http://www.mozilla.org/en-US/

Aside from these technical issues, we'll also need to figure out a reasonable UX for offline. Do we want to make it so that every page the user tries to go to while offline shows some message/button for viewing the cached page? This is simple from a UX perspective, but could be annoying to have to repeatedly click this button for every page I try to visit when my phone doesn't have network connectivity. Additionally, I may want to go offline even when I do have network connectivity (e.g., the connection is slow and/or expensive), and this UX doesn't offer any way to do that.

Another option might be to have a toggle to explicitly enable/disable offline mode, but that option results in UX problems of its own. Where will this toggle go, and when is it shown? Is it global or per-tab? Will there be some kind of status indicator that the user is currently offline? When the user regains network connectivity, will we react in any way (notify the user, automatically disable offline mode, etc.)?

Comment 3

[Wesley Johnston (:wesj)]

Attachment: Prototype screenshot

So I'd like to do this without requiring you to hit a button. i.e. If we've got something in the cache, I think we should probably show it? But I also think we may need to provide some indication to the user that the site they're seeing isn't be "live".

I wonder if we can reuse the site identity icon like this? i.e. we can show a different icon up there, and provide some general info in the popup (maybe with an "Get online/Enable wifi" link/button if its turned off?). Anthony, any ideas/icons scattered around for this type of thing?

Comment 4

[Wesley Johnston (:wesj)]

Attachment: WIP

Here's the rough patch. I don't think putting this in loadUri is probably the best, but it works for general testing. Using the docshell like this forces subframes/subsequent navigation to use the same flags and avoids some redirect hell.

Comment 5

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

I actually thought we were doing this (In reply to Wesley Johnston (:wesj) from comment #3)
> Created attachment 8492470 [details]
> Prototype screenshot
> 
> So I'd like to do this without requiring you to hit a button. i.e. If we've
> got something in the cache, I think we should probably show it? But I also
> think we may need to provide some indication to the user that the site
> they're seeing isn't be "live".

I agree. I think the best UX here is one that just works and gets out of your way. If I have no internet and you've got the page cached, we should just show it.

> I wonder if we can reuse the site identity icon like this? i.e. we can show
> a different icon up there, and provide some general info in the popup (maybe
> with an "Get online/Enable wifi" link/button if its turned off?). Anthony,
> any ideas/icons scattered around for this type of thing?

I'll think about this here a bit more. I'd prefer not to add even more functionality to that space. But we did have the "hey this page can be viewed in Reader mode" indicator. (See bug 1011712).

Comment 6

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

^ Just to be clear, I mean how that "notification" behaves when the user taps on the URL bar as it's displayed, when the user launches Firefox and they have no internet, etc... is what I think we need to experiment with a little bit here.

But, could we get a build to test that just uses the visuals of bug 1011712? For the time being we could get rid of the icon and just adjust the copy)

Comment 7

[Wesley Johnston (:wesj)]

Me and brian talked last week about (in a separate bug from this) a page action for slow loading pages that would (quickly) just use the cached one (or maybe use the cached one until the load was done and then flip?). Something like that might go nicely with brad's desire to show the "HelperApps" icon during page load as well (bug 137339)

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

For some outside context, here is what Chrome does: http://addyosmani.com/blog/offline-mode-chrome/

Note from the "Gotchas":
* In Chrome stable (35 at the time of writing), there’s no indication that a site has a cached version available. Instead, you’ll be shown the stale/cached copy without warning if you try navigating to a page while offline.

Also note some of the related features:
* “Show saved copy”
* “Offline Auto-Reload Mode”

I think we need some indicator as well. Because so much of the content might not work, I think we need the user to be well aware that they are viewing a "saved copy" of the site.

Comment 9

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Mark Finkle (:mfinkle) from comment #8)
> I think we need some indicator as well. Because so much of the content might
> not work, I think we need the user to be well aware that they are viewing a
> "saved copy" of the site.

Fair points raised about links within the page not working. But we must also remember that on most phones, there is a constant list of notifications about their network status along the top. I'm not sure how much notification a user would require at this point.

I think if the user tries to load a page that's already cached, it would be better if it "just worked". The extra step of calling out something that the user is (probably) already aware of (that they don't have connectivity) and not showing them what they wanted to see (the page that they've actively tried reloading) seems counter intuitive. Just as, if the user suddenly were to reach a WiFi spot again, the next link they clicked would just load seamlessly. 

(In reply to Wesley Johnston (:wesj) from comment #7)
> Me and brian talked last week about (in a separate bug from this) a page
> action for slow loading pages that would (quickly) just use the cached one
> (or maybe use the cached one until the load was done and then flip?).

I could see this potentially working. Could we make the page black and white (cached) and just have it refresh once the orange bar went all the way across to the right?

Comment 10

[Mark Finkle (:mfinkle) (use needinfo?)]

My advice:
1. Keep it simple to start
2. Watch for feedback 
3. Keep it simple when adding follow-on features
4. Repeat

Let's try to land something that won't get backed out for breaking the world.

Comment 11

[Wesley Johnston (:wesj)]

(In reply to Anthony Lam (:antlam) from comment #9)
> I could see this potentially working. Could we make the page black and white
> (cached) and just have it refresh once the orange bar went all the way
> across to the right?

We could try. I'll try to get this in first and do that in a separate bug. Loading from the cache is actually kinda slow, and the networking layer isn't really smart enough to do both in parallel, so it might not be worth it (i.e. it will slow down load of the actual page a lot). It would be neat if we had a chance to do some fast hw-accelerated compositing with this in Java, but we don't (yet), so we'd have to use CSS filters. I assume their performance will be awful.

Personally, I don't think I actually notice the network indicators on my phone much unless I wonder why something didn't load. If it did, I would assume I had network. That's basically my worry here. But once I have time to get this running better you can try yourself.

Comment 12

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Wesley Johnston (:wesj) from comment #11)
> (In reply to Anthony Lam (:antlam) from comment #9)
> > I could see this potentially working. Could we make the page black and white
> > (cached) and just have it refresh once the orange bar went all the way
> > across to the right?
> 
> We could try. I'll try to get this in first and do that in a separate bug.
> Loading from the cache is actually kinda slow, and the networking layer
> isn't really smart enough to do both in parallel, so it might not be worth
> it (i.e. it will slow down load of the actual page a lot). It would be neat
> if we had a chance to do some fast hw-accelerated compositing with this in
> Java, but we don't (yet), so we'd have to use CSS filters. I assume their
> performance will be awful.

I could be nudged either way on this "saturated" effect, but getting a build to test would be great!
 
> Personally, I don't think I actually notice the network indicators on my
> phone much unless I wonder why something didn't load. If it did, I would
> assume I had network. That's basically my worry here. But once I have time
> to get this running better you can try yourself.

Sounds good. I see where you're coming from for sure. Just want to avoid making an overly big deal out of something that might not need to be called out a second time :)

Comment 13

[Brad Lassey [:blassey] (use needinfo?)]

filter on [mass-p5]

Comment 14

[:Margaret Leibovic]

I think we should fix this as part of an "offline improvements" effort.

Comment 15

[Richard Newman [:rnewman]]

I think we should fix this as part of an "offline improvements" effort.

:P

Comment 16

[:Margaret Leibovic]

(In reply to Richard Newman [:rnewman] from comment #15)
> I think we should fix this as part of an "offline improvements" effort.
> 
> :P

Re-noming tracking-fennec:+ bugs for new features isn't the best way for them to get prioritized. I filed an Aha idea to get this somewhere in our roadmap, although I haven't seen it appear in the unprioritized list yet.

Comment 17

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: usecache-when-offline v0.1

Updated WIP based on a few of Wes and Brian ideas:
1. Use docShell.defaultLoadFlags to control reading from cache
2. Use nsINetworkService notifications to track network
3. Update browser docShells when network changes
4. Set network state on startup, since we won't get a network state notification
5. Set newly created browser docShells based on the current network state
6. Add a page notification to show the offline state

Comment 18

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: usecache-when-offline v0.2

Adds a prompt explaining that the network is offline and the page was loaded from cache.

TODO:
1. Figure out the UI: If we want to use a page action we need a different image
2. NIGHTLY_BUILD flag
3. Telemetry on how often the page loads from cache vs still shows the net error page.

Comment 19

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: fennec-pageload-cache.png

Shows a page loaded from cache. Note there is not wifi or cell indicators in the statusbar. The "history" page action is shown as an indicator.

Yes, the reader button is displayed and functional.

Comment 20

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: fennec-pageload-reader-cache.png

Tapping the reader action will load the cached page in Reader View

Comment 21

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: usecache-when-offline v0.3

Almost ready for review

1. Added a network-offline image for pageaction
2. Strings in properties file
3. Controlled via "browser.tabs.useCache" which is only true for NIGHTLY_BUILD
4. Pageaction shows a dialog explaining the situation

I'm wondering if the pageaction dialog should allow people to turn this feature off.

Comment 22

[Barbara Bermes [:barbara] - NI please!]

Aha card for prioritization created: https://mozilla.aha.io/features/FENN-386

Comment 23

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Attachment: prev_offline_error.png

Can we put this user interaction on the error page? A big button like this!

This UX would catch more of the use cases and it would also make it obvious to the user that they're entering a different "mode". I think requiring this deliberate action can better reinforce the unstable nature of these offline experiences too.

Comment 24

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

^blocked the wrong bug..

Comment 25

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: usecache-when-offline v0.4

Updated patch to use a snackbar, but only if we are offline, using the cache and not showing the neterror page. If we are showing the neterror page, we don't have the page in cache.

Comment 26

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: fennec-offline-snackbar.png

Showing a snackbar when displaying a page loaded from cache. String is "Loading page from offline storage".

The snackbar is shown with a "long" timeout. I wonder if we should use an "indefinite" timeout?

Comment 27

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8700907 [details] [diff] [review]
usecache-when-offline v0.4

Review of attachment 8700907 [details] [diff] [review]:
-----------------------------------------------------------------

Code is looking good. I have just two problems when running the patch locally (Airplane mode):

* Whenever I open a new tab the "Loading page from offline storage" snackbar is shown for about:home
* For some pages I see the snackbar twice (spiegel.de, heise.de): The snackbar appears, then disappears and appears again. I assume this is happening because of some redirect because the URL is changing too?

(In reply to Mark Finkle (:mfinkle) from comment #26)
> The snackbar is shown with a "long" timeout. I wonder if we should use an
> "indefinite" timeout?

I just landed the patch to dismiss a snackbar when the user taps outside (bug 1233467). With that we could use LENGTH_INDEFINITE here now (This snackbar should go away when switching to a different tab). If just "long" is not long enough: Besides using the constants we also support arbitrary duration values in ms.

::: mobile/android/chrome/content/browser.js
@@ +7410,5 @@
>      } else {
>        Services.obs.addObserver(this, "memory-pressure", false);
>      }
>  
> +    // Watch for opportunities to pre-connect to high probablity targets.

Typo: probablity

Comment 28

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Sebastian Kaspari (:sebastian) from comment #27)

> 
> * Whenever I open a new tab the "Loading page from offline storage" snackbar
> is shown for about:home

Now ignoring for any "about:" URL instead of only "about:neterror"

> * For some pages I see the snackbar twice (spiegel.de, heise.de): The
> snackbar appears, then disappears and appears again. I assume this is
> happening because of some redirect because the URL is changing too?

This was because of iframes causing multiple "pageshow" events. I am now looking for the top level document "pageshow" event.

> > The snackbar is shown with a "long" timeout. I wonder if we should use an
> > "indefinite" timeout?
> 
> I just landed the patch to dismiss a snackbar when the user taps outside
> (bug 1233467). With that we could use LENGTH_INDEFINITE here now (This
> snackbar should go away when switching to a different tab). If just "long"
> is not long enough: Besides using the constants we also support arbitrary
> duration values in ms.

Using LENGTH_INDEFINITE now. Seems to work fine.

> > +    // Watch for opportunities to pre-connect to high probablity targets.
> 
> Typo: probablity

Fixed

Comment 29

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: usecache-when-offline v0.5

Updated with fixes

Comment 30

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/2aad6036a8e5

Comment 31

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/2aad6036a8e5

Comment 32

[Barbara Bermes [:barbara] - NI please!]

Is there a way I can get an apk to play around with on my device?

Comment 33

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Barbara Bermes [:barbara] from comment #32)
> Is there a way I can get an apk to play around with on my device?

This landed on Nightly, so just update to the latest version and you'll have it.

Comment 34

[Teodora Vermesan (:TeoVermesan)]

Attachment: Screenshot_2016-01-07-12-49-47.png

Load mozilla.org, cut off wifi, close and reopen Firefox and from history choose mozilla.org, the page load correctly and a snackbar is displayed at the bottom of the page: "Loading page from offline storage".
Verified as fixed using:
Device: Samsung Galaxy S4 (Android 5.0) 
Build: Firefox for Android 46.0a1 (2016-01-06)

Comment 35

[Liz Henry (:lizzard) (relman/hg->git project)]

Noted as "Previously cached pages will be displayed when offline"

Comment 36

[:Margaret Leibovic]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #35)
> Noted as "Previously cached pages will be displayed when offline"

Note: We are planning to disable this in release, so we should make sure this doesn't end up in the 46 GA release notes.

Comment 37

[Ioana Chiorean]

Teo's QA work can be tracked here - https://wiki.mozilla.org/QA/Fennec/Add_option_to_load_cached_pages_when_offline