1046004 - Preloaded appcache is not populated correctly

Status: RESOLVED FIXED

(Core Graveyard :: DOM: Apps, defect)

Description

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

+++ This bug was initially created as a clone of Bug #1024980 +++

I tested SUMO and wikipedia app on v1.3 with offline content but it does not work.

STR:
1. Download Wikipedia app from firefoxos-gaia-spain[1] to external-apps/
2. |MOZILLA_OFFICIAL=1 make production|
3. click wikipedia app icon without internet connection

expected: show wikipedia app
actually: got a dialog

please get more information on bug 1024879

Comment 1

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

needinfo? same people on bug 1024890 comment 27, 28, 29

Comment 2

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

sorry it's bug 1024980 comment 27, 28, 29

Comment 3

[:gerard-majax]

(In reply to Yuren [:yurenju] from comment #0)
> +++ This bug was initially created as a clone of Bug #1024980 +++
> 
> I tested SUMO and wikipedia app on v1.3 with offline content but it does not
> work.
> 
> STR:
> 1. Download Wikipedia app from firefoxos-gaia-spain[1] to external-apps/
> 2. |MOZILLA_OFFICIAL=1 make production|
> 3. click wikipedia app icon without internet connection
> 
> expected: show wikipedia app
> actually: got a dialog
> 
> please get more information on bug 1024879

Do you know whether this still happens on 2.0 or on master ?

Comment 4

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

bug 1024980 is what I want to refer both for comment 0 and comment 1.

Comment 5

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I don't know but will verify it later.

Comment 6

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

[Blocking Requested - why for this release]:

(from :jsmith comment on bug 1024980 comment 26)
> [Blocking Requested - why for this release]:
> 
> I think we made a mistake here punting this bug out of 1.3. Turns out this
> affects multiple preinstalled b2g apps preloading appcache. This was working
> in past releases, so this is a regression.
> 
> We need to get this fixed in the next available release. Otherwise, we will
> risk not allowing partner apps with appcache from having offline access
> until an available network connection is established.

Comment 7

[:gerard-majax]

Okay it's broken even on master.

Comment 8

[:gerard-majax]

STR:
 0. Get preload.py from []
 1. |mkdir -p outoftree_apps && echo "Wikipedia,https://bits.wikimedia.org/WikipediaMobileFirefoxOS/manifest.webapp" >> outoftree_apps/list|
 2. |cd outoftree_apps && python preload.py && cd ..|
 3. |make production|
 4. Disable Wi-Fi and data connection

Expected:
 I can load offline Wikipedia.

Actual:
 I get a message to enable internet connection.

Comment 9

[:gerard-majax]

Fun fact: on first boot, wikipedia does not work offline. But after a second boot, I got an update notification. After downloading it, it does work offline. The app was preloaded a couple of minutes before.

Comment 10

[:gerard-majax]

So /data/local/OfflineCache/ exists, moz_cache_groups is okay but moz_cache table is empty. Comparing the database after applying the update, we have the proper content in the moz_cache table.

Comment 11

[:gerard-majax]

Nice,

> 01-02 04:50:31.219   312   312 E GeckoConsole: [JavaScript Error: "NS_ERROR_NOT_IMPLEMENTED: Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsICacheService.createSession]" {file: "resource://gre/modules/OfflineCacheInstaller.jsm" line: 50}]

Comment 12

[:gerard-majax]

This may be broken because of bug 999577, looking at the diff this is the one who added this NS_ERROR_NOT_IMPLEMENTED in CreateSession.

Comment 13

[:gerard-majax]

(In reply to Alexandre LISSY :gerard-majax from comment #12)
> This may be broken because of bug 999577, looking at the diff this is the
> one who added this NS_ERROR_NOT_IMPLEMENTED in CreateSession.

Okay this may be a separate issue :(

Comment 14

[:gerard-majax]

So we are also lacking the root entry point in the cache, i.e., the URL "https://bits.wikimedia.org/WikipediaMobileFirefoxOS/". Hacking this makes the app displays offline.

Comment 15

[:gerard-majax]

Attachment: Make OfflineCacheInstaller use AppCacheStorage

Comment 16

[:gerard-majax]

Attachment: Store launch_path in OfflineCache for AppCache

Comment 17

[:gerard-majax]

Fabrice, hacking an entry matching the launch_path of Wikipedia make it working. But I'm not sure whether adding an entry for launch_path explicitely is the proper way to go to fix this.

Comment 18

[[:fabrice] Fabrice Desré]

(In reply to Alexandre LISSY :gerard-majax from comment #17)
> Fabrice, hacking an entry matching the launch_path of Wikipedia make it
> working. But I'm not sure whether adding an entry for launch_path
> explicitely is the proper way to go to fix this.

Me neither, especially since your patch stores a url with the path to the index.html. We really need Honza to tell us what has changed here.

Comment 19

[Michal Novotny [:michal]]

(In reply to Alexandre LISSY :gerard-majax from comment #12)
> This may be broken because of bug 999577, looking at the diff this is the
> one who added this NS_ERROR_NOT_IMPLEMENTED in CreateSession.

Old cache API was disabled, but obviously some code is still trying to use it. See Honza's blog post to see how to use the new API: http://www.janbambas.cz/http-cache-v1-api-disabled/

Comment 20

[:gerard-majax]

(In reply to Michal Novotny (:michal) from comment #19)
> (In reply to Alexandre LISSY :gerard-majax from comment #12)
> > This may be broken because of bug 999577, looking at the diff this is the
> > one who added this NS_ERROR_NOT_IMPLEMENTED in CreateSession.
> 
> Old cache API was disabled, but obviously some code is still trying to use
> it. See Honza's blog post to see how to use the new API:
> http://www.janbambas.cz/http-cache-v1-api-disabled/

This is already fixed by attachment 8464704 [details] [diff] [review]

Honza, adding a fallback entry in the appcache manifest does the trick and thus the only fix needed is to update OfflineCacheInstaller to use the cache v2 API.

I have no idea whether this is what we want. Wikipedia's launch_path is a directory, not a file, since they assume server-side redirection to index.html.

Comment 21

[:gerard-majax]

And bug 999577 is in v2.0, so this should be 2.0+

Comment 22

[:gerard-majax]

Comment on attachment 8464704 [details] [diff] [review]
Make OfflineCacheInstaller use AppCacheStorage

Please find attached a patch that fixes the issue and makes OfflineCacheInstaller properly using the new Cache v2.

Matching try is green at: https://tbpl.mozilla.org/?tree=Try&rev=81c2186f08d9

Comment 23

[:gerard-majax]

Yuren, using the list of apps from https://github.com/telefonicaid/firefoxos-gaia-spain, I can state the following:
 - without attachment 8464704 [details] [diff] [review]:
  - AccuWeather and Wikipedia do not work
  - no cache is populated at all, and this is expected given that we should be using cache2

 - with attachment 8464704 [details] [diff] [review]:
  - AccuWeather works with the offline notification
  - Wikipedia do not work, because it lacks an entry for its launch_path
  - cache is populated properly besides

I don't think there is more problem. As far as I can tell, launch_path should not point to a directory but rather to a file. So I suspect Wikipedia's manifest is just wrong ?

Comment 24

[:gerard-majax]

In bug 1034730 there is some AppCache testing that has landed, but it seems limited. Maybe we want to improve those tests with this bug :)

Comment 25

[bhavana bajaj [:bajaj]]

(In reply to Alexandre LISSY :gerard-majax from comment #23)
> Yuren, using the list of apps from
> https://github.com/telefonicaid/firefoxos-gaia-spain, I can state the
> following:
>  - without attachment 8464704 [details] [diff] [review]:
>   - AccuWeather and Wikipedia do not work
>   - no cache is populated at all, and this is expected given that we should
> be using cache2
Blocking given this would be a hard businees requirement for partner apps.
> 
>  - with attachment 8464704 [details] [diff] [review]:
>   - AccuWeather works with the offline notification
>   - Wikipedia do not work, because it lacks an entry for its launch_path
>   - cache is populated properly besides
> 
> I don't think there is more problem. As far as I can tell, launch_path
> should not point to a directory but rather to a file. So I suspect
> Wikipedia's manifest is just wrong ?

NI, Harald to help confirm if we need to reachout to wikipedia for the above?

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/b2g-inbound/rev/feef90e576ae

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/feef90e576ae

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/97a32455f28a

Comment 29

[:gerard-majax]

Changing launch_path to add index.html at the end fixed Wikipedia, FYI.

Comment 30

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

seems problem was solved.