[TV][2.5] Preloaded hosted app offline support did not work

RESOLVED FIXED

Status

P1
normal
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jyeh, Assigned: Fischer)

Tracking

unspecified
ARM
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:2.5+, b2g-v2.5 fixed, b2g-master fixed)

Details

(Whiteboard: [ft:conndevices][partner-blocker][partner-cherry-pick])

Attachments

(1 attachment, 3 obsolete attachments)

(Reporter)

Description

3 years ago
When offline, open the preloaded hosted app for the first time (the app had never online before), screen shows server not found page instead of the preloaded cached page.

It works normally if the app had online before.

STR:

1. Use preload.py to bundle webapp from a given .webapp URL (e.g. https://bits.wikimedia.org/WikipediaMobileFirefoxOS/app.appcache).

2. Put the preloaded files under gaia/external-apps/ folder.

3. add the path external-apps/* to apps-engineering.list .

4. Rebuild gaia.

5. Make sure the preloaded files appear in profile/webapps folder.

6. Make your device offline.

7. Open the app from app deck.


Actual Result: 
 - Showing server not found page

Expected Result:
 - Showing preloaded cached page
(Reporter)

Comment 1

3 years ago
Hi Rickey, are there anything wrong with the process of preloading hosted app?
Flags: needinfo?(swu)
Flags: needinfo?(rchien)
(Assignee)

Updated

3 years ago
Assignee: nobody → fliu
(Assignee)

Comment 2

3 years ago
Created attachment 8694067 [details]
preload_appcache_example_wikipedia.zip

Hi Fabrice,

We are trying to preload hosted app with app cache by following the instructions from https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Market_customizations_guide#Building_Prebundled_web_apps

After running DEVICE_DEBUG=1 make command, we can find our example hosted wikipedia app inside ./profile/webapps/ and it contains the cache folder having resources pointed by the app.appcache file.

However, the appcache dosen't work when we open our hosted wikipedia app 1st time without internet connected.
We expect the pre-cached content shows up but actually it shows the message of no connection.

This issue happens both on the mulet and on a real Flame phone.
In fact, we could even see the message saying the cache is pushed to the Flame phone, like:

>  push: /Users/foxbrush/Projects/tmp/tmp_gaia/profile/webapps/{599095a7-8b4b-484d-af10-87eb019cf1fc}/cache/WikipediaMobileFirefoxOS/js/lib/MobileFrontend/javascripts/filepage.js
>  -> //system/b2g/webapps/{599095a7-8b4b-484d-af10-87eb019cf1fc}/cache/WikipediaMobileFirefoxOS/js/lib/MobileFrontend/javascripts/filepage.js
>

We are guessing maybe there is something under the Gecko we are missing.

Could you please look at this issue?

Thank you

p.s:
1) Our real purpose is to use this preload hosted app with appcache technique on the TV marketplace app so user can still see something on the TV marketplace even without internet.

2) The attachment: preload_appcache_example_wikipedia.zip is the hosted wikipedia app downloaded by the preload-app-toolkit. We are using it as the test example.
Flags: needinfo?(fabrice)

Comment 3

3 years ago
For Linux case, the appcache still doesn't work after the app was opened online and closed by window.close().
It shows only one word "error" 

Both case are with message "XXX FIXME : Dispatch a webapps-launch: undefined" from
https://dxr.mozilla.org/mozilla-central/source/b2g/components/SystemAppProxy.jsm#128
[Blocking Requested - why for this release]:

This blocks TV marketplace for 2.5.
Blocks: 1204762
blocking-b2g: --- → 2.5?
Flags: needinfo?(swu)

Updated

3 years ago
Priority: -- → P1
(In reply to Fischer from comment #2)
Hi Henry,
Could you specify the code of AppCache in Gecko that Fischer can debug?
Flags: needinfo?(hchang)

Comment 6

3 years ago
(In reply to Junior [:junior] (ooo 12/2-12/4) from comment #3)
> For Linux case, the appcache still doesn't work after the app was opened
> online and closed by window.close().
> It shows only one word "error" 
> 
This is nothing to do with this bug. The reason is |navigator.onLine| always returns true.

> Both case are with message "XXX FIXME : Dispatch a webapps-launch:
> undefined" from
> https://dxr.mozilla.org/mozilla-central/source/b2g/components/SystemAppProxy.
> jsm#128

Updated

3 years ago
blocking-b2g: 2.5? → 2.5+
The code in gecko that actually loads this content in the cache is at https://mxr.mozilla.org/mozilla-central/source/dom/apps/OfflineCacheInstaller.jsm

First thing to do would be to enable the debug messages at line 36
Flags: needinfo?(fabrice)
(Assignee)

Comment 8

3 years ago
Hi Fabrice,

We found that 
1) installCache is never called @Line:211, https://mxr.mozilla.org/mozilla-central/source/dom/apps/OfflineCacheInstaller.jsm

2) The DOMApplicationRegistry object at Webapps.jsm would call OfflineCacheInstaller.installCache through the updateOfflineCacheForApp method on init call @Line:221, https://dxr.mozilla.org/mozilla-central/source/dom/apps/Webapps.jsm
However, we did not see the call of DOMApplicationRegistry.init.
As a result, the DOMApplicationRegistry.updateOfflineCacheForApp and OfflineCacheInstaller.installCache are never called from our observation.

The above 2 situations happen both during 
  - the 1st time of the app launch without internet and 
  - launch the app with internet to have appcache downloaded once and then open the app again without internet

However, during app launch we can see DOMApplicationRegistry.launch is called, @Line1666, https://dxr.mozilla.org/mozilla-central/source/dom/apps/Webapps.jsm
We also can observe the app object is created with the kind property set to 'hosted-appcache'.

Could you please look at some questions below:
1) Does the kind property with 'hosted-appcache' represent the preload hosted app is recognized as the hosted app with preloaded appcache ?

2) When would DOMApplicationRegistry.init be called ? Why we can not observe its calling ?

Thanks for your help.
Flags: needinfo?(fabrice)
DOMApplicationRegistry.init() is only called once when b2g starts. If this is the very first start or after an OTA update, we also run code to initialize the application registry, including https://dxr.mozilla.org/mozilla-central/source/dom/apps/Webapps.jsm#824 that will preload the app-cache. Are you seeing that being called on first start? You need a fresh profile.

You're right that kind == 'hosted-appcache' means that we recognized this app as having an app cache. We base that on the appcache_path presence in the app manifest though, not on the preloading step.
Flags: needinfo?(fabrice)
(Assignee)

Updated

3 years ago
Depends on: 1230091
(Assignee)

Comment 10

3 years ago
Hi Fabrice,

We found a ReferenceError in OfflineCacheInstaller.jsm which would cause the operation of loading metadata is unable to proceed.
Please see bug 1230091.

However, even fixing that ReferenceError, the appcache still does not show for the 1st time launch without internet.

We can see inside OfflineCacheInstaller.jsm:
- During the b2g starts, the installCache function is invoked and the metadataLoaded promise is resolved.
- After metadata loaded, then storeCache function is called.
- Then, the onCacheEntryAvailable event handler is called @https://dxr.mozilla.org/mozilla-central/source/dom/apps/OfflineCacheInstaller.jsm#58
- Inside the onCacheEntryAvailable handler, we can see the cache files are loaded, such as:
> 12-03 03:45:52.676: I/Gecko(19694): -*-*- OfflineCacheInstaller.jsm :
>   /system/b2g/webapps/{7022c093-6e5c-6b4e-aa5b-2216a18743fa}/cache/WikipediaMobileFirefoxOS/404.html
>   -> https://bits.wikimedia.org/WikipediaMobileFirefoxOS/404.html (2)
> 12-03 03:45:53.016: I/Gecko(19694): -*-*- OfflineCacheInstaller.jsm :
>   /system/b2g/webapps/{7022c093-6e5c-6b4e-aa5b-2216a18743fa}/cache/WikipediaMobileFirefoxOS/css/image/ios/toolbar-forward.png
>   -> https://bits.wikimedia.org/WikipediaMobileFirefoxOS/css/image/ios/toolbar-forward.png (2)
  @https://dxr.mozilla.org/mozilla-central/source/dom/apps/OfflineCacheInstaller.jsm#85

If we open the hosted Wikipedia app with internet, we can see logs like
> 12-03 05:13:44.614: I/Gecko(21903): [Child 21903] WARNING: Could not get disk status from nsIDiskSpaceWatcher:
>   file /home/fischer/Projects/gecko-dev/uriloader/prefetch/nsOfflineCacheUpdateService.cpp, line 319
> 12-03 05:14:02.824: I/GeckoConsole(19694): Offline cache update done, URL=https://bits.wikimedia.org/WikipediaMobileFirefoxOS/app.appcache
From the logs, it seems the nsOfflineCacheUpdateService.cpp and nsOfflineCacheUpdate.cpp try to get and update offline cache.

Then turning off internet, open the app again. We can see similar operations again.
> 12-03 05:14:28.894: I/Gecko(21933): [Child 21933] WARNING: Could not get disk status from nsIDiskSpaceWatcher: 
>   file /home/fischer/Projects/gecko-dev/uriloader/prefetch/nsOfflineCacheUpdateService.cpp, line 319
> 12-03 05:14:31.544: I/GeckoConsole(19694): Offline cache update error, URL=https://bits.wikimedia.org/WikipediaMobileFirefoxOS/app.appcache

1) Does the call of the onCacheEntryAvailable handler means the preloaded cache is found and loaded successfully ?
2) Are the nsOfflineCacheUpdateService.cpp and nsOfflineCacheUpdate.cpp in charge of loading and serving appcache for web ?

Thanks for the help
Flags: needinfo?(fabrice)
(Assignee)

Comment 11

3 years ago
Created attachment 8695239 [details]
app_cache_log_20151203_1819.txt
Your description looks correct to me. I think one issue may be that you are trying to add https uri in the cache, but you don't have the security info at this time so we don't retrieve them later properly without hitting the network.

Can you try with a site served from http instead?
Flags: needinfo?(fabrice)
(Assignee)

Comment 13

3 years ago
Hi Fabrice,

The issue exists with a site served from http as well.
However, we didn't see the call from nsOfflineCacheUpdateService.cpp when opening the http-hosted preload app.
Besides this, we didn't see the call from AppCacheStorage.cpp either.

Where might be the entry point that gecko starts to find and load offline app cache ?

Thank you for the help.
Flags: needinfo?(fabrice)
Honza, is there any log level that we can turn on to see what's happening there?
Flags: needinfo?(fabrice) → needinfo?(honzab.moz)
Hmm.. there are, but may need a trained eye to overlook.  The modules are:  cache2:5,cache:5,nsOfflineCacheUpdate:5.  It's also good to add 'timestamp' (w/o any :N suffix) module to get timings in the log.
Flags: needinfo?(honzab.moz)
(Assignee)

Comment 16

3 years ago
Created attachment 8696599 [details]
preload_appcache_http_example_tvmarketplace.zip
STR is correct. Clear the needinfo request since Fischer is working on it. if you have further question please ni me =)
Flags: needinfo?(rchien)
Some random info:
 - Fischer said there is not sqlite db for appcache after building.
 - cache2 has move cache directory to $(HOME)/.cache from user's profile directory.
 - We has a new building system for b2g.
(In reply to Thinker Li [:sinker] from comment #18)
> Some random info:
>  - Fischer said there is not sqlite db for appcache after building.
>  - cache2 has move cache directory to $(HOME)/.cache from user's profile
> directory.
>  - We has a new building system for b2g.

None of that should be a problem since we don't prebuild the cache. We ship all the files in a $app/cache/ directory and recursively add them all to the cache at first run or after a system update.
(Assignee)

Comment 21

3 years ago
Created attachment 8697324 [details] [diff] [review]
correct the generation of  group ID

This patch correct the generation of group id so offline cache could be loaded with the right group id
Attachment #8694067 - Attachment is obsolete: true
Attachment #8695239 - Attachment is obsolete: true
Attachment #8696599 - Attachment is obsolete: true
Attachment #8697324 - Flags: review?(fabrice)
Attachment #8697324 - Flags: review?(fabrice) → review+
(Assignee)

Updated

3 years ago
Keywords: checkin-needed
(In reply to Shian-Yow Wu [:swu] from comment #4)
> [Blocking Requested - why for this release]:
> 
> This blocks TV marketplace for 2.5.

mahe, josh i guess this need your approval
Flags: needinfo?(mpotharaju)
Flags: needinfo?(jocheng)
Hi Fischer,
Please raise 2.5 Gaia uplift request for approval. Thanks
Flags: needinfo?(jocheng) → needinfo?(fliu)
(Assignee)

Updated

3 years ago
Flags: needinfo?(fliu)
Whiteboard: [ft:conndevices][partner-blocker][partner-cherry-pick]
(Assignee)

Comment 24

3 years ago
Comment on attachment 8697324 [details] [diff] [review]
correct the generation of  group ID

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): The wrong group id
User impact if declined: Preloaded offline cache wouldn't work so user wouldn;t be able to see content of TV marketplace at the 1st time launch if no internet connection.
Testing completed: Yes
Risk to taking this patch (and alternatives if risky): Low 
String or UUID changes made by this patch: None
Attachment #8697324 - Flags: approval‑mozilla‑b2g44?
(Assignee)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Comment on attachment 8697324 [details] [diff] [review]
correct the generation of  group ID

Approve for TV 2.5
Attachment #8697324 - Flags: approval‑mozilla‑b2g44? → approval‑mozilla‑b2g44+
(Assignee)

Comment 26

3 years ago
Hi Carsten,
Could you please help to land this patch?
thanks
Flags: needinfo?(cbook)

Updated

3 years ago
status-b2g-v2.5: --- → affected
status-b2g-master: --- → fixed
landed on 2.5 as https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/ff08f85000b5
status-b2g-v2.5: affected → fixed
Flags: needinfo?(cbook)
(Assignee)

Updated

3 years ago
Flags: needinfo?(cbook)
(Assignee)

Comment 28

3 years ago
Hi Carsten,

The patch is not landed to the master.
Could you please help to land ?
Thanks
(In reply to [:Fischer]Fischer from comment #28)
> Hi Carsten,
> 
> The patch is not landed to the master.
> Could you please help to land ?
> Thanks

yeah sure :) done !

https://hg.mozilla.org/integration/b2g-inbound/rev/4cfcce1089fb
Flags: needinfo?(cbook)
Keywords: checkin-needed
Removing NI
Flags: needinfo?(mpotharaju)
Flags: needinfo?(hchang)
You need to log in before you can comment on or make changes to this bug.