1155619 - We need a way to prepopulate a service worker cache

Status: RESOLVED DUPLICATE of bug 1156198

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Fernando Jiménez Moreno [:ferjm]]

Just like we did with OfflineCache, we need a way to prepopulate a service worker cache.

Comment 1

[Salvador de la Puente González [:salva]]

We need a name for the cache, by the way. :P

Comment 2

[Francisco Jordano [:arcturus] [:francisco]]

Hi Tim, Yura

do you think that the team owning the build scripts can give us a hand providing a method for prepopulating a cache (given a name as Salvador comments) from sw?

Comment 3

[Yura Zenevich [:yzen] (please use "needinfo")]

I think that would be awesome. 

One interesting, but understandable limitation of appcache is that we need to explicitly list all resources that need caching (e.g. no wildcards). This has consequences for pre-caching because unlike regular cache we need to know what to pre-fetch. It would be awesome to be able to, somehow, pre-fetch all of the app resources without the need to list all of the explicitly. Not sure if it's even feasible though, but something to consider.

Comment 4

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Francisco,

I would imagine the list and resources to push into image will be similar to that of AppCache, but the syntax would definitely be different.

Do you have the bug filed to FxOS::Runtime to consume the preload data? Who would be the person working on Gecko part? I think we should just start writing specs on the file structure of the app for build script to generate and for Gecko to consume. For example, for a packaged app it is as simple as:

/b2g/system/webapps/appname.gaiamobile.org/manifest.webapp
/b2g/system/webapps/appname.gaiamobile.org/application.zip (includes everything inc. manifest.webapp)

Once Gecko and Gaia build engineers are agreed on each other we could start make our code agree with each other as well.

Could you start working on the initial spec?

Comment 5

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Ricky, could you work with Francisco and evaluate the feasibility of the spec stated above? Thanks!

Comment 6

[Tim Guan-tin Chien [:timdream] (please needinfo)]

BTW Ricky, do we have some spec already written for AppCache-backed and Packaged apps? Would it make sense to start writing them down?

Comment 7

[[e-mail]]

FYR here is what we've documented for pre-fetch AppCache-backed hosted apps

https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Market_customizations_guide#Building_Prebundled_web_apps

Comment 8

[Francisco Jordano [:arcturus] [:francisco]]

Hi folks, thanks for all the info.

I think the process will be pretty similar, so what I'm trying to do now is to explain different techniques that could be used in service workers to fill a cache, and also will try to explain how the cache is structured (I'm sure for this part Ben Kelly can explain better than me, but will try and then will ask Ben to correct me :) ).

- Installing the SW:
There is a first step that is installing the SW and will happen once we register it from a window object. Until a SW is registered it cannot handle any request.
This is an example on how we usually ask for the installation of a SW:
https://github.com/arcturus/serviceworkerware/blob/master/demo/index.html#L8-L13
if (navigator.serviceWorker) {
        navigator.serviceWorker.register('./sw.js');
      } else {
        console.info('No SW present :(');
      }

Which means, please my SW code is on ./sw.js.

We can track the registration of the sw in the following url: about:serviceworkers (Nightly only), also the profile gets modified in the following way:
+ <profile_folder>/serviceworker.txt is created
+ it contains the information for each sw installed
+ an example for a domain is the following:
1
content
0
false
http://localhost:6666/
http://localhost:6666/
http://localhost:6666/sw.js
http://localhost:6666/sw.js
{f5ab416a-c763-014d-a3e8-19e1e6b1b704}

#

- Running the SW:
The sw has a life cycle (http://www.html5rocks.com/en/tutorials/service-worker/introduction/images/sw-lifecycle.png) installation > activation > fetch/message > die.

During the installation, is phase is when the caches use to be filled, there are several examples how to do this, at the end of the day most strategies are based on getting a list of files and adding them to the cache you are interested (each cache has a name). Whenever you get those files from a zip file, github, or your own domain is irrelevant, we need connectivity to do a first fetch for each resource we want to cache.

This will end up in the profile in the following way:
<profile_folder>/storage/default/{protocol}++++{domain}+{port}/cache

And there you can find a sqlite db and files stores in the cache, with it's specific format.

So that's what happens when we execute a SW while navigating to a web page, we should try to find an answer to how to get our profile generated with that information at build time.

Comment 9

[Francisco Jordano [:arcturus] [:francisco]]

Hi Ben,

I'm ni you cause I roughly explained, from my experience just observing the fs, how SW interact with the user profile.

Please if I made any mistake explaining anything in #comment8 feel free to correct anything.

Our goal for this bug is to prefill a profile for b2g with the gaia apps so we can start using sw but having the apps ready to use when they are launched for the first time without any internect connectivity.

Comment 10

[Ben Kelly [:bkelly, not reviewing]]

We have some conversation of possible implementations for this in bug 1156198, so I'm dup'ing there.  Its going to need some special logic to handle the install event on first load.