899079 - [User Story] Build process customization

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[Noemí Freire (:noemi)]

As an operator I'd need OEMs to customize the build process to include all the third party apps specified by several associated operators. The apps will be separated into two sets, a Common set (apps that are common to all the associated operators) and several Local sets (one per operator)

Acceptance Criteria: 
* There's a way to specify at build time a set of associated operators (SAO) to which the build applies. The operators will be identified by MCC and MNC
* There's a way to specify at build time a set of common third party apps (SCTPA) for the SAO
* There's a way to specify at build time a set of local third party apps (SLTPA) for each of the operators included in the SAO
* There will be a single SAO per build, a single SCTPA per build, and as many SLTPAs as operators included in the SAO
* The apps will be identified at build time only by their manifest URLs
* All the specified apps will be automatically downloaded at build time, and installed on the appropiate directory on the device

Comment 1

[Gregor Wagner [:gwagner]]

I think Yuren is currently working on a customer/operator facing UI.

Comment 2

[Antonio Manuel Amaya Calvo (:amac)]

This is not the UI... I think the UI Yuren is working on will provide a easier way to create the configuration files for the build. This build is to use those configuration files (created manually or with Yuren's tool) to create the image that get flashed on the phone. 

Please correct me if I'm wrong :)

Comment 3

[Albert [:albert]]

Attachment: SPAIN JSON file example to configure apps for operators depending on the mcc and mnc

Manifest url should be changed by the server where the manifests are served

Comment 4

[Albert [:albert]]

Attachment: Manifests to use with the JSON configuration file provided as a example

Comment 5

[Albert [:albert]]

Attachment: Patch

If installing gaia only sync with https://github.com/telefonicaid/firefoxos-gaia-spain repo and follow the README or copy the repo to $GAIA_HOME/distribution.

If making a build copy repo content to $GAIA_HOME/distribution and build as usuall with ./build.sh

Comment 6

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Fabrice, IIRC we have discussed about fetching app on build time and you didn't like this idea, but I don't remeber the reason. maybe you can give some feedback on this issue.

Comment 7

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

and we have preload apps management mechanism for picking apps into gaia. FYI: https://wiki.mozilla.org/B2G/MarketCustomizations#Preload_App_management

Comment 8

[Carmen Jimenez Cabezas]

Comment on attachment 786197 [details] [diff] [review]
Patch

Review of attachment 786197 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good to me, thanks Albert.

::: Android.mk
@@ +84,4 @@
>  	rm -rf $(GAIA_PATH)/profile/indexedDB; \
> +	fi; \
> +	if [ -d $(GAIA_PATH)/profile/svoperapps ]; then \
> +	rm -rf $(GAIA_PATH)/profile/svoperapps; \

Why are you deleting this? webapps isn't deleted and this directory should have the same treatment, shouldn't it?

::: build/local-apps.py
@@ +27,5 @@
> +
> +    """ Download manifest """
> +    print 'Download manifest ', manifest_url
> +    manifest_raw = (urllib2.urlopen(manifest_url)).read()
> +    manifest = json.loads(manifest_raw)

I've had some problems with Python json library and JSONs that are written with DOS endline or endfile characters. Have you tested this with those kind of files?

@@ +118,5 @@
> +
> +    if not options.local_apps_path:
> +        parser.error('local-aps-path not given')
> +
> +    if not options.profile_path:   

Nit: trailing white spaces

Comment 9

[Carmen Jimenez Cabezas]

Thanks Yuren!

This isn't strictly related with 3rd party apps, though, this is to include on the device a set of apps that will be, or won't be installed, depending on the actual SIM card used on the first run of the device. This allows us to have a single image for a set of operators that want to have different apps preinstalled on the device. So a static list as the one you referenced won't cut it.

Comment 10

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

OK, and don't forget to document it on wiki when this change landed :D

Comment 11

[Candice Serran (:cserran)]

Noemi Freire changed story state to started in Pivotal Tracker

Comment 12

[Albert [:albert]]

(In reply to Carmen Jimenez Cabezas from comment #8)
> Comment on attachment 786197 [details] [diff] [review]
> Patch
> 
> Review of attachment 786197 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> This looks good to me, thanks Albert.
> 
> ::: Android.mk
> @@ +84,4 @@
> >  	rm -rf $(GAIA_PATH)/profile/indexedDB; \
> > +	fi; \
> > +	if [ -d $(GAIA_PATH)/profile/svoperapps ]; then \
> > +	rm -rf $(GAIA_PATH)/profile/svoperapps; \
> 
> Why are you deleting this? webapps isn't deleted and this directory should
> have the same treatment, shouldn't it?

It is deleted to clean profile, if you configure 4 apps and next time 3 and it is not cleaned, you will have 4 apps again. webapps is cleaned in other line.

> ::: build/local-apps.py
> @@ +27,5 @@
> > +
> > +    """ Download manifest """
> > +    print 'Download manifest ', manifest_url
> > +    manifest_raw = (urllib2.urlopen(manifest_url)).read()
> > +    manifest = json.loads(manifest_raw)
> 
> I've had some problems with Python json library and JSONs that are written
> with DOS endline or endfile characters. Have you tested this with those kind
> of files?

Written in linux and tested.

Comment 13

[Albert [:albert]]

Attachment: LATAM JSON file example to configure apps for operators depending on the mcc and mnc

Comment 14

[[:fabrice] Fabrice Desré]

(In reply to Yuren Ju [:yurenju] from comment #6)
> Fabrice, IIRC we have discussed about fetching app on build time and you
> didn't like this idea, but I don't remeber the reason. maybe you can give
> some feedback on this issue.

I don't like relying on remote network resources during the build phase. Is this what we are doing here?

Comment 15

[Antonio Manuel Amaya Calvo (:amac)]

This is something that will be used only for the operator variants and what it does is download the apps if they haven't been downloaded yet or use the downloaded ones otherwise (or that's what it should do anyway). So... it's akind to what we do with the xulrunner actually. And those resources don't belong on the Gaia repo (since they're third party apps) and must be downloaded somehow anyway.

Comment 16

[Albert [:albert]]

Attachment: Patch

Comment 17

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Comment on attachment 788090 [details]
Patch

I decline to review this massive patch because it tried to do too many things at once. Please, file dependent engineering bugs and land your patches there, instead of putting everything on a user story bug.

Reading all the way down and the code briefly I found that you are trying to do at least this many things _in Gaia_:

A) Having Gaia to parse a JSON file in distribution repo and download apps if it does not exist locally.
B) Having Gaia to generate the same JSON file and generate a singlevariantconf.json for homescreen so that it could hide/show app based on mmc/mnc.

I would against using the same JSON for both purpose, AND using the same Python script for both purpose too. (B) should really go to applications-data.js as all current customization do, and (A) should be kept in the external preload.py until we figure out what we should do.

We would need to address (B) in Settings app too. Hiding an app has security implication and should be dealt in a clear way so we don't mess up.

Nice work on heavy lifting everything though, but I wish patch is written before everyone is happy with the apporach.

Comment 18

[Noemí Freire (:noemi)]

Per comment 17 the following bugs have been opened:

Bug 904004 - [Single Variant] Manage custom single variant apps
Bug 904005 - [Single Variant] Generate single variant JSON
Bug 904006 - [Single Variant] Integrate single variant in Gaia build scripts

Comment 19

[Albert [:albert]]

Dependencies resolved

Comment 20

[Candice Serran (:cserran)]

ALBERTO CRESPELL PEREZ changed story state to accepted in Pivotal Tracker