897325 - Create web UI for homescreen customization

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Create a web UI for homescreen customization which can create homescreen layout easily. we have a protype on github:

https://github.com/yurenju/gaia-preload-app

you can clone source and use |python wsgi.py| then open http://locahost:8000/ to see this prototype or watch demo on youtube: http://youtu.be/E3s2RbOSbX4

our goal on this bug is migrate the prototype into GAIA_DIR/tools/extensions, user can use |DEBUG=1 make| to build this tool into profile then click a button on Firefox OS developer panel to launch this UI.

note, I would like to use a FileHandler wrapper for file accessing, this app can be a firefox extension with nsIFile or a standalone webpage with XHR to query file on server side.

Comment 1

[Alexandre Poirot [:ochameau]]

Do you need any help in starting writing such addon?
We should be able to help you hook into firefox to host your tool somewhere in its UI.

Do you plan to display it as FF OS control/Screen reader panels? Or in a new tab?

Also, I imagine after the customization is done, we will have to close firefox, rerun DEBUG=1 make and restart firefox to see the updated gaia?
Do you already have a clear workflow how what you want to land in firefox in this bug? (We can iterate with something simple first)

Comment 2

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

(In reply to Alexandre Poirot (:ochameau) from comment #1)
> Do you need any help in starting writing such addon?
> We should be able to help you hook into firefox to host your tool somewhere
> in its UI.
> 
> Do you plan to display it as FF OS control/Screen reader panels? Or in a new
> tab?
I plan to add a button on developer panel (maybe right side of "import workload") and launch it in a new tab because we don't need the phone view on left side.

> Also, I imagine after the customization is done, we will have to close
> firefox, rerun DEBUG=1 make and restart firefox to see the updated gaia?
> Do you already have a clear workflow how what you want to land in firefox in
> this bug? (We can iterate with something simple first)

that's a good question. maybe we can:
1. as you said, quit firefox and re-run |DEBUG=1make|
2. using the same way as the prototype, run a tiny web service only for file access without a extension
3. clean homescreen IndexDB and re-write init.json to homescreen/js/init.json (sounds a bad idea...)

Vivien, do you have any idea?

Comment 3

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

if option 2, we may use |make dashboard| to launch a tiny web server on localhost, and we can use browser to access it.

Comment 4

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

(In reply to Albert [:albert] from bug 929602 comment #6)
> While bug 932223 is fixed, do you know if it's possible to know the amount
> of apps per homescreen at build time? You'll have to face the same problem
> when implementing your tool to configure singlevariant..

that's my plan for steps of this tool:
1. select production or engineer mode, then we get built-in apps
2. add apps by entering manifest url
3. customize preload apps per mnc-mcc, create a mnc-mcc pair and add apps for it.

so I thought we can get amount of apps, but I would like to know what your concern is.

Comment 5

[Albert [:albert]]

(In reply to Yuren Ju [:yurenju] from comment #4)
> (In reply to Albert [:albert] from bug 929602 comment #6)
> > While bug 932223 is fixed, do you know if it's possible to know the amount
> > of apps per homescreen at build time? You'll have to face the same problem
> > when implementing your tool to configure singlevariant..
> 
> that's my plan for steps of this tool:
> 1. select production or engineer mode, then we get built-in apps
> 2. add apps by entering manifest url
> 3. customize preload apps per mnc-mcc, create a mnc-mcc pair and add apps
> for it.
> 
> so I thought we can get amount of apps, but I would like to know what your
> concern is.

That steps are ok, but my concern is how will you determine the amount of apps that can be placed in one screen. As I see, number of apps per screen is set at run time. It is hardcoded to 14 for mobile, 15 for tablet and depending on the screen size it is incremented. So, how you are going to face this in this tool?

Comment 6

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I got it, we hardcode it in grid.js by MAX_ICONS_PER_PAGE. looks we can't know amount of apps for now.

now we generate tablet build by a specific distribution like |GAIA_DISTRIBUTION_DIR=distribution_table make production| and there is homescreen.json in this directory.

Comment 7

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: web-ui-tool.pdf

this is the wireframe for this tool, basicly the same as prototype[1] which I implemented before, but added some feature for preload apps per sim card.

[1] http://www.youtube.com/watch?v=E3s2RbOSbX4

Comment 8

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Alex, is it okay if I use bootstrap and backbone in this tool? that can help me to implement it easier.

Comment 9

[Alexandre Poirot [:ochameau]]

I think that's fine, the plan is still to ship that as an addon right?
An addon integrated with the gaia build one?
Also, that would be really great to see this interface being plugged into the app manager,
so that we can easily push custom gaias to simulator -or- devices!!
I can help you with that and tweak gaia build to integrate itself into the app manager.

Comment 10

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Yes, you are right, I would like to ship this as an addon and that's great, I filed bug 897325 for app manager integraion, we can ship this bug first and then integrate into app manager!

Comment 11

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

some progress... css/html only for now.
http://cssdesk.com/DJrWd

Comment 12

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

WIP today
https://github.com/yurenju/gaia/commit/6e2d3233b5809aa55a53e2a80c035f578d2b2397

Comment 13

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

WIP 11/14, implement adding page, add app to page and get build config (production, engineer) from gaia

https://github.com/yurenju/gaia/commit/d99420e876825e7dfbbc2782f4a06d08152a8322

Comment 14

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

it's Friday night, let's get some cool demo video ;)
http://youtu.be/o2R80ZfayT0


WIP here:
https://github.com/yurenju/gaia/commit/e3e3fafc63bda63b163edffb8784135e02b2b26c

Comment 15

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: github PR: https://github.com/mozilla-b2g/gaia/pull/13792

Alex, could you review this PR? thank you :)

Steps to test:
1. export FIREFOX=<PATH_OF_FIREFOX>
2. cd GAIA_DIR/tools/extensions/[e-mail]
3. make run
4. enable gaia-build extension in addons manager
5. select gaia and choose apps which you want to preload.
6. click preview to launch gaia in browser
7. click save to open distribution directory to see what you generate.

and we will have another two parts of pull request for perlaod apps by sim card and preload apps from marketplace.

Comment 16

[Alexandre Poirot [:ochameau]]

(In reply to Yuren Ju [:yurenju] from comment #15)
> 4. enable gaia-build extension in addons manager

To get rid of this step, you can add that in the makefile:

diff --git a/tools/extensions/[e-mail]/Makefile b/tools/extensions/[e-mail]/Makefile
index 35fa348..ce919ac 100644
--- a/tools/extensions/[e-mail]/Makefile
+++ b/tools/extensions/[e-mail]/Makefile
@@ -4,4 +4,6 @@ xpi:
 run:
 	mkdir -p profile/extensions/
 	echo "../../" > profile/extensions/[e-mail]
+	echo "user_pref(\"extensions.autoDisableScopes\", 14);" > profile/user.js
+	echo "user_pref(\"browser.shell.checkDefaultBrowser\", false);" >> profile/user.js
 	$(FIREFOX) -profile $(shell pwd)/profile -no-remote about:gaia

Comment 17

[Alexandre Poirot [:ochameau]]

Comment on attachment 8333752 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13792

Thanks Yuren, that's a really nice tool!

I made various comments in the pull request, mostly some simplifications...
Please request another review once addressed.

Comment 18

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Alex, I found if redirect chrome://gaia-build/content/aboutGaia.html to about:gaia, I can't use hash tag anymore which is used to show lightbox.

do you have any idea for that?

Comment 19

[Alexandre Poirot [:ochameau]]

Oh yes, about: uris have various limitation.
The easiest way to work around that should be to load a more regular uri, like chrome://gaia-build/content/aboutGaia.html in an iframe so that about:gaia is just a empty document with just this iframe.
I'll try to search for ways to make hashes to work.

Comment 20

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

now I redirect to chrome uri if I got about:gaia:

> if (window.location.href === 'about:gaia') {
>   window.location = 'chrome://gaia-build/content/aboutGaia.html';
> }

I think it's fine because there is no a problem if we integrate into app manager.

Comment 21

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8333752 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13792

Hi Alex, I updated pull request! please take another look, thanks :D

Major changes:
- Added a lightbox to force user choosing gaia folder and added a reset
  button to reset gaia dir.
- Removed <input id="gaia-path"> and store gaiaPath in prefs
- Used element.remove() instead of parent.removeChild()
- Used dataset instead of parsing innerHTML
- Extracted selectBuild()
- Used Observer to notify prefs change
- redirect to chrome uri if got about:gaia

Comment 22

[Alexandre Poirot [:ochameau]]

Comment on attachment 8333752 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13792

Works great, thanks for this nice tool!

Comment 23

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

part 1 merged.
https://github.com/mozilla-b2g/gaia/commit/e28aeafc70f11f9c72359b766dc1fd6266f43dd2

Comment 24

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

BTW I made a badges for early adopter of customization tool
https://badges.mozilla.org/en-US/badges/badge/Customization-Early-Adopter

just have fun :)

Comment 25

[Albert [:albert]]

Nice work!

I miss the configuration based on mcc/mnc and the config of ringtones, wallpapers, etc. There are plans to add these features?

Comment 26

[Francisco Jordano [:arcturus] [:francisco]]

Looking forward to start using it :)

Thanks a lot Yuren!

Comment 27

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Albert, No, but we can file follow-up to do that, and we also need UX input to re-design the flow and polish UI and that could be another follow up issue.

Francisco, feature for installing app from marketplace is coming soon! that would be the second part pull request on this issue.

Comment 28

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Albert, how do we set a default preload apps without mcc/mnc? like if I want to install wikipedia for all mcc/mnc. I know we can put wikipedia in DISTRIBUTIOIN_DIR/external-apps, but I still want to know how did you handle this situation, thanks.

Comment 29

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

WIP here: https://github.com/yurenju/gaia/commit/5d3f4c988b5481c516c0f5cab12341f38deae16b

now we can search marketplace and preload it, that looks cool :D
https://www.dropbox.com/s/qf8nduxj9yoor4o/%E8%9E%A2%E5%B9%95%E6%88%AA%E5%9C%96%202013-11-22%2004.47.15.png

and I still don't know how to use variant.json to add default preload apps for all mcc/mnc. I will attach file which is generated by customization tool later.

Comment 30

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: variant.json

Comment 31

[Albert [:albert]]

(In reply to Yuren Ju [:yurenju] from comment #28)
> Albert, how do we set a default preload apps without mcc/mnc? like if I want
> to install wikipedia for all mcc/mnc. I know we can put wikipedia in
> DISTRIBUTIOIN_DIR/external-apps, but I still want to know how did you handle
> this situation, thanks.

As you said, if you want to install an app for all mcc/mnc you have to put it in external-apps folder, it is not a single variant application because it is going to be installed for any mcc/mnc. However, you can add the application to each element of the operator's array if you want to set it in your variant.json

Comment 32

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Albert, I found we use http url for packaged app in origin field of metadata.json. do we support packaged app for preload apps per sim card?

IIRC we need to use 'app://APP_NAME' for packaged app in origin field.

Comment 33

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

oh I got it, we need reference variant.json to make sure origin is app://APP_NAME.

Comment 34

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: github PR: https://github.com/mozilla-b2g/gaia/pull/13978

this is pull reuqest part 2 for preload apps from marketplace.

and we will totally resolve this bug if we land feature part 3 which is preload apps per sim card.

Comment 35

[Alexandre Poirot [:ochameau]]

Comment on attachment 8337398 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13978

Looks good overall. I'm seeing some UX issues that would be ideally fixed before landing. Mainly the packaged app story.

Comment 36

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8337398 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13978

Hi Alex, I updated Pull request and added packaged app label and noted in search result. please take a look again :D

Comment 37

[Alexandre Poirot [:ochameau]]

Comment on attachment 8337398 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13978

When I install the app called "Where is package?", I get the following exception when trying to run the preview:
  Error: App doesn't exist: whereispackage aboutGaia.js:462
And firefox doesn't launch.

Some more code tweaks and just this issue to at least understand.

Comment 38

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I use a wrong origin with space and it is fixed.

Comment 39

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I also found a bug for preloading packaged app and filed bug 943730

Comment 40

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8337398 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13978

let's try again, I refactored downloadApp and remove redundant arguments and did some minor fixing.

Comment 41

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I also added packageEtag support for metadata.json, please see this commit:
https://github.com/yurenju/gaia/commit/67f35e11f41b5442180f9684755d08c685caad93

Comment 42

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Part 3 WIP here:
https://github.com/yurenju/gaia/commit/ae38f1d789e53f1273a777eb8bd7855f7714f7e0

demo video for preload apps per sim card:
http://youtu.be/W81ELDG9eEY

Comment 43

[Alexandre Poirot [:ochameau]]

Comment on attachment 8337398 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/13978

Good to land with some last tweaks.
Thank Yuren!

I was wondering if you have a plan after this bug and its last "preloaded sim apps" step?
I'm concerned about the overall UX of this tool:
 - For example, it isn't obvious you have to click on the red strip to select a page.
 - I have the feeling that many errors are just ignored, not logged at all. (like any error happening during profile creation, that would be great to display its output and progress)
 - Better error report (console.warn and alert are the poor's man error reporting, maybe we should have a console where we dump all warning/errors)
 - We should try to match App manager theme and match its look'n feel. Could be part of bug 935811 goals.

I'm concerned mainly because all these tweaks won't bring any new feature, it is only about polish. So I hope we will have some resources allocated for that!

And what's next? Is there some next big features you want to add? Do we plan to give this tool to partners? If yes, when? Do we plan to release this addon on AMO?

Comment 44

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

We definitely need to polish UX and visual, but as you saw I don't have a lot of resource to implement this tool even I drew wireframe from myself not from UX, another problem is I can't delicate to work on this tool.

it definitely can help partner to do customization easily but as you said we need more resources for it.

I think Peter and Karen are interested in this tool. Peter, do we have chance to put more resource into this tool?

BTW, there are featue list for customizatoin: https://wiki.mozilla.org/B2G/MarketCustomizations

Comment 45

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

And I will also discuss with my manager for it.

Comment 46

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Part 2 merged.
https://github.com/mozilla-b2g/gaia/commit/44c20536f22ab3a38db74889a36bf7d5b89939a0

Comment 47

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

MCC/MNC information:
http://www.mcc-mnc.com/

Comment 48

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: github PR: part 3 - https://github.com/mozilla-b2g/gaia/pull/14175

Part 3: preload apps for certain carriers

Comment 49

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Preload apps for certain carriers is a feature which can allow user/operator preload apps for certain MCC/MNC, for example if we want to preload twitter for AT&T in US and MNCs for AT&T are 150, 170, then you can put variant.json into distribution directory with:

> {
>   "apps": {
>     "twitter": 
>       {
>         "origin": "https://mobile.twitter.com",
>         "manifestURL": "https://mobile.twitter.com/cache/twitter.webapp",
>         "installOrigin": "https://marketplace.firefox.com",
>         "etag": "\"3471914f48f497d06248794ae4d4dc2d\""
>       }
>   }
>   "operators": [
>     {
>       "id": "AT&T",
>       "mcc-mnc": ["310-150", "310-170"],
>       "apps": [
>         { "id": "twitter", "screen": 1, "location": 4 }
>       ]
>     }
>   ]
> }

when first time user boot up device, FTU will detect MCC/MNC and enable certain apps for matched MCC/MNC

this feature of customization tool is only available for apps which download from marketplace (or other places but currently we only support marketplace), so you will see 'Built-in app can't preload for certain carriers." when you click a built-in app.

when click an app which downloaded from marketplace, you can enable this app for certain MCC/MNC pairs.

for variant.json example you can reference variant.json in qa-testcast-data.

https://github.com/mozilla/qa-testcase-data/blob/gh-pages/customization/reference/variant.json

Comment 50

[Alexandre Poirot [:ochameau]]

Comment on attachment 8339900 [details] [review]
github PR: part 3 - https://github.com/mozilla-b2g/gaia/pull/14175

Looks good, just some code style nits.

But you may want to f?/r? someone that actually masters variant-mcc-mnc stuff.
I have no idea how to test that, and don't know much about variant.json.

Comment 51

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8339900 [details] [review]
github PR: part 3 - https://github.com/mozilla-b2g/gaia/pull/14175

Hi Albert, Could you give some feedback for this tool? 

you can go to GAIA_DIR/tools/extensions/[e-mail] and use |make xpi| to get xpi file and install it on nightly.

when add a app into a page, you can click app icon to enable this app for certain MCC/MNC, and click save can get distribution files include variant.json.

I'll upload a variant.json which is generated by this tool.

Comment 52

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Attachment: variant.json which is generated by this customization tool

Comment 53

[Jason Smith [:jsmith]]

This is not a committed feature for 1.3, so this should not block the release.

Comment 54

[Albert [:albert]]

I can't configure mcc/mnc for built-in apps and when trying to add some apps from market I see the popup with market apps but can't click on then to add to screens. Am I doing something wrong?

Comment 55

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

We definitely need to polish UX, right? :-)

here is a demo video for preload apps per sim card.
http://youtu.be/AQ7es26drTg

Comment 56

[Albert [:albert]]

Comment on attachment 8339900 [details] [review]
github PR: part 3 - https://github.com/mozilla-b2g/gaia/pull/14175

I have three comments about the app, only the first one is critical..

I see a problem when introducing mcc/mnc, popup says: Use commas to split different mcc/mnc pairs, e.g., "310-410, 310-260". So I put "214-001, 214-002, 214-005, 214-007" and it is set in variant.json as "\"214-001","214-002","214-005","214-007\"".

Regarding to mcc/mnc format, I know that gecko works with 3 digits for each, but maybe you can let operators to put it like 214-07 or 214-7 and fill with zeros when necessary.

Is a bit difficult to configure different apps in the same screen, if I want to put twiter for Movistar spain at place 1 and facebook for latam at place 1 I have to put twiter first and facebook at place 2 and then configure the mcc/mnc pairs, but is difficult to have an overview of the configuration. Maybe will be better to have a select to change the whole view for a given mcc/mnc. This is something UX team should review :P

Comment 57

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

I added more description for MCC/MNC and use regex to check it.

let's resolve this bug first and  then we can dicuss for more details :-)

Comment 58

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Merged! \o/

https://github.com/mozilla-b2g/gaia/commit/c16190cea49a4bcd414e9ff4614e1fe886775165

Comment 59

[Tim Guan-tin Chien [:timdream] (please needinfo)]

(In reply to Jason Smith [:jsmith] from comment #53)
> This is not a committed feature for 1.3, so this should not block the
> release.

There was some confusion on flagging committed feature v.s. targeted feature. We should use user story bug and meta bugs to track targeted feature instead of blocking flag.

Thanks for the good work, Yuren, by the way.