Closed Bug 1058555 Opened 10 years ago Closed 10 years ago

Updating preloaded apps in FOTA/OTA results in duplicate apps in /data/local/webapps

Categories

(Firefox OS Graveyard :: Gaia::Build, defect)

x86
macOS
defect
Not set
normal

Tracking

(blocking-b2g:1.4+, b2g-v1.4 fixed, b2g-v2.0 fixed, b2g-v2.1 fixed, b2g-v2.2 fixed)

RESOLVED FIXED
2.1 S4 (12sep)
blocking-b2g 1.4+
Tracking Status
b2g-v1.4 --- fixed
b2g-v2.0 --- fixed
b2g-v2.1 --- fixed
b2g-v2.2 --- fixed

People

(Reporter: pehrsons, Assigned: yurenju)

References

Details

Attachments

(4 files)

Steps to reproduce:
1. Flash your device with a userdebug (for root access) build containing some preloaded apps (https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Market_customizations_guide#Building_Prebundled_web_apps)
2. The preloaded apps should now be located at /system/b2g/webapps/{UUID}/
3. Clean your build files and gaia profile. Start a new build.
4. Update your device using OTA or FOTA with the new preloaded apps. Their UUIDs should have changed.
5. Booting into the new version, check Settings->App permissions. Your preloaded apps now appear twice.


This seems to be caused by the code picking up apps in system and putting them in your profile in /data, NOT checking for duplicate apps. It just assumes that two apps are different because they are at different paths (different UUIDs).
(See http://dxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#627 for a starter)

The quickest fix here might be to not use UUIDs in the path of preloaded apps. The proper long term fix might be to check for duplicates in /data/local/webapps and clear the old ones out.
[Blocking Requested - why for this release]:
Less space available on /data for each OTA/FOTA update when shipping preloaded apps. At some point it will run out and the user cannot download new apps or updates. Only fixable by a factory reset.

This is not blocking our v1.4 release, but the first v1.4 update we send out.


ni fabrice, you seem to have written a lot of the updating code. How much work would it be to check for duplicates and delete them?

ni yurenju, I believe you have done a lot of work with preloaded apps. Is it feasible to change from UUIDs for the apps to a fixed ID like for certified apps?
blocking-b2g: --- → 1.4?
Flags: needinfo?(yurenju.mozilla)
Flags: needinfo?(fabrice)
Component: General → DOM: Apps
Product: Firefox OS → Core
The way it was designed was to lock the ID at the preloading stage, so I don't really understand why at 4. you will get new UUIDs. How do you build your 

Clearing "old" ones is problematic as you want to keep the user data. In general we like to avoid deleting user data without notification.
Flags: needinfo?(fabrice)
Andreas,

Discussed with :schien, we can use checkedInstall() in Webapps.jsm::installPreinstalledApp() to check if the app is installed.

[1] http://dxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#435
Flags: needinfo?(yurenju.mozilla)
(In reply to Yuren [:yurenju] (PTO during 8/28~8/31) from comment #3)
> Andreas,
> 
> Discussed with :schien, we can use checkedInstall() in
> Webapps.jsm::installPreinstalledApp() to check if the app is installed.
> 
> [1]
> http://dxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#435

Read my previous comment. I don't think the issue is in Webapps.jsm, but in the way the customization is done. Why are they ending up with 2 different uuids?
we generated uuid for preload packaged app directory on bug 929602 and the uuid is random for each build[1].

I think we have two options to solve this problem:

1. generate a fixed uuid for preload packaged app
2. use checkedInstall() in Webapps.jsm to check if the app exists in device

this problem can't be fixed by (1) since 1.4 has been shipped, we probably need to fix it both on gecko and gaia::build.

[1] https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-manifests.js#L46
Well, the uuid used for 1.4 is known, so it's possible to fix the preinstalled uuids manually to match this one no? I'm really not keen on doing gecko side changes for that.
We have not shipped yet so if you have a quick fix to getting the same ID for preloaded apps on every build, I'd be happy to take it. It needs to be done today, however.
Fabrice, is that possible to get a same uuid from a fixed seed from nsIUUIDGenerator?
Flags: needinfo?(fabrice)
(In reply to Yuren [:yurenju] (PTO during 8/28~8/31) from comment #8)
> Fabrice, is that possible to get a same uuid from a fixed seed from
> nsIUUIDGenerator?

I don't think so, but what about using the md5 sum of the manifest url instead of the random uuid?
Flags: needinfo?(fabrice)
another way is generating a uuid.json in build_stage on build time and vendor can copy this file to GAIA_DISTRIBUTION_DIR/uuid.json then we will use the mapping file to generate same uuid for same app.
(In reply to Yuren [:yurenju] (PTO during 8/28~8/31) from comment #10)
> another way is generating a uuid.json in build_stage on build time and
> vendor can copy this file to GAIA_DISTRIBUTION_DIR/uuid.json then we will
> use the mapping file to generate same uuid for same app.

A mapping file is way better than change the way uuid is generated.

BTW can we store the mapping in metadata.json of each preloaded app?
that we will modify metadata.json in GAIA_DISTRIBUTION_DIR, but it's easier to use.
I like the mapping, it allows us to create it by hand after the first version has shipped - then we can avoid the panic of landing something today.

If we add an ID to metadata.json, then we'll have to fix it by hand whenever we re-run preload.py, right? I'd like to avoid that.
yes, I'm making a pull request for uuid.json.
we should also fix it on 2.0 and master.
Attachment #8479690 - Flags: review?(gduan)
Andreas,

this commit will generate an uuid.json in build_stage, if you want to preload app with same uuids, you need to copy it into $GAIA_DISTRIBUTION_DIR (or you can pull system/b2g/webapps from device via adb if you didn't keep this file). Does that solve your issue?
Flags: needinfo?(pehrsons)
for pulling /systemb2g/webapps, I mean you need to rebuild uuid.json manually.
Comment on attachment 8479690 [details] [review]
github PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/23353

From my understanding,
this patch will ..
1. We won't create duplicate uuid app by distribution/uuid.json if it exists
2. if partner want to refer previous uuid.json, they can manually copy it from build_stage if they have built before.

Hi Andreas,
is that what you expect?
It sounds good, will test in a few hours. Thanks.

Please make sure the docs are appropriately updated to be clear on how this file should be used, after landing.
Flags: needinfo?(pehrsons)
we will update MDN article.
this issue is not 1.4+ for now, so let's wait triage and merge if it becomes 1.4+.
Comment on attachment 8479690 [details] [review]
github PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/23353

I have tested this while building and rebuilding gaia with preloaded apps. Works exactly as intended! Thanks!

Nit: A pretty-printed json file would be nice for readability. Especially since it will have to be passed around by hand, checked into repos, etc.
Attachment #8479690 - Flags: feedback+
Yuren,

Can you also do the approval request for 2.0? (to :bajaj)

thanks
blocking-b2g: 1.4? → 1.4+
Assignee: nobody → yurenju.mozilla
Status: NEW → ASSIGNED
[Blocking Requested - why for this release]:

we will get duplicate preload apps if we have first OTA for 2.0
blocking-b2g: 1.4+ → 2.0?
this commit is cherry picked from v2.0, so it can be landed if we got r+ for commit of v2.0.
Comment on attachment 8482158 [details] [review]
github PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/23554

Looks fine. I'll spend some time to complete the unit test for build.
r=gduan.
Attachment #8482158 - Flags: review?(gduan) → review+
Switch to v1.4 -- Yuren, please set approval-gaia-v2.0 on the patch instead.
blocking-b2g: 2.0? → 1.4+
Switch *back* to 1.4+ I meant.
Component: DOM: Apps → Gaia::Build
Product: Core → Firefox OS
Comment on attachment 8482158 [details] [review]
github PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/23554

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): bug 929602
[User impact] if declined: preload app appears twice when apply first OTA
[Testing completed]: we have a integration test case for it
[Risk to taking this patch] (and alternatives if risky): No
[String changes made]: No
Attachment #8482158 - Flags: approval-gaia-v2.0?(bbajaj)
Attachment #8482158 - Flags: approval-gaia-v2.0?(bbajaj) → approval-gaia-v2.0+
Comment on attachment 8482158 [details] [review]
github PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/23554

switching back the a+ to a?, until this lands on master.
Attachment #8482158 - Flags: approval-gaia-v2.0+ → approval-gaia-v2.0?
Attachment #8482158 - Flags: approval-gaia-v2.0? → approval-gaia-v2.0+
Flags: needinfo?(bbajaj)
the test case which is error on Gij is disabled on bug 1043870 and we got green Gip for linux64-opt and mac build, so it seems a try server Infrastructure issue.
merged.

https://github.com/mozilla-b2g/gaia/commit/f55572653b88acf28c355c77196bf5191a7ee0b7
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
AFAICT, this still needs to land on v2.1 (it landed on master after this week's uplift). Please request Gaia v2.1 approval when you get a chance.
Flags: needinfo?(yurenju.mozilla)
Target Milestone: --- → 2.1 S4 (12sep)
Flags: needinfo?(yurenju.mozilla)
Gb is red again, we should uplift bug 1061563.
Flags: needinfo?(yurenju.mozilla)
See Also: → 1061563
(In reply to Yuren [:yurenju] from comment #47)
> Gb is red again, we should uplift bug 1061563.

That's done now, right? What needs to be done here still?
Flags: needinfo?(yurenju)
Comment on attachment 8486158 [details] [review]
github PR for v2.1: https://github.com/mozilla-b2g/gaia/pull/23830

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): bug 929602
[User impact] if declined: preload app appears twice when apply first OTA 
[Testing completed]: we have a integration test case for it
[Risk to taking this patch] (and alternatives if risky): No
[String changes made]: No
Attachment #8486158 - Flags: approval-gaia-v2.1?(bbajaj)
Attachment #8486158 - Flags: approval-gaia-v2.1?(bbajaj) → approval-gaia-v2.1+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: