1110119 - When updating via OTA/FOTA, already installed third party apps are installed again

Status: RESOLVED FIXED

(Core Graveyard :: DOM: Apps, defect)

Description

[Antonio Manuel Amaya Calvo (:amac)]

STR: 

  * Install FirefoxOS 1.3 on a device. The build must include some preinstalled third party apps that doesn't have a custom origin. It doesn't really matter if the app is hosted or packaged.

  * Update to 2.x via OTA/FOTA. The new build must include preinstalled some of the same third party apps.

EXPECTED
  * The apps already preinstalled by 1.3 are kept, and no changes are done to them.

ACTUAL
  * The preinstalled apps shared between the 1.3 and 2.0 builds are duplicated. The apps don't show on the homescreen, but anytime there's an update the update will be offered twice, and on the App Manager will show the app twice also.

Uninstalling the app will remove the icon from the homescreen, but it'll reappear after the phone is rebooted (since only one of the copies was actually uninstalled).

The problem is that until 1.3 (or 1.4, I have the details slightly fuzzy by now) the build system kept the directory names that the preinstalled third party apps had on the local filesystem. So if I installed a third party app that was on the build tree on a directory called 'mythirdpartyapp' it would be copied into the device as app://mythirdpartyapp. From 2.0 onwards, all the non privileged third party apps, and privileged ones that don't specify an origin are given a uuid identifier and copied as app://{uuid-here}. 

Since the app id will then change between 1.3 and 2.0, the update process thinks it's a new application, and so installs it. What we should do at first boot time is check by manifestURL instead (so if an app that has the same manifestURL than the currently being processed one exists, we don't install it again).

Comment 1

[Beatriz Rodríguez [:brg]]

[Blocking Requested - why for this release]: This is a very urgent issue blocking us on the OTA process for 2.0 release.

Comment 2

[Antonio Manuel Amaya Calvo (:amac)]

Attachment: V1. Keep track of preinstalled apps by manifestURL

This is quite straightforward. I have the patch ready for 2.0 also (actually I wrote it for 2.0 and then downlifted it :)).

Comment 3

[Antonio Manuel Amaya Calvo (:amac)]

Attachment: B2G 32 version

Comment 4

[KTucker [:KTucker][Inactive 3/4/2016]]

This appears to be a very similar to bug 979175. Also, this was fixed in bug 1073335.

Comment 5

[Antonio Manuel Amaya Calvo (:amac)]

Yep, it's the same. And nope, it wasn't fixed in bug 1073335 (are you sure you got the right bug number? that does seem to be related with formatting the internal storage, not initial app installation). In fact it doesn't seem to be fixed...

Comment 7

[[:fabrice] Fabrice Desré]

Comment on attachment 8535024 [details] [diff] [review]
V1. Keep track of preinstalled apps by manifestURL

Review of attachment 8535024 [details] [diff] [review]:
-----------------------------------------------------------------

Right. that would really be nice to have tests around this...

Comment 8

[Antonio Manuel Amaya Calvo (:amac)]

Actually I opened a bug to implement the installation at boot tests some time ago. I haven't had any time for it though. Yet. I hope :)

Comment 9

[Antonio Manuel Amaya Calvo (:amac)]

Comment on attachment 8535024 [details] [diff] [review]
V1. Keep track of preinstalled apps by manifestURL

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 
User impact if declined: during the OTA some apps will be duplicated. This is also a certification blocker. 
Testing completed: manually by the developer and the OEM. 
Risk to taking this patch (and alternatives if risky): pretty low
String or UUID changes made by this patch:none

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Try link? And should this land with tests?

Comment 11

[Antonio Manuel Amaya Calvo (:amac)]

There you go: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=882abced600a 

Tests should/will be done (some time...) in bug 1016889. Currently there are no tests for the initial load of apps at all, nor it would be easy nor quick to write them now. So... I would love to have tests for this (and Fabrice too, I'm sure) but I don't think it's feasible to have them in this bug (specially not in the time frame we have for the 1.3->2.0 OTAs which this blocks).

Comment 12

[Antonio Manuel Amaya Calvo (:amac)]

Try run is mostly green, re-requesting checkin

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d4640dc77563

Comment 14

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d4640dc77563

Comment 15

[bhavana bajaj [:bajaj]]

Comment on attachment 8535024 [details] [diff] [review]
V1. Keep track of preinstalled apps by manifestURL

:amac, can you please help with manual testing once this lands on branches. The update code really scares me and the lack of tests here given how complex they are demands more manual testing here. I'll also in parallel try to get Mozilla QA to verify if they can

Comment 16

[Antonio Manuel Amaya Calvo (:amac)]

Sure thing, we can take care of testing this. Actually, we kinda already did this for the 2.0 branch at least.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/39eaa987093b
https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/11f45a1d3bf8

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g32_v2_0m/rev/11f45a1d3bf8

Comment 19

[Shally Li（leave from Mozilla）]

Hi All,

    This bug has been successfully verified on Flame v2.0&2.1&2.2 after OTA (Base image:v18D).
    See attachment: verified_v2.1.MP4.
    Reproduce rate: 0/2

Flame 2.2 user build
Gaia-Rev        bdedbaf9f18a43c091ede770407d68d38582fe29
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/6d5a1e68f248
Build-ID        20141228160203
Version         37.0a1

After OTA:
Gaia-Rev        322ef5116a5827a30c9a3cd9b842449a9c66a5b3
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/2c0ef7315830
Build-ID        20141229160214
Version         37.0a1
----------------------------------------------------------------------------

Flame 2.1 user build:
Gaia-Rev        17c7ad2e4919a994f0844239b483116090412dee
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/39dfb662c82a
Build-ID        20141222161203
Version         34.0

After OTA:
Gaia-Rev        73be51f998031f06db0cd660c0e388fa621c9f4c
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/ea426e47bfc4
Build-ID        20141229161203
Version         34.0
----------------------------------------------------------------------------

Flame 2.0 user build:
Gaia-Rev        ce83ea7b8e3fa2d1c3fd771fc22b654c18b3c381
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/4dd6d9b58f42
Build-ID        20141222160204
Version         32.0

After OTA:
Gaia-Rev        01c32dcdc08b3c8fc8b3082870439fa2c1089f4f
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/b4ea79d5c1b2
Build-ID        20141229160215
Version         32.0

Comment 20

[Shally Li（leave from Mozilla）]

Attachment: verified_2.1.MP4

Comment 22

[Josh Cheng [:josh]]

Hi Sahlly,
Please verify again as you need to check the version of 3rd party app still the same.
Bug 1125034 is complaining about pre-installed 3rd party app version been downgraded.
Thanks.

Comment 23

[Shally Li（leave from Mozilla）]

Hi Josh,

    This bug has been successfully verified on Flame v2.0&2.1&2.2 after OTA (Base image:v18D),and the pre-installed 3rd party app version is the same.
    See attachment: new_verified_ota_v2.1.MP4.
    Reproduce rate: 0/4(Flame v2.0),0/2(v2.1),0/2(v2.2).

STR:
1.Flash user build.
2.Download some 3rd party apps from Marketplace and note their versions.
3.Install OTA.
4.After OTA,check these 3rd party apps and their versions.
**All 3rd party apps show on the homescreen and their versions are the same.(such as "LINE:v1.7.0","Candy Crush:v0.2.1","Loqui IM:v0.4.0")

---------------------------------------------------------------
Flame 2.0 user build:
Build ID               20150304160234
Gaia Revision          8eaa7ab64bc2c6fa882d6e16766f27ed89c86f34
Gaia Date              2015-03-04 16:10:52
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/cec3b489ef2c
Gecko Version          32.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150304.191831
Firmware Date          Wed Mar  4 19:18:40 EST 2015
Bootloader             L1TC000118D0

After OTA:
Build ID               20150305160227
Gaia Revision          8463d1c9142f32c8ea175048dac52e41620443ce
Gaia Date              2015-03-05 16:28:56
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/765674521c35
Gecko Version          32.0

-------------------------------------
Flame 2.1 user build:
Build ID               20150304161203
Gaia Revision          89db88a76ae8363cd7f8e7a07d35ae24058aec36
Gaia Date              2015-03-04 16:11:03
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/b64b3a28d459
Gecko Version          34.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150304.194211
Firmware Date          Wed Mar  4 19:42:22 EST 2015
Bootloader             L1TC000118D0

After OTA:
Build ID               20150305161203
Gaia Revision          ea97a87048a4c1e2a479bbea1d75e0a182b2c4c9
Gaia Date              2015-03-05 16:30:05
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/871071010b5b
Gecko Version          34.0

-------------------------------------
Flame 2.2 user build:
Build ID               20150304162536
Gaia Revision          89af288bad6751248ff84504fa898206fee127fe
Gaia Date              2015-03-04 18:00:05
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/c7737301ad14
Gecko Version          37.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150304.195555
Firmware Date          Wed Mar  4 19:56:06 EST 2015
Bootloader             L1TC000118D0

After OTA:
Build ID               20150305162505
Gaia Revision          eb86137e247224e86d17ed1a0a133b2a318dce3c
Gaia Date              2015-03-05 16:51:07
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/e9d29a3b94e5
Gecko Version          37.0
----------------------------------------------------------------
I can't do FOTA in our side,leaving "verifyme".

Comment 24

[Shally Li（leave from Mozilla）]

Attachment: new_verified_ota_v2.1.MP4

Comment 25

[Beatriz Rodríguez [:brg]]

(In reply to Shally from comment #23)
> Hi Josh,
> 
>     This bug has been successfully verified on Flame v2.0&2.1&2.2 after OTA
> (Base image:v18D),and the pre-installed 3rd party app version is the same.
>     See attachment: new_verified_ota_v2.1.MP4.
>     Reproduce rate: 0/4(Flame v2.0),0/2(v2.1),0/2(v2.2).
> 
> STR:
> 1.Flash user build.
> 2.Download some 3rd party apps from Marketplace and note their versions.
> 3.Install OTA.
> 4.After OTA,check these 3rd party apps and their versions.
> **All 3rd party apps show on the homescreen and their versions are the
> same.(such as "LINE:v1.7.0","Candy Crush:v0.2.1","Loqui IM:v0.4.0")
> 
I do not think above steps are correct to reproduce the issue reported in this bug and described in comment 0:
>The build must include some preinstalled third party apps that doesn't have a custom origin. It doesn't >really matter if the app is hosted or packaged.

The issue here is about how preinstalled apps are modified after the OTA process. Apps downloaded from Marketplace are a different scenario.
Shally, if you need more information about the scenario, lets ask the reporter in bug 125034.

Comment 26

[Shally Li（leave from Mozilla）]

Hi Reporter,

    Per Comment 25, could you help to confirm the "preinstalled apps" in Comment 0?

Thank you very much.

Comment 27

[Beatriz Rodríguez [:brg]]

mm I think you wanted to ask some body else. Please reporter help Mozilla to check the issue by describing the scenario for testing.

Comment 28

[Jack Liu]

(In reply to Shally from comment #26)
> Hi Reporter,
> 
>     Per Comment 25, could you help to confirm the "preinstalled apps" in
> Comment 0?
> 
> Thank you very much.

Hi,

'preinstalled' apps are but not limited to 'Firefox Hello', 'Latch', 'Pinterest'.

Actually the issue can be reproduced with any 'preinstalled' app with a newer version of it in marketplace and user manually update it before FOTA.

Please refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1125034#c10 for more details.

Thanks.

Comment 29

[Qu Yi]

Dear Wesly:

It's urgent issue for TEF FireE2.0 SW.
Please help to fix it asap.
Thanks.

Comment 30

[Wesly Huang (EPM)]

Started from comment#21 we were actually talking about 1125034, which was set to dup this one. However after discussion with EPM we think they might be related but not the same, hence here I would like to separate these 2 again to avoid further mix/confusion. (there was some in comment#23~#27 already)

This one will be set to previous status (fixed), and let 1125034 handle what it need to do. Thanks.