1022654 - FTU not properly shown after update

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::First Time Experience, defect)

Description

[:gerard-majax]

Updating device, a file directory listing of communications.gaiamobile.org is displayed instead of the FTU app.

The same happens when launching FTU by hand from Settings > Developper.

I suspect some migration of the settings value for ftu.manifestURL key is not properly handled.

Comment 1

[:gerard-majax]

Hacking some debug in FTU launcher in Settings app:

E/GeckoConsole( 1938): Content JS DEBUG at app://settings.gaiamobile.org/js/developer.js:1 in Developer.launchFTU: key=ftu.manifestURL
E/GeckoConsole( 1938): Content JS DEBUG at app://settings.gaiamobile.org/js/developer.js:1 in Developer.launchFTU/n.onsuccess: value=app://communications.gaiamobile.org/manifest.webapp

Comment 2

[Jason Smith [:jsmith]]

Saw this as well - thanks for filing. Obvious blocker, so marking 2.0+.

Comment 3

[Sam Foster [:sfoster] (he/him)]

Is this fallout from bug 1010782? Or an upgrade issue (ni? Kyle on that) I've been launching the FTU from the developer settings screen all day, so did this regress today or does it only show up with some particular build config or something?

Comment 4

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Oh. That's fun. We moved the FTU out of the communications app, but the FTU app manifest is a setting, so in an upgrade situation that setting isn't getting changed. So we need the system app to update the value of the ftu manifest location on first launch of v2.0 I guess, somehow /before/ the FTU launches? Either that or hardcode it into the FTU launcher and make some new setting for saying whether or not the FTU has launched.

Yet another case where we really need those damn profile upgrade tests I've been promising for 6 months. Soon, damnit.

Comment 5

[:gerard-majax]

(In reply to Sam Foster [:sfoster] from comment #3)
> Is this fallout from bug 1010782? Or an upgrade issue (ni? Kyle on that)
> I've been launching the FTU from the developer settings screen all day, so
> did this regress today or does it only show up with some particular build
> config or something?

Install a version from before your change (bug 1010782), then update to current version.

Comment 6

[Sam Foster [:sfoster] (he/him)]

The simplest fix here seems to be to hard-code the ftu app manifest url. Is there historical (and present) reason for storing this as a setting?

Comment 7

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

The setting actually came in as part of bug 803101, alive was just responsible for moving it out of the window manager. I'm ni?'ing arcturus on this for further info also.

Comment 8

[Francisco Jordano [:arcturus] [:francisco]]

Hi folks,

It used to be a condition when we created the FTU some years ago :_) ... we were not sure of it's ussage and putting the manifest as a setting was aggreed with Vivien for two reasons:

- Allow 3rd party to setup their own experience (thing that so far I don't know how is doing this)
- In case of no setting, skipping the FTU

So, I guess we could fall back to hardcoded value. If looking for FTU manifest setting and app not found we could launch a hardcoded value, but I guess that will conflict with the NOFTU param (that "used to" no setup ftu manifest)

Comment 9

[:gerard-majax]

So the FTU setting landed in https://github.com/mozilla-b2g/gaia/commit/f84370c649a41e382aaec6864e81c87250b75eb9 (bug 1010782).

My checking on my personnal profile shows that upgrading is done:
> I/Gecko   ( 1837): -*- SettingsDB: value.settingName: cursor.update: {"settingName":"ftu.manifestURL","defaultValue":"app://ftu.gaiamobile.org/manifest.webapp","userValue":"app://communications.gaiamobile.org/manifest.webapp"}

But there's a user-defined value that makes the new FTU not active in fact.

Comment 10

[:gerard-majax]

STR:
 0. checkout gaia at 84d12a6ed7d6171967455ba3f418d853b534a31b, make sure you have a matching gecko (SETTINGSDB_VERSION=3)
 1. make MOZILLA_OFFICIAL=1 PRODUCTION=1 REMOTE_DEBUGGER=1 NOFTU=0 GAIA_DEV_PIXELS_PER_PX=1.5 GAIA_OPTIMIZE=1 reset-gaia
 2. adb shell cat /system/b2g/defaults/settings.json | json_pp | grep ftu.manifestURL; the value should be "app://communications.gaiamobile.org/manifest.webapp"
 3. have a Flame B2G tree with current master checked out, SETTINGSDB_VERSION=4 and DEBUG enabled in SettingsDB.jsm
 4. produce fota with gecko/gaia, adb logcat | egrep SettingsDB

Comment 11

[:gerard-majax]

Following STRs in comment 10, manifestURL is properly updated:
> 06-17 14:38:10.721  5795  5795 I Gecko   : -*- SettingsDB: Set new:ftu.manifestURL, app://communications.gaiamobile.org/manifest.webapp
> 01-06 23:11:22.879   293   293 I Gecko   : -*- SettingsDB: value.settingName: cursor.update: {"settingName":"ftu.manifestURL","defaultValue":"app://ftu.gaiamobile.org/manifest.webapp"}

It means we are probably not in a too bad state. But my profile exposes an issue that needs to be fixed whatever.

Comment 12

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

> So, I guess we could fall back to hardcoded value. If looking for FTU
> manifest setting and app not found we could launch a hardcoded value, but I
> guess that will conflict with the NOFTU param (that "used to" no setup ftu
> manifest)

I was thinking we could hardcode the manifest URL, and then have another setting that's just "FTU already run". We'll need that anyways, and we can convert NOFTU to use that.

Comment 13

[:gerard-majax]

(In reply to Kyle Machulis [:kmachulis] [:qdot] from comment #12)
> > So, I guess we could fall back to hardcoded value. If looking for FTU
> > manifest setting and app not found we could launch a hardcoded value, but I
> > guess that will conflict with the NOFTU param (that "used to" no setup ftu
> > manifest)
> 
> I was thinking we could hardcode the manifest URL, and then have another
> setting that's just "FTU already run". We'll need that anyways, and we can
> convert NOFTU to use that.

Gregor was okay to implement some mechanism that would allow us to force migration/discard of a user value for a setting. This would fix the migration of my profile, and this could be useful on other usecases.

Comment 14

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Sam Foster [:sfoster] from comment #6)
> The simplest fix here seems to be to hard-code the ftu app manifest url. Is
> there historical (and present) reason for storing this as a setting?

Replaceable FTU is valuable in case someone wants to implement its own app for that. Hardcoding the value just seems a workaround to avoid this migration issue.

Comment 15

[:gerard-majax]

> I/Gecko   ( 4631): -*- SettingsDB: value: {"settingName":"ftu.manifestURL","defaultValue":"app://communications.gaiamobile.org/manifest.webapp","userValue":"app://communications.gaiamobile.org/manifest.webapp"}
> I/Gecko   ( 4631): -*- SettingsDB: branch value.settingName
> I/Gecko   ( 4631): -*- SettingsDB: cursor.update(value): {"settingName":"ftu.manifestURL","defaultValue":"app://ftu.gaiamobile.org/manifest.webapp","userValue":"app://communications.gaiamobile.org/manifest.webapp"}

Comment 16

[:gerard-majax]

Upgrading from an old settingsdb backup from the 1.1 period:

> I/Gecko   ( 5073): -*- SettingsDB: value: {"settingName":"ftu.manifestURL","defaultValue":"app://communications.gaiamobile.org/manifest.webapp","userValue":"app://communications.gaiamobile.org/manifest.webapp"}
> I/Gecko   ( 5073): -*- SettingsDB: branch value.settingName
> I/Gecko   ( 5073): -*- SettingsDB: cursor.update(value): {"settingName":"ftu.manifestURL","defaultValue":"app://ftu.gaiamobile.org/manifest.webapp","userValue":"app://communications.gaiamobile.org/manifest.webapp"}

Vivien suggested that the issue may come from my profile being living from before the 1.0.1 era, and thus exposing issue that cannot be triggered by any user starting from v1.0.1.

Comment 17

[:gerard-majax]

After careful checking,

> I/Gecko   ( 5714): -*- SettingsDB: value: {"settingName":"ftu.manifestURL","defaultValue":"app://communications.gaiamobile.org/manifest.webapp"}
> I/Gecko   ( 5714): -*- SettingsDB: branch value.settingName
> I/Gecko   ( 5714): -*- SettingsDB: cursor.update(value): {"settingName":"ftu.manifestURL","defaultValue":"app://ftu.gaiamobile.org/manifest.webapp"}

This is when updating with current master using a v1.0.1 settings database. I took v1.0.1 from pvtbuilds back in early may 2013, https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2g_ril/inari-eng-mozilla-b2g18_v1_0_1-20130507070205-ril01.00.01.019.098.zip.

That's the oldest I could find. Hence, as Vivien suggested, this may be a leftover from my profile dating from the early days of B2G. And I'm unsure whether we should still consider this to block v2.0.

Comment 18

[Jason Smith [:jsmith]]

That doesn't align with what I saw. I reproduced this from migrating from when the FTE was in the comms app to when the FTE was in a separate app (2.0 --> 2.0).

Although the important update path is 1.4 --> 2.0, so I'll mark qawanted to check that.

Comment 19

[:gerard-majax]

(In reply to Jason Smith [:jsmith] from comment #18)
> That doesn't align with what I saw. I reproduced this from migrating from
> when the FTE was in the comms app to when the FTE was in a separate app (2.0
> --> 2.0).
> 
> Although the important update path is 1.4 --> 2.0, so I'll mark qawanted to
> check that.

I respectfully disagree. I could only reproduce regenerating a sqlite settings database using a very old b2g gecko, from http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2012/12/2012-12-31-23-02-01-mozilla-b2g18/b2g-18.0.multi.linux-x86_64.tar.bz2.

Only in this case we trigger the upgrade path in Gecko that inserts the userValue that is the cause of the ftu not properly launched.

Comment 20

[Jason Smith [:jsmith]]

(In reply to Alexandre LISSY :gerard-majax from comment #19)
> (In reply to Jason Smith [:jsmith] from comment #18)
> > That doesn't align with what I saw. I reproduced this from migrating from
> > when the FTE was in the comms app to when the FTE was in a separate app (2.0
> > --> 2.0).
> > 
> > Although the important update path is 1.4 --> 2.0, so I'll mark qawanted to
> > check that.
> 
> I respectfully disagree. I could only reproduce regenerating a sqlite
> settings database using a very old b2g gecko, from
> http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2012/12/2012-12-31-23-02-
> 01-mozilla-b2g18/b2g-18.0.multi.linux-x86_64.tar.bz2.
> 
> Only in this case we trigger the upgrade path in Gecko that inserts the
> userValue that is the cause of the ftu not properly launched.

I don't see how you can disagree with something that someone else saw? I know what I saw during my daily testing, which showed this happening in 2.0 --> 2.0 migration between old & new FTE. But the right test here to verify this is check using a 1.4 --> 2.0 migration on the latest, since that's the most likely migration path. I want to be sure that this won't happen in that case because if does, then this is a bad bug.

Comment 21

[:gerard-majax]

The bug was originally reported by myself, because I got this issue. On my reproduction case, FTU is migrated properly but the profile being old, it dates back to the days of SETTINGSDB_VERSION = 1. At that time, the settings database got upgraded to SETTINGSDB_VERSION = 2 and this upgrade took the previously stored setting value and saved it as userValue.

This means any profile produced and migrated since has had a "custom" FTU application that was pointing to the communications apps.

Now, introducing the new app, then the defaultValue of the setting is properly migrated to the app://ftu value. But the upgrade code keeps userValue, and when the setting is requested it makes use of the userValue. Which is exactly what userValue/defaultValue is about: allowing user to customize values.

On those old profiles, however, all pre-existing settings were forced to have a userValue. So any new value is properly upgraded for the defaultValue field, but the userValue will remain.

I don't know what you exactly saw trying to reproduce on 2.0, whether this was the same root issue as mine that could have been triggered in another way, or whether this was another issue with the same symptom.

Comment 22

[:gerard-majax]

(In reply to Jason Smith [:jsmith] from comment #2)
> Saw this as well - thanks for filing. Obvious blocker, so marking 2.0+.

Jason, considering the date of your comment 2, I suspect the build you reproduced this bug with migration from 2.0 to 2.0 did not had the patch for bug 1021351. Fixing bug 1021351 will also trigger the proper migration of the ftu.manifestURL setting value. That may explain why you saw the same issue as me at this time, but if you confirm then it means the root cause for our two symptoms were different. And in this case, yours is already fixed and mine should probably not block 2.0.

Comment 23

[Jason Smith [:jsmith]]

Ah that makes sense. Clearing the blocking flag then since a 1.01 --> 2.0 update won't happen. Do we even need to keep this bug open?

Comment 24

[Jason Smith [:jsmith]]

Leaving qawanted to still do the 1.4 --> 2.0 check.

Comment 25

[Sam Foster [:sfoster] (he/him)]

I'm not clear where we ended up here. Do we need to open a new bug to implement comment #12 ? Sounds like we need QA to confirm if this is a 2.0 blocker?

Comment 26

[Francisco Jordano [:arcturus] [:francisco]]

(In reply to Sam Foster [:sfoster] from comment #25)
> I'm not clear where we ended up here. Do we need to open a new bug to
> implement comment #12 ? Sounds like we need QA to confirm if this is a 2.0
> blocker?

Agree with Jason here, let's check 1.4 -> 2.0 migration.

Jason, do you think it makes sense to check as well 1.3 -> 2.0 migration? Open C phones sell in ebay come with 1.3.

Comment 27

[Jason Smith [:jsmith]]

(In reply to Francisco Jordano [:arcturus] [:francisco] from comment #26)
> (In reply to Sam Foster [:sfoster] from comment #25)
> > I'm not clear where we ended up here. Do we need to open a new bug to
> > implement comment #12 ? Sounds like we need QA to confirm if this is a 2.0
> > blocker?
> 
> Agree with Jason here, let's check 1.4 -> 2.0 migration.
> 
> Jason, do you think it makes sense to check as well 1.3 -> 2.0 migration?
> Open C phones sell in ebay come with 1.3.

Sure.

Comment 28

[:gerard-majax]

So ? Do you have any news on QA, Jason?

Comment 29

[Jason Smith [:jsmith]]

We're slammed with QA Wanted requests right now, so no update yet.

Comment 30

[Gregor Wagner [:gwagner]]

QA ping

Comment 31

[Pi Wei Cheng [:piwei] (inactive)]

Attachment: first screen after OTA from 1.4 to 2.0

(In reply to Jason Smith [:jsmith] from comment #18)
> Although the important update path is 1.4 --> 2.0, so I'll mark qawanted to
> check that.

I OTA updated from this build:
Device: Flame
BuildID: 20140630000201
Gaia: aa896d5db1b4929f3bf31a0f4bb7de50530222a8
Gecko: 8cba60bc12ef
Version: 30.0 (1.4)
Firmware version: v122

To this build:
Device: Flame
Build ID: 20140703000248
Gaia: 7ad00b0bd84d5d97e0801e3b3ceaac33fcd90e05
Gecko: e87f7b398fce
Version: 32.0a2 (2.0)
Firmware Version: v122

The first screen after the OTA update is the attached screenshot. The rest of the FTU looks as expected.
I also launched First Time Use from Developer settings and everything looks correct.

Comment 32

[Joshua Mitchell (Inactive)]

1.4 --> 2.0 seems to work correctly. Not closing this as fixed yet as I'm not sure there are other update paths required before this is considered 'fixed'.

Comment 33

[Jason Smith [:jsmith]]

Seems like this is wfm from the testing I see above.