1017069 - home1 to home2 Data Migration

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::Homescreen, defect)

Comment 1

[Johan Lorenzo [:jlorenzo]]

What is the point of this bug? Is it to track home2 default enabling? If so, I think that Bug 1009111 is enough.

Comment 2

[Amir Nissim (:amirn)]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> What is the point of this bug? Is it to track home2 default enabling? If so,
> I think that Bug 1009111 is enough.

This bug is about creating a data migration script that copies data from the old homescreen (like Collections) to the new one

Comment 3

[Amir Nissim (:amirn)]

Cristian, can you take this?

Comment 4

[Kevin Grandon :kgrandon]

Do we have any data on the number of collections that users are making? I am just wondering if this is really a critical feature to have for 2.0. If it is, we should probably follow what bookmarks is doing to migrate data. We will need to test these upgrade steps thoroughly, I think we might need to actually launch the old homescreen, migrate the data, then launch the new homescreen.

Comment 5

[Jason Smith [:jsmith]]

(In reply to Kevin Grandon :kgrandon from comment #4)
> Do we have any data on the number of collections that users are making? I am
> just wondering if this is really a critical feature to have for 2.0. If it
> is, we should probably follow what bookmarks is doing to migrate data. We
> will need to test these upgrade steps thoroughly, I think we might need to
> actually launch the old homescreen, migrate the data, then launch the new
> homescreen.

Ran - Do you know?

Comment 6

[Cristian Rodriguez (:crdlc)]

Well, I have to understand properly what we want to do here. We have two scenarios:

1) Someone buys a new device 2.0:

   No migration, bye bye

2) Someone updates a device from 1.x to 2.0:

   * I assume that the FTU is launched after upgrading
   * Behind scene we have to launch the old home (or open an iFrame with the same origin)
   * Iterate the old indexedDB, filtering by type=collection and migrate data to the new collection datastore
   * Uninstall the old home at the end

Regarding to point 2, I don't know if we can do it or if it is feasible. Any suggestion? More ideas how to face it?

Thanks in advance

-----

I don't know if we can do it.... Release the new vertical-homescreen in v2.0 under the same origin than old one in order to migrate data when it will be launched the first time

Comment 7

[Cristian Rodriguez (:crdlc)]

BTW we have exactly the same work for bookmarks

Comment 8

[Cristian Rodriguez (:crdlc)]

I think that we could do the work in the same task.. iterating indexedDB by "collection" type and calling CollectionDatabase.add().... and indexed by "bookmarks" type and calling to BookmkarkDatabase.add()

Comment 9

[Kevin Grandon :kgrandon]

(In reply to Cristian Rodriguez (:crdlc) from comment #6)
> 2) Someone updates a device from 1.x to 2.0:
> 
>    * I assume that the FTU is launched after upgrading
>    * Behind scene we have to launch the old home (or open an iFrame with the
> same origin)
>    * Iterate the old indexedDB, filtering by type=collection and migrate
> data to the new collection datastore
>    * Uninstall the old home at the end
> 
> Regarding to point 2, I don't know if we can do it or if it is feasible. Any
> suggestion? More ideas how to face it?

I think this is a pretty good approach to take, and something I would like to investigate. I know we were investigating doing this before, and I think Kyle might know something about it, so ni? on him. It doesn't seem like it would be too bad to:

1 - On upgrade send an IAC message to the old homescreen.
2 - Migrate bookmarks, instantiate collections.
3 - Respond to the IAC message.
4 - Launch the new homescreen.

You might even be able to do this from the homescreen (and not FTU).


> I don't know if we can do it.... Release the new vertical-homescreen in v2.0
> under the same origin than old one in order to migrate data when it will be
> launched the first time

Worst case scenario I suppose we can do this. I would like to investigate to see if we can use a new origin though. If it turns out to be way too painful, let's plan on this.

Comment 10

[Jason Smith [:jsmith]]

(In reply to Kevin Grandon :kgrandon from comment #4)
> Do we have any data on the number of collections that users are making? I am
> just wondering if this is really a critical feature to have for 2.0. If it
> is, we should probably follow what bookmarks is doing to migrate data. We
> will need to test these upgrade steps thoroughly, I think we might need to
> actually launch the old homescreen, migrate the data, then launch the new
> homescreen.

Thinking about this more, I don't think we can cause intentional data loss on an update with a front-facing feature. That's going to cause a nasty spike of support complaints even if there's a small group of people using it. Collections also was considered a flagship feature in the release it shipped in, so I am a bit nervous to take any data loss risks on an update.

Comment 11

[Kevin Grandon :kgrandon]

Yeah, that's why it would be nice to have data on this issue. Without real data we're just making guesses. In the meantime I'm fine with planning on doing a data migration. The hardest thing to do here is to probably write some solid tests to make sure this works and we don't regress it.

Comment 12

[Cristian Rodriguez (:crdlc)]

Well, I think that we are on the same page here, my plan could be something like this:

1) On upgrade send an IAC message to the old homescreen, e.g. "migrate my old friend".
2) Perform the migration of bookmarks and collections (old-homescreen should have access to new datastores).
3) - Respond to the IAC message. e.g "ready"
4) - Launch the new homescreen because both datastores are populated

My question, when and from where can I send that IAC message? from system?

Comment 13

[Kevin Grandon :kgrandon]

Let's see if Kyle responds about where the communication should be, I think he may have looked into it before?

Collections already blocks homescreen, so let's just block the collections metabug here.

Comment 14

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Looked at, yes, implemented, no. 

FTU bringup on upgrade got rolled into bug 939174 in January and is /still/ trying to land. If you need it before next week, I recommend someone go in, make it a new bug, and decouple it from bug 939174, unless Guillaume will finish it before next week.

However, I never really figured out how this was going to work because we could never decide when the old browser was going to be taken out, so the work never happened. The only solution I had is exactly what Cristian laid out in Comment #6; write a stub app that comes up, migrates, uninstalls itself.

Comment 15

[Sam Foster [:sfoster] (he/him)]

I'm in the process of rebasing/manually merging the work from bug 939174 as I also need it for bug 990036 (which should be able to land travis & reviewer-availability willing.), The tutorial config approach in 939174 is now supplanted by a different-but-similar idea in bug 1017314. So please cc me on any new bug created for the FTU/upgrade scenario.

Comment 16

[Cristian Rodriguez (:crdlc)]

Well, I gonna work just on a mechanism to migrate data from indexedDB to datastores in the old homescreen today. We will call to this mechanism via IAC. I gonna split into two different bugs this one. The mechanism in old homescreen to migrate data and also another bug to start the migration from FTE or from where we'll decide

Comment 17

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Cristian Rodriguez (:crdlc) from comment #12)
> Well, I think that we are on the same page here, my plan could be something
> like this:
> 
> 1) On upgrade send an IAC message to the old homescreen, e.g. "migrate my
> old friend".

Why do you need IAC ? Can you simply open the homescreen into a <iframe mozbrowser mozapp> in the FTU app, and let the homescreen perform the work ?

> 2) Perform the migration of bookmarks and collections (old-homescreen should
> have access to new datastores).
> 3) - Respond to the IAC message. e.g "ready"

Then the homescreen just call window.close() when it is ready.

> 4) - Launch the new homescreen because both datastores are populated
> 
> My question, when and from where can I send that IAC message? from system?

So basically one step of the FTU would be to open an <iframe mozbrowser mozapp> on the old homescreen, this one can display a 'Migrating Data' something as a UI. Then when it is ready the 'Next' button of the FTU is activated and the user can go forward.

Does it makes sense ?

Comment 18

[Cristian Rodriguez (:crdlc)]

It has been implemented with IAC because there is not UI to show (it will be performed behind scene) and that solution although nice could be more complex thought.

Comment 19

[Cristian Rodriguez (:crdlc)]

All dependencies already landed on master so closing...

Comment 20

[Johan Lorenzo [:jlorenzo]]

Verified from 1.4 to 2.0. Privileged and hosted apps are migrated, not bookmarks. Icons of hosted apps are set to the default one until an Internet connection is established.

See bug 1026737 and bug 1026733 for details.

Comment 21

[Kevin Grandon :kgrandon]

Dependent bugs have been uplifted. Marking as fixed in 2.0, 2.1 to simplify tracking.

Comment 22

[Johan Lorenzo [:jlorenzo]]

https://moztrap.mozilla.org/manage/case/13218/
https://moztrap.mozilla.org/manage/case/13222/
https://moztrap.mozilla.org/manage/case/13223/
https://moztrap.mozilla.org/manage/case/13225/
https://moztrap.mozilla.org/manage/case/13226/
https://moztrap.mozilla.org/manage/case/13227/
https://moztrap.mozilla.org/manage/case/13228/
https://moztrap.mozilla.org/manage/case/13229/
https://moztrap.mozilla.org/manage/case/13230/