1077637 - [System] Browsing History not migrated after OTA from 2.0 to 2.1

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[Yeojin Chung [:YeojinC]]

Attachment: logcat_20141003_History_Topsite_Not_Migrated.txt

Description:
After OTA upgrade from 2.0 to 2.1, the history section appears empty on New Window screen. 
   
Repro Steps:
1) Update a Flame device to 2.0 BuildID: 
2) Change update channel and url on device to appropriate 2.1 build (20141003000203)
3) Go to a few website to create a list of browsing history.
4) Perform OTA update.
5) Open New window screen on system browser on 2.1 after OTA upgrade.
  
Actual:
"History" list is empty.
  
Expected: 
"History" displays browsing history from 2.0.

Note:
Top Site and rocket bar result show the browsing history from 2.0 after upgrade to 2.1.

[Before OTA - Flame 2.0]
 
Environmental Variables:
Device: Flame 2.0 (319mb)(Full Flash)
BuildID: 20141003000204
Gaia: fa797854bfe708129ed54a158ad4336df1015c39
Gecko: 9b7fd1f78a15
Gonk: 2c909e821d107d414f851e267dedcd7aae2cebf
Version: 32.0 (2.0)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0
  
[After OTA - Flame 2.1]

Environmental Variables:
Device: Flame 2.1
BuildID: 20141003000203
Gaia: 9861c61ec302fb0316c753a2e1c0f592180515fd
Gecko: da68900d1c66
Gonk: 2c909e821d107d414f851e267dedcd7aae2cebf
Version: 34.0a2 (2.1)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

Repro Frequency: 100%
See attached: logcat, screenshot

This issue fails verification of Bug 1055593.

Comment 1

[Yeojin Chung [:YeojinC]]

Attachment: BroswerHistoryNotMigrated.png

This issue also reproduces when upgrading from 2.0 to 2.2:

[Before OTA - Flame 2.0]
 
Environmental Variables:
Device: Flame 2.0 (319mb)(Full Flash)
BuildID: 20141003000204
Gaia: fa797854bfe708129ed54a158ad4336df1015c39
Gecko: 9b7fd1f78a15
Gonk: 2c909e821d107d414f851e267dedcd7aae2cebf
Version: 32.0 (2.0)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0
  
[After OTA - Flame 2.2]

Environmental Variables:
Device: Flame 2.2 Master
BuildID: 20141003040207
Gaia: d711d1e469eeeecf25a02b2407a542a598918b2c
Gecko: b85c260821ab
Gonk: 2c909e821d107d414f851e267dedcd7aae2cebf
Version: 35.0a1 (2.2 Master)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:35.0) Gecko/35.0 Firefox/35.0

"History" is empty after upgrading to 2.2.

Comment 2

[Marcia Knous [:marcia]]

[Blocking Requested - why for this release]: Nominating for blocker since this is dataloss and if we support this upgrade path then this needs to work.

Comment 3

[Marcia Knous [:marcia]]

Yeojin: How many times did you repeat the upgrade, and which profiles were you using - the ones that are typically used for OTA?

Comment 4

[Yeojin Chung [:YeojinC]]

I tried 3 times total, and I did not use any profile. I just went to a few website on Browser, and performed OTA.

Comment 5

[Gregor Wagner [:gwagner]]

Dataloss and regression

Comment 6

[Dale Harvey (:daleharvey)]

My internet gets installed today so will be able to start working on it in a few hours

Comment 7

[Dale Harvey (:daleharvey)]

Kyle mentioned he may look at bugmail, but pinging Alexandre as well

Either of you remember where I need to copy the browsers idb sqlite files form / to in order to test this?

Comment 8

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Steps for testing:

- Make 2.0 profile
- Run FxOS, make history in browser app
- Go into profile/storage/persistent (b2g desktop) or /data/local/storage/persistent (phone, at least, I think this was it? I can't remember and don't want to boot my phone. Ask fabrice if that's wrong.)
- Look for the directory name with "browser" in it. Copy that somewhere.
- Checkout a 2.1/2.2 repo. Wedge the migrator to always run somehow (usually just comment out the version check around the promise and run the function in the then parameters).
- Make a 2.1/2.2 profile. 
- If on b2g desktop, run FxOS on profile, just to set up databases. If you're doing this on the phone, this step isn't needed because we prebuild the databases.
- Go into 2.1/2.2 profile, storage/persistent, then the directory with "system" in it.
- Copy contents of the folder in the browser storage directory you went to earlier into the system folder. You should now have multiple databases in the 'idb' directory, and one of them should be named "2959517650brreosw.sqlite"
- Run FxOS again. This should cause system to run the update.

Lemme know if that works for you. Doing this from memory and it's been a couple of weeks.

Comment 9

[Dale Harvey (:daleharvey)]

Cheers

Comment 10

[Dale Harvey (:daleharvey)]

Finally reproduced this, the migration is running and it saves the old visits into the places store, however it doesnt correctly pull out the timestamps of the history so it only shows them as top sites but cant create the history view, should have a fix shortly

Comment 11

[Dale Harvey (:daleharvey)]

Attachment: https://github.com/mozilla-b2g/gaia/pull/24990

Silly mistake, apologies

Comment 12

[Kevin Grandon :kgrandon]

In master: https://github.com/mozilla-b2g/gaia/commit/88514af18d8abb4f1beaded08d0c48ee17d8118d

Comment 13

[Tony Chung [:tchung]]

kevin, please have someone verifyme this bug is fixed on master.  this will need uplift to 2.1

Comment 14

[Kevin Grandon :kgrandon]

Comment on attachment 8502414 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/24990

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): Feature implementation.
[User impact] if declined: Missing bookmarks on upgrade.
[Testing completed]: Manual and unit testing.
[Risk to taking this patch] (and alternatives if risky): Low risk, one-liner change.
[String changes made]: None.

Comment 15

[KTucker [:KTucker][Inactive 3/4/2016]]

Adam, please verify this on master and then 2.1 once it has been uplifted.

Comment 16

[Adam Alldredge [:AdamA][Inactive]]

This issue is verified fixed on Flame 2.2:

[Before OTA - Flame 2.0]
  
Environmental Variables:
Device: Flame 2.0 (319mb, Full Flash)
BuildID: 20141010000201
Gaia: 9791940d5093c37336dcd85f7e66e9a8445a218e
Gecko: 00d3c843aeec
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 32.0 (2.0)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0
  
[After OTA - Flame 2.2]

Environmental Variables:
Device: Flame 2.2 Master
BuildID: 20141010040202
Gaia: 1036b544b7e102592bd9fab95cd9317329ac1293
Gecko: 50b689feab5f
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 35.0a1 (2.2 Master)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:35.0) Gecko/35.0 Firefox/35.0

"History" displays browsing history from 2.0.

Comment 17

[bhavana bajaj [:bajaj]]

Comment on attachment 8502414 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/24990

thanks for the pre-empt QA verification on trunk here !

Comment 18

[Ryan VanderMeulen [:RyanVM]]

v2.1: https://github.com/mozilla-b2g/gaia/commit/4d691707825960d9e296d793008d72de6f43b078

Comment 19

[Brogan Zumwalt [Inactive]]

This issue is verified fixed on Flame 2.1:

[Before OTA - Flame 2.0]
  
Environmental Variables:
Device: Flame 2.0 KK (319mb) (Full Flash)
BuildID: 20141010000201
Gaia: 9791940d5093c37336dcd85f7e66e9a8445a218e
Gecko: 00d3c843aeec
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 32.0 (2.0)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0
  
[After OTA - Flame 2.1]

Environmental Variables:
Device: Flame 2.1 KK (319mb)
BuildID: 20141011000201
Gaia: f5d4ff60ffed8961f7d0380ada9d0facfdfd56b1
Gecko: d813d79d3eae
Gonk: 52c909e821d107d414f851e267dedcd7aae2cebf
Version: 34.0a2 (2.1)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

"History" displays browsing history from 2.0 after OTA

Comment 20

[Dale Harvey (:daleharvey)]

IS there a tracking bug for integration tests that involve OTA's? this was fairly impossible to write an integration test for when I looked into it

Comment 21

[Kevin Grandon :kgrandon]

I'm fine going with testsuite- for now, but let's be sure we have a bug on file to make this actionable before doing so. For now I've filed bug 1083253 to track upgrade testing support. This may or may not be a dupe. I'll do a NI? dance in that bug.