Closed Bug 805967 Opened 12 years ago Closed 12 years ago

[OTA update] much of my user data blown away in today's update

Categories

(Firefox OS Graveyard :: General, defect, P1)

Product:
Firefox OS Graveyard
Component:
General
Platform:
x86
macOS
Type:
defect

Tracking

(blocking-basecamp:+, firefox18 fixed)

Status:
RESOLVED FIXED
Project Flags:
blocking-basecamp +
Tracking Flags:
Tracking Status
firefox18 --- fixed

People

(Reporter: dietrich, Assigned: fabrice)

References

Details

(Keywords: smoketest, Whiteboard: [dogfooding-blocker])

Attachments

(2 files)

logcat
185.12 KB, text/plain
Details
patch
1.57 KB, patch
sicking
: review+
Details | Diff | Splinter Review 
* homescreen customizations gone
* browser bookmarks and history gone
* all calendar accounts deleted
* all email accounts deleted
* not sure what else

Settings seem to be intact, as well as apps installed from Marketplace.
blocking-basecamp: --- → ?
The updates themselves can't touch user data, but we need to understand what happened.

Dietrich, do you see any errors in logcat?

Were there any changes to the app DB schemas that might have caused this?

Did something change in the identification of apps that would have led to us stranding their old DBs?

Did something in IDB change?  (I sure hope not, on aurora!)
Severity: normal → critical
blocking-basecamp: ? → +
Given that all the apps private data has been removed, but not the chrome databases (like the settings) I tend to think that we hit this codepath:
https://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#215

But since the apps from the marketplace are still there, we loaded the apps registry from /data/local/webapps/webapps.json properly and we should not remove anything.

unfortunately Dietrich's logcat is not useful thanks to the ONCRPC flood.
Coincidence-wise, SQLite 3.7.14.1 landed last night in Bug 801281, but we were only updating from 3.7.14 and if you look at the diff there were almost no changes.  So we should rule that out.
(In reply to Fabrice Desré [:fabrice] from comment #2)
> Given that all the apps private data has been removed, but not the chrome
> databases (like the settings) I tend to think that we hit this codepath:
> https://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#215
> 

Is there any logging we can add there to determine why the apps are blown away?

Guys, this blocks updates on the stable channel.  Sorry Dietrich, but thanks for being a guinea pig :/.
Whiteboard: [dogfooding-blocker]
Priority: -- → P1
:cjones :fabrice :marshall_law
who owns this?  for the time being assigning to :marshall_law unless some other owner pops up.
Assignee: nobody → marshall
We first need to reproduce it. Dietrich, were you on the stable channel? (where the last update is from 2012-10-24)
I reflashed a stable build from 2012-10-23, installed a couple of apps, played a bit with the browser, added some contacts and changed some settings.

Then I updated to the 2012-10-24, but no data loss occured nor setting changes.
I believe that dietrich was on the nightly channel.  There wasn't a 10-26 stable update.

dietrich, please confirm.
Flags: needinfo?(dietrich)
Dietrich confirmed that he was on the nightly channel. I will try to reproduce again.
Flags: needinfo?(dietrich)
Attached file logcat — 
I can confirm that a poor man's update from 10-25 nightly to 10-26 (re-flash /system with 10-26) blew away my homescreen customizations, but not my settings or contacts.  So I think this is the same problem as described in comment 0.

A *very* suspicious-looking character in logcat is

E/GeckoConsole(  106): [JavaScript Error: "PermissionsTable.jsm: expandPermissions: Unknown Permission: background-sensors" {file: "resource://gre/modules/PermissionsInstaller.jsm" line: 260}]
E/GeckoConsole(  106): [JavaScript Error: "Error: PermissionsTable.jsm: expandPermissions: Unknown Permission: background-sensors" {file: "resource://gre/modules/PermissionsInstaller.jsm" line: 389}]
Doing the same poor man's update from 10-25 --> 10-29 exhibited the same symptom, loss of data.

The permissions error wasn't in logcat, so that's not the culprit.
There's also Bug 786295 - That would fit "nicely" with comment #0, where everything backed by indexedDB disappears.
(In reply to Fabrice Desré [:fabrice] from comment #14)
Yeah, before bug 786295 we didn't have the ability to clear indexedDB apps in response to the uninstall notification.
But we still should not clear anything. Assigning to me since this is triggered from Webapps.jsm
Assignee: marshall → fabrice
(In reply to Chris Jones [:cjones] [:warhammer] from comment #18)
> Confirmed to be bug 786295.

Wait, to be clear, the bug is that we're calling this clearPrivateData function during an update, not that IndexedDB is listening to that signal.
That's fine, but bug 786295 introduced this regression.  I understand that that cset was an innocent bystander, of sorts.
I put a dump() at all the _clearPrivateData() calls in Webapps.jsm, including _clearPrivateData() itself, and none of them were hit ...  The plot thickens.
yeah, just did the same here, forcing runUpdate to be true. We never hit this path, as expected.
Oh! I think this happened because bug 786295 change the naming scheme for indexedDB databases. So technically no data was deleted, we're just looking for it at the "wrong" filename.

So yes, this will lose a bunch of data on update, but it's a one-shot deal. It's not expected to happen again. I.e. this will only happen when updating this once, it won't happen on every update.
Attached patch patch — — Splinter Review 
Patch renaming indexedDB folders when running an update.

Before:

fabrice@fabrice-x230:~/b2g-distro$ adb shell ls /data/local/indexedDB
1002+app+++browser.gaiamobile.org+f
1004+app+++costcontrol.gaiamobile.org+f
1006+app+++homescreen.gaiamobile.org+f
1009+app+++system.gaiamobile.org+f
chrome

After:

fabrice@fabrice-x230:~/b2g-distro$ adb shell ls /data/local/indexedDB
1002+f+app+++browser.gaiamobile.org
1004+f+app+++costcontrol.gaiamobile.org
1006+f+app+++homescreen.gaiamobile.org
1009+f+app+++system.gaiamobile.org
chrome
Attachment #676490 - Flags: review?(jonas)
We need some temporary pref to force this upgrade path right?  Won't firstRun be false for deployed dogfood devices?
(In reply to Chris Jones [:cjones] [:warhammer] from comment #25)
> We need some temporary pref to force this upgrade path right?  Won't
> firstRun be false for deployed dogfood devices?

No, this test is true for "real" first run and for gecko+gaia updates.
Keywords: smoketest
Backed out along with several others in order to get inbound green again, after a busted landing meant we lost test coverage for 7 pushes, and now have multiple failures:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=a145ded68994

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/553fb59b9ca0
\o/

Do you mind filing a followup to remove this temporary upgrade code?
Component: Gaia → General
(In reply to Chris Jones [:cjones] [:warhammer] from comment #32)
> \o/
> 
> Do you mind filing a followup to remove this temporary upgrade code?

I filed bug 807216
https://hg.mozilla.org/mozilla-central/rev/58ddfb88815a
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: