1092090 - Reverting to previous version of Firefox makes indexeddb database inaccessible

Status: RESOLVED WONTFIX

(Core :: Storage: IndexedDB, defect, P3)

Description

[Mark Banner (:standard8)]

STR:

1) Get the latest Nightly & Aurora (I'm using builds from 30th Oct 2014).
2) Create a new profile
3) Load profile in Firefox Nightly
4) Sign into Firefox Hello
5) On the contacts tab of the Hello Panel, create a new contact. See that it is visible in the list.
6) Exit Firefox
7) Start the profile in Firefox Aurora
8) Open the Hello panel and select the contacts tab

Expected Results

=> Contact is displayed

Actual Results

=> No Contacts
=> Notice on the browser console:

IndexedDB UnknownErr: ActorsParent.cpp:413
Error:  LoopStorage.jsm:98

=> Switch back to Firefox Nightly - the same issue remains.

Run a custom debug build, see the following errors on the output:

[65276] WARNING: '!validSubdirs.GetEntry(subdirName)', file /Users/mark/loop/gecko-dev/dom/indexedDB/ActorsParent.cpp, line 9273
[65276] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x8000FFFF: file /Users/mark/loop/gecko-dev/dom/quota/QuotaManager.cpp, line 1724
[65276] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x8000FFFF: file /Users/mark/loop/gecko-dev/dom/quota/QuotaManager.cpp, line 1861
[65276] WARNING: 'NS_FAILED(rv)', file /Users/mark/loop/gecko-dev/dom/indexedDB/ActorsParent.cpp, line 11208
[65276] WARNING: 'NS_FAILED(rv)', file /Users/mark/loop/gecko-dev/dom/indexedDB/ActorsParent.cpp, line 11069
[65276] WARNING: Converting non-IndexedDB error code (0x8000FFFF) to NS_ERROR_DOM_INDEXEDDB_UNKNOWN_ERR: file /Users/mark/loop/gecko-dev/dom/indexedDB/ActorsParent.cpp, line 408
[65276] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file /Users/mark/loop/gecko-dev/dom/quota/QuotaManager.cpp, line 2099

The only way to recover a profile from this, is to go into the profile and remove everything in storage/persistent/chrome.

Comment 1

[Jan Varga [:janv]]

Hm, do we officially support going back to older releases with the same profile ?

Comment 2

[Mark Banner (:standard8)]

(In reply to Jan Varga [:janv] from comment #1)
> Hm, do we officially support going back to older releases with the same
> profile ?

Even if we don't entirely, this is a pain for developers switching between branches - especially as it completely ruins the indexeddb use in the profile - without manually removing files.

i.e. once in the broken state, you're broken whatever version you use.

Comment 3

[Ben Turner (not reading bugmail, use the needinfo flag!)]

(In reply to Mark Banner (:standard8) from comment #0)
> The only way to recover a profile from this, is to go into the profile and
> remove everything in storage/persistent/chrome.

Wait, you can't access the data again once you run with the newer version again?

Comment 4

[Mark Banner (:standard8)]

(In reply to ben turner [:bent] (use the needinfo? flag!) from comment #3)
> (In reply to Mark Banner (:standard8) from comment #0)
> > The only way to recover a profile from this, is to go into the profile and
> > remove everything in storage/persistent/chrome.
> 
> Wait, you can't access the data again once you run with the newer version
> again?

Yes, that is correct. The profile stays broken.

Comment 5

[Jan Varga [:janv]]

Yeah, that can happen, because the origin dir now contains two directories for files, one without the suffix and the other one with it.

Comment 6

[Mark Banner (:standard8)]

[Tracking Requested - why for this release]: Dataloss and usability issue - profile breaks if switching back to the previous release and is not recoverable when reverting to the current release.

The only way out of it is to manually remove profile files on disk.

Comment 7

[Lukas Blakk [:lsblakk] use ?needinfo]

Is this true of any data prior to Hello though?  I hear the developer pain issue but if this type of data corruption occurs with other indexddb entries going back to older releases then this isn't a tracking issue as it's not a supported user behaviour or expected in the wild outside of Firefox developers.

Comment 8

[Jan Varga [:janv]]

(In reply to Jan Varga [:janv] from comment #5)
> Yeah, that can happen, because the origin dir now contains two directories
> for files, one without the suffix and the other one with it.

Actually that shouldn't happen ...
I found out what the problem is, when you go back to Aurora, opening of the loop database fails which is expected, but the already existing loop database with .sqlite suffix is also removed from the disk, but the directory for stored files is not removed because aurora doesn't use the .files suffix for stored files.

So to fix this bug, we have to allow the stored files directory without the suffix to exist (even without corresponding .sqlite file) in the current nigthly. Hopefully, this won't cause more problems.

Comment 9

[Mark Banner (:standard8)]

(In reply to Lukas Blakk [:lsblakk] use ?needinfo from comment #7)
> Is this true of any data prior to Hello though?  I hear the developer pain
> issue but if this type of data corruption occurs with other indexddb entries
> going back to older releases then this isn't a tracking issue as it's not a
> supported user behaviour or expected in the wild outside of Firefox
> developers.

I only tested this with Hello. Unfortunately I don't know what other parts of Firefox use indexeddb, and I don't really have time to test for the next week or so.

It seems to be a generic issue from the comments in the bug... maybe the folks here know where else indexeddb gets used.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

Can you confirm if this is a recent regression or just the way it's always been?

Comment 11

[Jan Varga [:janv]]

(In reply to Lukas Blakk [:lsblakk] use ?needinfo from comment #10)
> Can you confirm if this is a recent regression or just the way it's always
> been?

This is a regression from bug 1056939.

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

Thank you - in that case, this has been the circumstance for a while and if a low risk fix is available we can consider it for uplift, but not tracking this for a specific release.

Comment 14

[Brad Lassey [:blassey] (use needinfo?)]

Ben, can you have a look at this? It is suspected of being the root cause of loop not working for some users.

Comment 15

[Jan Varga [:janv]]

Ben, I think we just need to ignore *.files sub directories (if there's no corresponding .sqlite file) during origin directory initialization. We can then remove this exception in some later release.

I can try to do that, if you agree.

Comment 16

[Ben Turner (not reading bugmail, use the needinfo flag!)]

(In reply to Brad Lassey [:blassey] (use needinfo?) from comment #14)
> Ben, can you have a look at this? It is suspected of being the root cause of
> loop not working for some users.

I don't understand... Loop users are commonly jumping back and forth between various versions of Firefox with the same profile? This is not something that can be supported generally, as we do one-way upgrades of databases all the time...

Comment 17

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Brad, see comment 16.

Comment 18

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to ben turner [:bent] (use the needinfo? flag!) from comment #16)
> (In reply to Brad Lassey [:blassey] (use needinfo?) from comment #14)
> > Ben, can you have a look at this? It is suspected of being the root cause of
> > loop not working for some users.
> 
> I don't understand... Loop users are commonly jumping back and forth between
> various versions of Firefox with the same profile? This is not something
> that can be supported generally, as we do one-way upgrades of databases all
> the time...

Ben, this was called out in the Loop meeting as potentially causing issues I'm seeing on Nightly. Maybe dmose can shed some more light on it

Comment 19

[Dan Mosedale (:dmosedale, :dmose)]

Ben and I just spent some time chatting about this, and he pointed out that the strategy of having duplicated data for people that flip between versions as suggested in comment 15 has some problems of its own (i.e. users may end up with the versions being out of sync, and not realizing that stuff they saved in the old version never will be seen in the new version).

More to the point, after Brad and I did some debugging on Friday, I no longer have a strong reason to believe that this is the root cause of any of the problems that Brad has been seeing.  Unless we actually come to the point where this does appear to be the problem, I'd propose backburnering this.

Comment 22

[Mark Banner (:standard8)]

Given that Hello is no longer in development, but mainly that we now explicitly don't allow downgrades, this is no longer applicable.

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.