1100663 - deleteDatabase with newer schema doesn't remove everything

Status: RESOLVED WONTFIX

(Core :: Storage: IndexedDB, defect)

Description

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

While investigating bug 1096310 (WebIDE's project list is broken if you use a Fx 36 profile with Fx 35), I noticed that deleting an IDB database also fails in the older Firefox version.

It would be nice to at least be able to reset the entire DB from JS one way or another.  At the moment, you're left in a corrupted state because IDB will allow |deleteDatabase| to proceed in 35, but then any |open| request will still fail.

It seems this could be fixed to some degree at least for the future, so it's possible to handle this case at the application level.

Here is an example scenario from WebIDE:

For WebIDE, we create an "AppProjects" IDB database.

In Firefox 36, this adds the files:

* <PROF>/storage/persistent/indexeddb+++fx-devtools/idb/4268914080AsptpcPerjo.files/
* <PROF>/storage/persistent/indexeddb+++fx-devtools/idb/4268914080AsptpcPerjo.sqlite

If I then switch to Firefox 35 and try to reset everything via |deleteDatabase|, the request will succeed, but only the "4268914080AsptpcPerjo.sqlite" is deleted.

At this point the data is now gone, but Firefox 35 will refuse to open any new IDB databases, including the supposedly deleted one.  The "4268914080AsptpcPerjo.files/" directory (which is empty) is left behind.  If I manually delete the *.files directory, I can now create a new IDB in Firefox 35.

Could we possibly change "deleteDatabase" to remove everything it finds that starts the database name (I assume this is the "4268914080AsptpcPerjo" part)?

Comment 1

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Jan, is the any hope to be "more thorough" with |deleteDatabase| as suggested above?

If no, I guess it's best to WONTFIX this.

Comment 2

[Jan Varga [:janv]]

(In reply to J. Ryan Stinnett [:jryans] from comment #1)
> Jan, is the any hope to be "more thorough" with |deleteDatabase| as
> suggested above?
> 
> If no, I guess it's best to WONTFIX this.

Yeah, deleteDatabase should probably remove the old directory (without the .files suffix).
And we should do it before the uplift to Aurora.

Comment 3

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Jan, 35 is now on Beta, and almost to Release...  Any chance of something happening here?

Or, at least something for future versions would be nice.

Comment 4

[Jan Varga [:janv]]

Hm, so you are actually proposing to patch Firefox 35 (before it is released) to also remove the directory with the .files suffix (even though the suffix is not used in 35 at all) ?
Well, that could work, but it doesn't handle the case, when someone upgrades to Firefox 36 from an older release, are you ok with that ?

Comment 5

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Jan Varga [:janv] from comment #4)
> Hm, so you are actually proposing to patch Firefox 35 (before it is
> released) to also remove the directory with the .files suffix (even though
> the suffix is not used in 35 at all) ?

Yes, I was hoping 35's |deleteDatabase| would do the equivalent of "enough removal to make a new database work again".  So in the case that 36 was used and then you go back to 35, I guess it would need to also remove the .files suffix.  As I mentioned in comment 0, could we instead remove all files and directories that start with the origin ID "4268914080AsptpcPerjo"?  That seems like it could future proof against further file moves, but of course is a bit riskier since it could remove more things than desired at some future time.

At the same time, this is another symptom of moving a profile from newer to older Gecko, and you've said in bug 1117129 comment 2 that's not officially supported.  So, I wasn't sure if we even want to try fixing these kinds of issues.

> Well, that could work, but it doesn't handle the case, when someone upgrades
> to Firefox 36 from an older release, are you ok with that ?

I am not sure what you mean here.  If you are moving a profile forward Gecko versions, I believe things should work correctly with the version migrations IndexedDB already does, right?  I don't see why this |deleteDatabase| idea would affect that.

If we made a change in |deleteDatabase| here to 35, I would want it also be in all later Gecko versions as well (as usual), if that's what you mean.

Comment 6

[Ben Turner (not reading bugmail, use the needinfo flag!)]

35 is basically done. I don't think we'll be able to make any changes to that version.

Comment 7

[Jan Varga [:janv]]

(In reply to J. Ryan Stinnett [:jryans] from comment #5)

> As I mentioned in comment 0, could we instead remove all files and directories that start with the origin > ID "4268914080AsptpcPerjo"?

FYI, it's not origin ID, it's a hash + scrambled database name

I would rather not delete everything which begins with the prefix.

>> Well, that could work, but it doesn't handle the case, when someone upgrades
>> to Firefox 36 from an older release, are you ok with that ?

> I am not sure what you mean here.  If you are moving a profile forward Gecko
> versions, I believe things should work correctly with the version migrations
> IndexedDB already does, right?  I don't see why this |deleteDatabase| idea
> would affect that.
> 
> If we made a change in |deleteDatabase| here to 35, I would want it also be
> in all later Gecko versions as well (as usual), if that's what you mean.

Sorry, I was not clear. Let's say someone uses 36 and then goes back to 34 or earlier. In that case the .files directory wouldn't be deleted. I guess that's quite rare.

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Jan Varga [:janv] from comment #7)
> (In reply to J. Ryan Stinnett [:jryans] from comment #5)
> 
> > As I mentioned in comment 0, could we instead remove all files and directories that start with the origin > ID "4268914080AsptpcPerjo"?
> 
> FYI, it's not origin ID, it's a hash + scrambled database name
> 
> I would rather not delete everything which begins with the prefix.

Okay.  Given that and comment 6 that 35 is basically done, I think there's not much we can do here.

Maybe I'll try to dream up some future world where this kind of backward profile movement can be better supported. :)

> >> Well, that could work, but it doesn't handle the case, when someone upgrades
> >> to Firefox 36 from an older release, are you ok with that ?
> 
> > I am not sure what you mean here.  If you are moving a profile forward Gecko
> > versions, I believe things should work correctly with the version migrations
> > IndexedDB already does, right?  I don't see why this |deleteDatabase| idea
> > would affect that.
> > 
> > If we made a change in |deleteDatabase| here to 35, I would want it also be
> > in all later Gecko versions as well (as usual), if that's what you mean.
> 
> Sorry, I was not clear. Let's say someone uses 36 and then goes back to 34
> or earlier. In that case the .files directory wouldn't be deleted. I guess
> that's quite rare.

Ah, I see.  Well, that would break, you are right, but I had just put 34 out of my mind, since I was quite sure we would not patch Release. :)

At this point, I don't see a change we can make to improve this.  I'll file a new bug if I happen to concoct some kind future-proof plan down the road.