253935 - shift+delete doesnt delete form entries due to corrupted formhistory.dat

Status: VERIFIED FIXED

(Toolkit :: Form Manager, defect)

Description

[José Jeria]

After bug 242250 was fixed, the form entries still came back when browser was
restarted. This is due to that formhistory.dat is corrupted (see bug 242250,
comment 22). This was tested on Windows 2000 and on a XP machine.

Steps to reproduce:
1. Delete an form entry with shift+delete
2. Restart browser

Actual results:
Deleted form entry is back

Expected results:
Deleted form entry should not appear anymore

This bug will not appear no more if formhistory.dat is renamed or deleted. Form
entry deletion will work as expected.

Comment 1

[Brian Ryner (not reading)]

Ok, so we have some sort of mork database corruption happening here.  I think
the main thing we need here is a corrupted formhistory.dat to analyze, but
unfortunately due to the mork format it's difficult to tell if your
formhistory.dat contains any sensitive data.  I'll test out my corrupted
formhistory.dat in a debug build and see if I'm triggering any assertions in mork.

Comment 2

[alanjstr]

I know this has been asked many times, but why do we keep using mork?

Comment 3

[Brian Ryner (not reading)]

So, in my case, the symptom I notice is that I'm unable to get a particular
value to stick in the "Branch" input on
http://bonsai.mozilla.org/cvsqueryform.cgi .  I tried, just now, hooking up a
nsIMdbErrorHook to satchel's mork instance.  I got this error upon loading that
page, and nothing else:

mork error: undefined cell value alias

bienvenu, does that mean anything to you?

Comment 4

[David :Bienvenu]

It sounds like an atom id is not in the dictionary, but is in a row cell -
Brian, do you have a reproducible set of steps to corrupt a previously
uncorrupted mork db? That would be a first - the only way I know of to corrupt a
mork db is to have two instances of mozilla writing to a mork db at the same
time (formhistory uses mork? Is that new? I knew plain old history used mork,
but not formhistory)

Comment 5

[alanjstr]

I only have on instance running at a time, except during the DDExec problem
Firefox encountered recently.  Is it possible to uncorrupt a mork?  I guess
that's what you're working on.

Comment 6

[David :Bienvenu]

Looking at the formhistory.dat file, it looks to me that it's likely that two
instances of Firefox were writing to it, specifically, appending rows with the
same id to it. Since the formhistory code lets Mork pick the row ids, and Mork
just uses monotonically increasing row ids, if you have two instances of Firefox
writing to Mork, they're quite likely to pick the same row id. And since new
rows are added by just appending transactions to the file, both instances will
end up in the file. That's if you're lucky, I guess. You can end up with much
worse corruption.

Once this happens, I believe formhistory changes are never remembered, because
the formhistory code is ignoring the error that Mork returns about the db being
corrupted. So it only takes one incident of getting two instances of Firefox
writing to formhistory.dat to make this problem permanent. I can attach a patch
that will cause the form history code to delete the formhistory.dat file and get
on with life, which is what the code intended to do, I believe (it's currently
just ignoring one of the error paths). However, I think it's possible, depending
on how ambitious you are, to recover the data in this case, by writing out the
history we were able to parse to a temp formhistory.dat file and coying that
over the original, instead of simply blowing it away - I know if it was up to
me, I'd like to keep as much of my form history as possible. I'll spend a few
minutes seeing if that approach will work.

Comment 7

[David :Bienvenu]

Attachment: delete corrupted formhistory.dat

I haven't tried this yet, but it should work. If it does, next I'll see if it's
possible to save some of the data...

Comment 8

[alanjstr]

Deleting this file will result in someone filing a dataloss bug.  Full recovery
would of course be best, but some has to be better than none.  Is it possible to
hack Mork to be able to either A) recover or B) prevent this from happening in
the first place?  I don't think I've ever had 2 browsers open within the past 6
months.

Comment 9

[David :Bienvenu]

Attachment: fix that tries to retain old data

this fix successfully restored the data in the corrupted formhistory.dat that I
have access to, and wrote it out to a new, uncorrupted formhistory.dat. It's
rather specific to the particular failure path caused by this form of
corruption, where we were actually able to read in the table almost all the way
to the end.

Re how this got corrupted, as I said before, it looks very much like two
instances wrote to the file - we count on the profile manager locking to
prevent this. Once the DB got corrupted, it would have stayed corrupted
forever, with the existing code. Without reproducible steps to corrupt a
formhistory.dat file, that's about all I can say - it's possible of course that
there's a bug in Mork that can produce this corruption, but I'd need some sort
of way to reproduce the problem.

Comment 10

[Ryan Polk (Quark)]

*** Bug 255276 has been marked as a duplicate of this bug. ***

Comment 11

[chris hofmann]

plussing...

Comment 12

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Thought I'd note recorruption of formhistory.dat after renaming and receiving
fresh one is reappearing as of 20040812 Firefox/0.9 build.

Will be watching closely for how/when conditions appear. It may be during
browser crashes though I have no idea.

Comment 13

[Tracy Walker [:tracy]]

*** Bug 242836 has been marked as a duplicate of this bug. ***

Comment 14

[Brian Ryner (not reading)]

checked in

Comment 15

[Hasham]

Verified on Gran Paradiso 3.0.1pre.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1pre) Gecko/2008062306 GranParadiso/3.0.1pre