509252 - formhistory corrupted a few days after 3.5.2 beta install

Status: RESOLVED DUPLICATE of bug 506905

(Toolkit :: Form Manager, defect)

Description

[Peter Beckman]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2

A few days after installing 3.5.2 beta (installed on 8/3/2009 11:04am EDT), my Built-in Search Bar stopped working.  A further look into things found a corrupted formhistory.sqlite file was to blame, and an export and subsequent import resolved the issue.

Reproducible: Didn't try

Actual Results:  
Search bar stopped working, neither Enter/Return nor clicking on the magnifying glass worked. 

Expected Results:  
Not corrupted the formhistory.sqlite.

I was following Shiretoko for a while, and I got a notice that 3.5.2 beta was available.  Being a good developer, I installed.

I'm not sure exactly when my Search Bar started to not work, but I did my research and found that a corrupt .sqlite file could likely be to blame.  I  searched bugzilla for a recently opened bug relating to the integrated search toolbar not working, and working again after removing a .sqlite file in the profile, and found a few related issues: bug 503492 (requested I make this a new issue), bug 494272 (my mac pro isn't going to sleep, and I'm not on windows)

I posted the process I took to restore my Search Bar functionality:
https://support.mozilla.com/tiki-view_forum_thread.php?forumId=1&comments_parentId=399500

I'm using a profile I started fresh when 3.5 came out.

I followed a process mentioned in a blog about how to recover from a corrupted
.sqlite file.  I used the same process using my formhistory.sqlite and found
the following interesting:

// Broken formhistory.sqlite file
sqlite> select count(*) from moz_formhistory;
1918

// Dump to a file, read it back in
sqlite> .read was_formhistory_sql
SQL error near line 1: PRIMARY KEY must be unique
SQL error near line 1879: moz_formhistory.fieldname may not be NULL
SQL error near line 1880: moz_formhistory.fieldname may not be NULL
SQL error near line 1881: moz_formhistory.fieldname may not be NULL
SQL error near line 1882: moz_formhistory.fieldname may not be NULL
SQL error near line 1883: moz_formhistory.fieldname may not be NULL
SQL error near line 1884: moz_formhistory.fieldname may not be NULL
SQL error near line 1885: moz_formhistory.fieldname may not be NULL
SQL error near line 1886: moz_formhistory.fieldname may not be NULL
SQL error near line 1887: moz_formhistory.fieldname may not be NULL
SQL error near line 1888: moz_formhistory.fieldname may not be NULL
SQL error near line 1889: moz_formhistory.fieldname may not be NULL
SQL error near line 1890: moz_formhistory.fieldname may not be NULL
SQL error near line 1891: moz_formhistory.fieldname may not be NULL
SQL error near line 1892: moz_formhistory.fieldname may not be NULL
SQL error near line 1893: moz_formhistory.fieldname may not be NULL
SQL error near line 1894: moz_formhistory.fieldname may not be NULL
SQL error near line 1895: moz_formhistory.fieldname may not be NULL
SQL error near line 1896: moz_formhistory.fieldname may not be NULL
SQL error near line 1897: moz_formhistory.fieldname may not be NULL

I opened up the broken formhistory.sqlite file and tried a few things:
sqlite> select * from moz_formhistory where fieldname is null;
SQL error: database disk image is malformed
sqlite> select * from moz_formhistory;
[...]
1878|searchbar-history|firefox rules|1|1249452582092832|1249452582092832
47|||||
48|||||
36|||||
62|||||
45|||||
49|||||
53|||||
55|||||
52|||||
35|||||
37|||||
41|||||
35|||||
38|||||
35|||||
70|||||
79|||||
54|||||
39|||||
1898|exchange|478|1|1249495711716646|1249495711716646

The timestamps prior to and just after the corrupted period:

Wed, 05 Aug 2009 06:09:42 GMT
[..corruption..]
Wed, 05 Aug 2009 18:08:31 GMT

Please note my computer is on America/New_York time zone, so those may be local
times rather than GMT.  I'm not sure when I updated, but I'm guessing this
might indicate that it was earlier this week, which matches the timestamp on
the Firefox binary:

/Applications/Firefox.app/Contents/MacOS/updates:
[...]
-rw-r--r--@  1 beckman  admin  151043 Aug  3 11:04 last-update.log

Is there a log of what AddOns you've updated or installed and when that
occurred?  There probably should be, along with their versions.

It's clear there was some period of time where adding things to the formhistory
was either broken or corrupted.  Maybe it will happen to me again.  I am running this on my Mac Pro and my Macbook Pro, both running 3.5.2, and I have only seen this happen with my Mac Pro thus far.  My last crash was 7/31/09, so it wasn't a crash.

I am on a Mac, OS X 10.5.7
Darwin max 9.7.0 Darwin Kernel Version 9.7.0: Tue Mar 31 22:52:17 PDT 2009;
root:xnu-1228.12.14~1/RELEASE_I386 i386 i386

I was running 3.5 just fine until I got a notice to update to 3.5.2 beta.  I'm
not sure if I was running 3.5.1 prior to that.  Is there an
"about:versionhistory" that shows me all the previous versions with their date
of release, as well as date/time I installed them?  That would be wicked sexy
and handy for this sort of thing.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Peter, did you had a crash or system freeze in the last days? Those corruption cannot happen by Firefox itself. I had a similar problem with the places.sqlite. See bug 506905. Eventually both bugs could be related.

Comment 2

[Peter Beckman]

Now that you mention it, I'm remembering that my Mac Pro completely shut off when my UPS overloaded... *looks up UPS data stored in syslog*

Ah! Maybe not coincidence.  The crash happened at around 2am EDT on August 5.  Strange though that I would have a valid entry after that time, unless the times are truly in GMT, at which point it would be spot on that the last valid entry was at 6:09 am (2:09am EDT) and then corruption after that.

This was a full power loss situation.  My UPS got into an overload and shut off (epic fail).  I'm now certain that was it.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks Peter! Would you see any chance that such a situation could be reproduced? I already tried to find a way on my box but wasn't able to freeze it by using VLC again. Peter, can you please give a short wrap-up from your comment 2 in bug 506905? We are still searching a possible path how such a situation can happen so we can forward it to the sqlite team. Lets move over to bug 506905.