Closed Bug 359564 Opened 18 years ago Closed 12 years ago

search.sqlite can be corrupted when running out of disk space, resulting in broken search functionality

Categories

(Firefox :: Search, defect)

defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: rkeyes, Unassigned)

References

Details

(Keywords: dataloss)

Attachments

(1 file, 1 obsolete file)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0

user experience: searching for anything in search window produces no result. selecting 'Manage Search Engines' produces popup showing no search engines installed. Selecting 'restore defaults' or 'get more search engines' produces no results. Neither does selection 'OK' or 'cancel'. Only selecting 'X' in window pane closes window. After performing these actions, the Error Console shows the following.

For Manage Search Engines:


Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824



Error: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]



For a search on string 'testsearch':

Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824

Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824

Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824

Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824

Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824


Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js :: epsGetAttr :: line 2824"  data: no]
Source File: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js
Line: 2824


AddOns installed:
 AdBlock 0.5.3.043
Google Toolbar for Firefox 2.1.20060807W



Reproducible: Always

Steps to Reproduce:
1. See above
2.
3.

Actual Results:  
crash

Expected Results:  
searches

see  above
Does it work if you rename search.sqlite in your profile? Don't delete it - if it's corrupted and is causing these problems, it would be good if you could attach it here for analysis.
Renaming this file (moving it out of the way) causes firefox to create a new one and solves the problems I was having. It would be nice if, somehow, Firefox was able to detect this corruption and handle it gracefully.
Attached file search.sqlite with obsolete entries (obsolete) —
Not sure if it's of interest or not but I had a look at my search.sqlite file which seems to be working just fine but includes many engines that I've since deleted (I can't really tell whether it's all of them or not or just the sherlock ones or something but I only have 10 installed at the moment)

Should it be saving all of them?
Should I file this somewhere else?
Attachment #245250 - Attachment mime type: application/octet-stream → text/plain
(In reply to comment #3)
> Should it be saving all of them?
> Should I file this somewhere else?

Currently it does not remove engine metadata when an engine is removed. Bug 341833 exists for that.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Attachment #245250 - Attachment is obsolete: true
Robert: It would be good to know what caused the search file to be corrupted - could it have been a crash or sudden reboot, or something like that?

Either way, you're right that it would be good ensure that the search service doesn't fail so spectacularly when search.sqlite corruption.
(In reply to comment #5)
> Robert: It would be good to know what caused the search file to be corrupted -
> could it have been a crash or sudden reboot, or something like that?
> 
> Either way, you're right that it would be good ensure that the search service
> doesn't fail so spectacularly when search.sqlite corruption.
> 

I am perpetually low on disk space: perhaps I ran out as I was upgrading to 2.0. Other applications were running at the time (I had closed firefox) which could have written to the disk in between a free-space check at the beginning of the installation and when the search.sqlite file was written. 
This happened again. I was able to verify that this was during a low disk space event (there was only 20kb left on the drive). Renaming search.sqllite and restarting worked. 
Thanks for confirming that, Robert. I'll try to see if I can reproduce, and add the code needed to protect the search service from such failures.
Summary: SearchService totally broken nssearchservice.js component returned failure code: 0x80004005 → search.sqlite can be corrupted when running out of disk space, resulting in broken search functionality
Assignee: nobody → gavin.sharp
*** Bug 361631 has been marked as a duplicate of this bug. ***
I hit this error as well, but I am not low on disk space (20GB available)
Flags: blocking-firefox3?
http://lxr.mozilla.org/mozilla/source/browser/components/search/nsSearchService.js#2906 seems like it should handle this, but let's at least make sure we recover properly from a corrupt search.sqlite

I'm not sure why we aren't using Cr.NS_ERROR_FILE_CORRUPTED here, fwiw.
Flags: blocking-firefox3? → blocking-firefox3+
Target Milestone: --- → Firefox 3 beta1
OS: Windows XP → All
Priority: -- → P2
Hardware: PC → All
Target Milestone: Firefox 3 M7 → Firefox 3 M8
Version: unspecified → Trunk
Target Milestone: Firefox 3 M8 → Firefox 3 M9
Target Milestone: Firefox 3 M9 → Firefox 3 M10
I hit this today on my mac. I have my hard drive partitioned with Leopard/Tiger and when I was downloading an updated VM I must have maxed out on disk space. The browser got in a very disturbed state and I had to restart. The scary thing is that my bookmarks did not show, among other things.
I hit this last night, and--like Marcia--my bookmarks toolbar folder was empty; not only that, but new tabs wouldn't populate their location bar with the URL's string.  A couple of other quick things.

1) I'm also not running out of disk space
2) Removing finished downloads from the DM failed, with:

Error: uncaught exception: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIDownloadManager.removeDownload]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: chrome://mozapps/content/downloads/downloads.js :: removeDownload :: line 212"  data: no]

3) I also see _other_ exceptions that might be important, such as:

Error: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageStatementWrapper.step]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: file:///Applications/Minefield.app/Contents/MacOS/components/nsContentPrefService.js :: ContentPrefService__selectPrefs :: line 586"  data: no]
Source File: file:///Applications/Minefield.app/Contents/MacOS/components/nsContentPrefService.js
Line: 586

Error: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageStatementWrapper.step]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: file:///Applications/Minefield.app/Contents/MacOS/components/nsSearchService.js :: epsGetAttr :: line 2872"  data: no]
Source File: file:///Applications/Minefield.app/Contents/MacOS/components/nsSearchService.js
Line: 2872

Error: Failed to init content pref service:
[Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageStatementWrapper.step]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: file:///Applications/Minefield.app/Contents/MacOS/components/nsContentPrefService.js :: ContentPrefService__selectGlobalPref :: line 334"  data: no]
Source File: chrome://browser/content/browser.js
Line: 1101

One last note: when I hit this, I was searching in the search bar for a PDF using the Google search engine, clicking on the PDF file (which loads in Preview.app), and then trying to type a new URL in the location bar and hit Enter/Return to load it; clicking on links within a page worked just fine, thankfully.  A restart of Firefox fixed all of the above issues, as everyone else notes.
(In reply to comment #0)
> Error: [Exception... "Component returned failure code: 0x80004005
> (NS_ERROR_FAILURE) [mozIStorageStatementWrapper.step]"  nsresult: "0x80004005
> (NS_ERROR_FAILURE)"  location: "JS frame ::
> file:///C:/Program%20Files/Mozilla%20Firefox/components/nsSearchService.js ::
> epsGetAttr :: line 2824"  data: no]
> Source File: nsSearchService.js, Line: 2824

This seems to be caused by an SQLite error that we don't catch in ConvertResultCode() (or the early return in mozStorageStatementWrapper::Step, but I think that's unlikely).

(In reply to comment #16)
> Error: [Exception... "Component returned failure code: 0x80520015
> (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageStatementWrapper.step]"  nsresult:
> "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame ::
> nsSearchService.js :: epsGetAttr :: line 2872"  data: no]
> Source File: nsSearchService.js, Line: 2872

While this one (NS_ERROR_FILE_ACCESS_DENIED) seems to map to SQLITE_PERM or SQLITE_CANTOPEN.

Only the first case seems to be caused by low disk space. If the file written to disk is corrupted, I'm not sure how much the search service can do to protect itself. Perhaps we should turn this bug into "protect against SQLite failures in the search service" and file another bug on "SQLite doesn't deal very well in low-disk-space situations".

Shawn, do you know of any existing issues with SQLite and low-disk-space?
There's been interesting issues with SQLite (or at least our wrapper of it) when it comes to non-standard disk operations (see Bug 398912 for another example).

We are supposed to handle disk IO here:
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/storage/src/mozStorageAsyncIO.cpp&rev=1.20#1537

However, I also know we completely ignore that for windows (see http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/storage/src/mozStorageAsyncIO.cpp&rev=1.20#1436).

I hope that information helps - if not, ask questions.
Target Milestone: Firefox 3 M10 → Firefox 3 M11
Priority: P2 → P5
Keywords: dataloss
Priority: P5 → P3
Target Milestone: Firefox 3 Mx → Firefox 3 M11
Would lsof output during an episoe--posted elsewhere and only linked to here--help?
Not a regression from Firefox 2, not blocking although we really want to get a fix for this in mozStorage ASAP.
Flags: wanted-firefox3+
Flags: blocking-firefox3-
Flags: blocking-firefox3+
(In reply to comment #20)
> Not a regression from Firefox 2, not blocking although we really want to get a
> fix for this in mozStorage ASAP.

Are we planning on re-enabling asynch sqlite?  Since bug 408914 landed, I haven't run into this corruption bug, and I used to hit it quite often >5 times per day...
No, we are not planning on re-enabling it.
Blocks: 406450
No longer blocks: 406450
Depends on: 406450
Assignee: gavin.sharp → nobody
Priority: P3 → --
Target Milestone: Firefox 3 beta3 → ---
bug 406450 was a places.sqlite version of the same problem. however, maybe comment #21 is on to something, because i haven't seen that failure in a long time, and have definitely hit my hard drive limits.
(In reply to comment #24)
> bug 406450 was a places.sqlite version of the same problem. however, maybe
> comment #21 is on to something, because i haven't seen that failure in a long
> time, and have definitely hit my hard drive limits.

so this can be closed WFM or dup of bug 408914?
Severity: normal → critical
This issue can no longer occur in Firefox 14 and later - bug 699856 replaced our use of SQLite with a simple JSON file.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: