Closed Bug 493291 Opened 15 years ago Closed 13 years ago

places.sqlite-nnn.corrupt created in a loop downgrading to betas/nightlies before May 13

Categories

(Firefox :: Bookmarks & History, defect)

Product:
Firefox
Component:
Bookmarks & History
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: tuukka.tolvanen, Unassigned)

References

Details

(Keywords: regression)

Attachments

(1 file)

Terminal copy/paste from many corrupted DB
6.94 KB, text/plain
Details 
linux, trunk. this was probably mostly the 20090505, although the one from today might / triggered things, too. I got

 16M 2009-05-15 23:04 places.sqlite
 16M 2009-05-15 23:04 places.sqlite.corrupt
 16M 2009-05-15 23:05 places.sqlite-4.corrupt
...
 16M 2009-05-15 23:05 places.sqlite-34.corrupt
 16M 2009-05-15 23:06 places.sqlite-35.corrupt
...
 16M 2009-05-15 23:06 places.sqlite-76.corrupt
 16M 2009-05-15 23:07 places.sqlite-79.corrupt
...
 16M 2009-05-15 23:07 places.sqlite-96.corrupt
 16M 2009-05-15 23:13 places.sqlite-99.corrupt
...
 16M 2009-05-15 23:13 places.sqlite-140.corrupt
 16M 2009-05-15 23:14 places.sqlite-143.corrupt
...
 16M 2009-05-15 23:14 places.sqlite-190.corrupt
 16M 2009-05-15 23:15 places.sqlite-191.corrupt
...
 16M 2009-05-15 23:15 places.sqlite-204.corrupt
 16M 2009-05-15 23:16 places.sqlite-208.corrupt
...
 16M 2009-05-15 23:16 places.sqlite-217.corrupt
   0 2009-05-15 23:16 places.sqlite-223.corrupt
   0 2009-05-15 23:16 places.sqlite-222.corrupt
 11M 2009-05-15 23:16 places.sqlite-221.corrupt
 16M 2009-05-15 23:16 places.sqlite-220.corrupt
 16M 2009-05-15 23:16 places.sqlite-219.corrupt
 16M 2009-05-15 23:16 places.sqlite-218.corrupt
   0 2009-05-15 23:16 places.sqlite-226.corrupt
...
   0 2009-05-15 23:16 places.sqlite-243.corrupt
   0 2009-05-15 23:17 places.sqlite-244.corrupt
...
   0 2009-05-15 23:17 places.sqlite-258.corrupt
   0 2009-05-15 23:18 places.sqlite-265.corrupt
...
   0 2009-05-15 23:18 places.sqlite-296.corrupt
   0 2009-05-15 23:19 places.sqlite-297.corrupt
...
   0 2009-05-15 23:19 places.sqlite-300.corrupt
   0 2009-05-15 23:20 places.sqlite-301.corrupt
...
   0 2009-05-15 23:20 places.sqlite-318.corrupt
512K 2009-05-15 23:20 places.sqlite-317.corrupt
   0 2009-05-15 23:20 places.sqlite-322.corrupt
...
   0 2009-05-15 23:20 places.sqlite-328.corrupt

I'll guesstimate the sizes dropped once I ran to <16M free disk after these had eaten >3G in 12min. most of these were generated while sitting in about:sessionrestore I think. related bugs like bug 404565 tend to be fairly old and closed.
dunno how I got there exactly -- there were a few normal restarts and a kill due to a hang, can't say if that would have been before or after these started showing up.
hmm. could I blame http://blog.mozilla.com/dolske/2009/05/13/happy-form-history-expiration-day/ for giving me some kill-hazardous freeziness?
	<dolske>	sp3000: seems unlikely, form data isn't in places.sqlite.

meh.
Similar to bug 492146.
We should probably block on at least figuring this out.  Reporter - are you on PPC or some other architecture?
Flags: blocking-firefox3.5?
this is x86.
local 2009-05-05-20Z bad
local 2009-05-15-20Z ok
m.o 2009-05-15-04-mozilla1.9.0 ok
m.o 2009-05-15-03-mozilla-1.9.1 ok
m.o 2009-05-15-03-mozilla-central ok

boringly enough, looks like this can't be triggered on later builds. 2009-05-05-20Z certainly worked fine enough for quite a few days, dunno what made it all frothy all of a sudden. the same places.sqlite given to another profile triggers this issue on that build, too.

so this then looks like wfm as such... though it might be useful to look into whether the error handling situation that ends up causing a disk-filling repeat copy of the db might still exist.
I just want to make sure we are clear - current nightlies of 1.9.1, 1.9.2, and 1.9.0 do not exhibit this problem?
at least we must ensure we don't create corrupts in loop, if anything is wrong we should only ONCE create a corrupt and then a new one.
did you move back from today's build to 2009-05-05 build? that is not allowed sadly.
so per comment 10, comment 7 is expected, see bug 492379.
we can't downgrade users automatically to builds before 2009-05-13.
It's not just Linux. Same thing happened here under WinXP. System drive filled up with 10.5 Gb worth of places.sqlite.corrupt files. Deleted them and all is well so far.
sdwilsh, that's right, the latest 1.9.0 / 1.9.1 / central nightlies appear to be ok. marco, yes, I did run a current dbg build before hopping back to opt from the 5th, that probably coincides with this appearing.

fwiw the files are created approx. a dozen for startup with no session to restore, 4/min idle and 3/site loaded or so, so I suppose roughly every time something tries to touch history.
OK, I just observed something that might be helpful.

I have been testing both FF 3.5b4 and 3.6a1 (Minefield). Until very recently, there has been no problem switching back and forth using the same profile. Each would check the extensions for compatibility, ask to be default browser, then run normally.

The difference now is that there appears to be an incompatibility between 3.5 and 3.6 WRT the way they handle history and bookmarks. These places.sqlite-xx.corrupt files form when attempting to run 3.5 with the same profile that has been used by 3.6.

Here is what I just observed using 3.5b4 Build 20090423204732:

Following launch, a places.sqlite-xx.corrupt file formed approximately every 15 seconds, almost like clockwork. Each file was 22.3 Mb. In a few minutes, until I terminated the program, 52 such files formed for a total of 1.16 Gb. The browser never did succeed in loading its bookmarks, although the rest of the profile appeared to have loaded normally. 

I then re-launched 3.6a1 (Minefield) Build 20090515044249. It launched normally (including bookmarks), formed no corrupt places.sqlite files, and now appears to be running normally. In addition, the excessive memory usage I had recently reported on Hendrix is now gone. At the moment, with 4 tabs open, memory usage is just 105,544 K.

If someone else can confirm this behavior, it might be wise to issue a warning to use separate profiles if testing both 3.5 and 3.6 on the same machine.
Morton, which 1.9.1 build?

you can move up and down between CURRENT 1.9.1 and trunk and with 3.0... not with any nightly (3.6 or 1.9.1) before May 13.
The one I'm using now is 1.9.2apre (3.6a1 Build 20090515044249). 

This is the first I heard that you can't move up and down between 3.5 and 3.6 -- because I was doing it regularly using the same profile and having no problems until this came up. May 13 sounds about right for when the trouble started.

Guess I'll just stick to the nightlies and stop using 3.5.
Changed my mind. 

Duplicated my profile so I can use separate copies for different builds. Now have both 1.9.1b4 (3.5b4 Build 20090423204732) and 1.9.2apre (3.6a1 Build 20090515044249) running normally on the same machine with no interference. Unless someone else experienced the places.sqlite-xx.corrupt file behavior without moving between builds with the same profile, perhaps this bug should be closed.
if you were using 1.9.1b4, yes you can't move between them, you could move to 1.9.1b5pre. It's a good idea to have separate profiles for different versions.

Removing blocking request since is a sadly expected issue. Just looking if we can avoid hitting similar issues in future.
Flags: blocking-firefox3.5?
Summary: places.sqlite-nnn.corrupt created in a loop → places.sqlite-nnn.corrupt created in a loop downgrading current trunk nughtlies
We need to relnote this though.
Keywords: relnote
Happens btw. on all platforms.
OS: Linux → All
Hardware: x86 → All
So what will happen with this bug? Sounds like a wontfix or is there any work planned?
Keywords: regression
bug 493374 has fixed the looping creation
Removing relnote as per comment 22; we will already have a similar relnote anyway as per bug 488966
Keywords: relnote
Summary: places.sqlite-nnn.corrupt created in a loop downgrading current trunk nughtlies → places.sqlite-nnn.corrupt created in a loop downgrading to betas/nightlies before May 13
Firefox 3.5rc2 and Firefox 3.5rc3 have both exhibited this for me.  It's really annoying when, after opening Firefox, my disk starts churning and apps start complaining about having no more disk space.  

I'm not sure whether most users will be able to diagnose this problem.  I can imagine the rest of my family updating to 3.5, experiencing this once, and being like "My computer's broken, I think I have a virus, etc." 

Would it be possible to not repeatedly create places.sqlite-166.corrupt instances when the first dozen or so have failed in a row, and instead pop-up a warning to the user that they have issues?  I don't know what my issues are yet, since things seem to be working Just Fine, except for running out of almost 2GB of disk space in a few seconds :| 

Cheers!
Right, I'm running the Linux version of Firefox 3.5rc3 on a recent Fedora 11 install.
Richard it seems like that you were using a build from a date before May 13th which also applies to Beta 4 after you have used this profile with any of the RC versions. Please check the date of those corrupt files. I believe they are a bit older.
Right, in fact 3.5b4 is still installed on this machine since F11 shipped it.  I probably ran that version instead at some point after running the RCs (normally PATH'd in my ~/.local).  The corrupted files were recent, but that's almost certainly what I did.  I'll be more careful in the future.

Thanks.
How should this bug be resolved now that 3.5 has shipped?  Wontfix?  Bug 493374 takes care of the looping places.sqlite.corrupt issue, and bug 492379 is the downgrade script.
i was just expecting for all dependencies to be fixed. Then this will be fixed since we have fixed the issue for future.
Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv
Component: Places → Bookmarks & History
QA Contact: places → bookmarks
Reasons for this to stay open are rare, the script is most likely not that useful and expensive. I'd rather consider history backups for downgrades or incompatibilities.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → WONTFIX
I'm not sure if I may post a comment here, as this bug's been closed. But I have this same "places.sqlite-<n>.corrupt" files problem, but the circumstances are different from any of the scenarios posted here.

- I have no "special" version, I have a regular version "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.17) Gecko/20110420 Firefox/3.6.17"
- I had never installed any "special" version of Firefox, nor did I switch back and forth between versions. I just had the regular updates.
- I have a freshly installed Firefox on a freshly installed WinXP--but with my old profile from a Firefox of the same version.
- As I wanted to backup my old profile before the new installation of Firefox, I found over 10GiB(!) of these files. I put them somewhere else, and after the first run of Firefox I found new ones.
- The files aren't created in a loop when Firefox is running. But one is created each time soon after Firefox has been started. As I started it two times until now, I currently have two of the files, one with no. 1, one w/o a number.
- My temp folders are all existing as listed in the system properties.
- At my old WinXP I frequently experienced crashes of my Firefox. Up to now there were no new crashes of it.
- My bookmarks always load(ed) successfully and completely without any obvious problem. Beside the frequent crashed, there were no obvious problems at all.
- I didn't even notice any problem of my bookmarks before I found these files accidentally in the profiles folder. Only my Firefox starts rather slowly, but this may be blamed to thousands of reasons.

I consider making a JSON backup of my bookmarks via Firefox menu and then shut down Firefox, move away any file that has a name like places.sqlite, restart Firefox and restore the bookmarks from the backup. Wonder if this could solve the problem. (But at first I have to make sure, it's ok to delete all "places" files.)

If this doesn't work, I will end up just deleting these redundant files every few days/weeks or so.

Hope this information is kind of helpful
Regarding the frequent crashes, did you send reports and if so could you post some report id taken from about:crashes? I suggest to do a deep virus/malware check (often crashes are caused by those) and uninstall any extension/plug-in you don't use regularly.

To fix the situation probably you should do that json restore thing, your db seems completely messed up. If then it should still happen, please file a new bug so we can figure out what's up.

Regarding the 10GB, I'm really sorry, I thought we had a bug to remove those entries but I can't find it, will file one and cc you.
For months and yeard I sent the crash reports, but several week ago I switched that off, because this costs always several seconds for Firefox to restart--and with quiet many crashes this adds up. I must have had hundreds of report ids, but obviously after the WinXP and Firefox new installation all this data has gone. But seems I accidently made a report yesterday, although I can't remember. Nevertheless, there is an report id:

5fd618a1-55c8-49b8-90dc-3340f3aa2f59

This has been kind of a crash as was usual formerly, Firefox now started with these crashes again. :-(

Maybe the crashes are coming from any add-on or any combination of them. And I have a lot of. I checked for viruses/malware repeatedly--nothing found. I also checked my set of add-ons several times for any to remove. But the ones I have, I also really need. (Many of them I just need to compensate for the "damages" the Firefox developers do to Firefox, when they remove more and more good parts and properties of Firefox. E.g. to upgrade to Firefox 4 I again have to wait for add-ons that restore all the useful behaviours of Firefox 3, that were killed by the developers once again.) There may also be a hardware problem with the network devices, since several programs that connect to the internet crash regularly. And last but not least, several times I relied on the buggy WinXP system recovery/restore mechanism, which might have raped my old system, and also might have caused a permanent damage to my Firefox profile.

I can live with the many crashes, although sometimes it's annoying, but I have Session Manager, so normally the biggest problem is one of patience, nothing else. Maybe, as soon as possible, I will make a new clean(er) installation of Firefox w/o reusing the (complete) old profile, but installing add-ons and preferences manually.

The 10GiB haven't been that terrible, as there were still over 30GiB of free space in that partition. Now I have a Batch script on the desktop with two simple "del" statements. This totally shrinks the problem practically to almost zero.

My old WinXP crashed because the hard disk got a malfunction. The firefox program folder was/is on that drive. Maybe I will be able to access it in the nearer future. So can you tell me, if the crash reports are stored in the program folder, or where else? Obviously they do not appear in my new Firefox installation, although connected to the old profile. So I guess, the reports are not stored in the profile.

Thanx
(In reply to comment #36)
> all this data has gone. But seems I accidently made a report yesterday,
> although I can't remember. Nevertheless, there is an report id:
> 
> 5fd618a1-55c8-49b8-90dc-3340f3aa2f59

This report has not been sent. You can open 'about:crashes' and send it now. Thanks.
My problem is fixed, now. I want to give a feedback, how I fixed it: I followed the advice to rebuilt the bookmarks from an HTML export. So I exported the bookmarks as HTML, deleted all files in my profile folder that hold bookmark data, and restored the bookmarks via the bookmark manager. This happend several weeks ago, and since then, there have never been any "places.sqlite-<n>.corrupt" files again in my profile folder.

Thanks for the good advices.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: