123929 - (profile-corrupt) [meta] Prevent and repair profile corruption (tracking bug)

Status: RESOLVED INVALID

(Toolkit :: Startup and Profile System, defect, P3)

Description

[Andrew Hagen]

This is a tracking bug for profile corruption. There are various responses to
profile corruption, from the proactive (fixing bugs that allow profile
corruption to occur) to the reactive (fixing existing profile corruption before
it has deleterious effect on the operations of Mozilla).

Please make any changes to this bug that are necessary, including adding to or
subtracting from the dependency list.

Comment 1

[Andrew Hagen]

Need to add bookmark and history corruption bugs.

Comment 2

[Ashley Bischoff (blog at handcoding.com)]

Adding bug 118404. Of course, that one's fixed, but it was nonetheless due to
profile weirdness.

Comment 3

[Andrew Hagen]

Search terms to find bugs include:   prefs.js    profile corruption   corrupt  
 profile data   user.js   localstore.rdf.

Bug 98476 has a milestone of 0.9.9 and could help solve many of these problems.

Comment 4

[Brian Nesse (gone)]

Bug 98476 is in. The basic necessary functionality is found in the files
nsSaveSaveFile.cpp and .h. I placed these files in the prefs library as I really
had no better suggestion. They can easily be moved to a more common location if
others wish to make use of this functionality.

Comment 5

[Alexandru Savulov]

bug 105636 is not profile related (will be dup-ed!)

Comment 6

[Andrew Hagen]

taking

Comment 7

[Jim Crumley]

Adding bug 154054, a bug dealing with profile corruptions after crashes on
Solaris.  May be a dupe, but I didn't see it.

Comment 8

[Serge Gautherie (:sgautherie)]

After reading comment 3,
You may want to add bug 166455 comment 6 (and 5),
and may be bug 134576 !?

Comment 9

[Prognathous]

Users who encounter bookmarks loss (way too common) can't find this bug and this
in turn leads to many duplicates. I would like to open a tracking bug
specifically for this issue - somthing like "Meta bug for tracking Boomarks
dataloss bugs (Bookmarks gone, lost, corrupted, wiped)"

Are there any objections?

Prog.

Comment 10

[Prognathous]

Ok, I just opened Bug 203343 for specifically tracking Bookmarks dataloss bugs.

I would be happy to move all related bugs from the "depends"-list here, to the
new bug, but unfortunately I don't have "editbug" permissions. 

Prog.

Comment 11

[Paul Bensel]

I found Prog's meta, completely missed this one. I've commented it, but I just
had two incidents of losing all bookmarks, and I've got to say he's absolutely
on track with the amount of end user fury when everything goes away, with no
warning, and no recovery method.

You might also consider restore points instead of rolling backups with only two
stages back (when I at least open my browser, I might not NOTICE the bookmarks
are gone on a given session if I'm using history autocompletion/going to a
specific site, which means I could easily overwrite a 'two revisions old'
bookmark file unless it only writes with discrete changes.

Comment repeated from 203343:
I am being hit by this bug, or one very similar. In the last day, I had one
abnormal shutdown. I believe the browser was open. I'm not sure of the exact
chain of events, unfortunately, only that when I now open my browser it goes to
the correct homepage, but ALL bookmarks are gone.

I did some websearching, added a few links into a new folder, and again tonight,
I open the browser and all bookmarks are gone. Hopefully I have a usable backup
on my laptop (barring changes in the last few weeks) but this is an absolutely
miserable bug if it's as triggerable as having windows crash while you have any
mozilla files open.

I'm using:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3) Gecko/20030312

And I'm very, very cranky.

Comment 12

[Andrew Hagen]

Join the club. Do not post off-topic material on this bug. Do not cross-post,
either.

Comment 13

[Stewart Gordon]

Bug 190136 was duped to bug 192425, which is already listed.  Removing redundant
depend.

Comment 14

[Andrew Hagen]

I don't have time to keep this up to date anymore. If someone wants to maintain
it, please take it over. Thanks.

Comment 15

[Oscar Manuel Gómez Senovilla]

I'm not sure what's the curren purpose of this bug. Anyway, I can confirm that
this still happens in mozilla 1.7.8, when the disk is full, at least the
bookmarks get lost, and the bookmarks.html is 0 size.

Comment 16

[Worcester12345]

This bug should block bug 315312

Comment 17

[Matthias Versen [:Matti]]

*** Bug 332340 has been marked as a duplicate of this bug. ***

Comment 18

[edward ash]

I would wonder WHY a RECOVER button hasnt been installed...SOMEPLACE..
reinstall bookmarks, history, settings, and the rest...
Or, arnt they ALL backedup someplace?

Comment 19

[Jürgen 'JiHa' Harter]

bug 230494 deals with corrupt IMAP and/or news messagebases when downloading messages for offline use.

Comment 20

[Ben Bucksch (:BenB)]

Adding dependencies from dup tracker bug 193638.

Comment 21

[Andrew Hagen]

This bug report is over seven years old. Today much of the profile data is managed by the new Storage API, which I believe is also known as mozStorage. The backend is a SQLite database. https://developer.mozilla.org/En/Storage We have a track record of years of much greater profile stability, based on anecdotal experience. 

The new SQLite system was tracked in the following bug reports. 

Bug 314553 - Bookmarks
Bug 374945 - Places (history, bookmarks)
Bug 288040 - signons3.txt (Password Manager) (recent addition)
Bug 230933 - Cookies
Bug 380250 - Download Manager's RDF backend

The following profile data files have not moved to the new system. Extension manager backend storage (bug 449585), cache, prefs.js, some CSS files, some XML files, some rdf files, compreg.dat, secmod.db, key3.db, xpc.mfl, xul.mfl, some BAK files (for backup), and some JSON files. Other than extension manager backend storage and cache, the rest should probably be left as they are. 

With SQLite, we still have bugs, even profile data corruption bugs, like bug 360729. Overall, however, we have a system that is much more protective of data. There is even built-in protection against data corruption. https://wiki.mozilla.org/Storage:DetectingCorruption and bug 431558. 

When I originally filed this bug, the situation was not altogether pleasant for profile data. That situation has significantly changed for the better thanks to the good work of the developers, particularly in the implementations of SQLite. 

I'm not certain what should be done with this bug report. It has become cluttered. I am leaning toward the following course. First, I may resolve this bug report, and then I will create a new bug report that is more relevant to today to address current profile data concerns. If you have any comments on this plan of action, please post them here or just e-mail them to me. Thanks.

Comment 22

[Andrew Hagen]

In light of bug 319196, (current status: open) (customized toolbar resets to default due to localstore.rdf corruption), I've decided to keep this bug open for now. I don't know if the contents of localstore.rdf should be migrated to "localstore.sqlite" but in any case, that bug is open, so this bug stays open as well.

Comment 23

[chris hofmann]

possible profile corruption going on as part of bug 519356.  maybe the reactive fixes can help to mitigate that crash and the people hitting that crash can help to isolate and area of profile coruption that affects about 1000 people per day.

Comment 24

[MarjaE]

bug 302087 - "Inadequate warning before Profile Manager deletes non-Mozilla files" - this can also accidentally delete profiles

Comment 25

[Dave Townsend [:mossop]]

I don't think this meta bug serves a useful purpose at this point.