bugzilla.mozilla.org will be intermittently unavailable on Saturday, March 24th, from 16:00 until 20:00 UTC.


7 years ago
3 years ago


(Reporter: Cedric Knight, Unassigned)


(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)




7 years ago
Occasional situations can cause prefs.js to be corrupted or disappear, such as perhaps bug 597329.  In my case, I had a profile stored on a vfat (FAT32) partition, and was performing a folder operation when the OS crashed.  A Windows chkdsk removed an IMAP folder (no loss), and prefs.js, and unfortunately I had set up a few miscellaneous mail and news accounts since the last backup (and restoring the prefs.js on the same profile has caused Thunderbird hanfs).

Given that so much information that is time-consuming and tedious to recover is stored in a file that's rarely more than 100K, how about storing a copy of the last good prefs.js in the profile folder, and ideally recovering automatically if needed?  Bug 487368 and for Firefox during discussion of bug 364440, but not carried forward.
See dependency tree for meta bug 193638 for prefs.js corruption issues.
See dependency tree for meta bug 123929 for requests of protection from prefs.js corruption.
Blocks: 123929
I also experienced prefs.js loss due to crash of SeaMonkey or Thunderbird a few times during my Win 98/Win-2K era. And, I saw next once on NTFS/Win-XP last year.
  Close SeaMonkey window. During Seamonkey is executing shutdown process,
  just after close of SeaMonkey window, I pushed down power-key long time.
  (i.e. I forced immediate power off of PC, instead of suspend of Win.    )
  (As I set config.trim_on_minimize=true for Sm, shutdown of SM takes long.)
  Upon next power on, CHKDSK process run, and prefs.js was removed from
  Tb's profile directory once, and prefs.js was recovered.
It indicates that prefs.js was generated by rename, but it was incomplete.

This is very normal phenomenon, because current prefs.js writing is like next;
  (i)   write data to prefs-1.js
  (ii)  delete prefs.js
  (iii) rename prefs-1.js to prefs.js  
  Above process is called "safe file writing" by developers of Mozilla.
Power failure during (iii) can cause prefs.js loss or corruption. 

On Linux, cache is aggressively used for file writing, then prefs.js corruption frequently occurred during step (i), so FLUSH was issued after (i) to avoid corruption.
However, because (iii) consists of many internal steps of OS's action on file system, power failure, crash of OS etc. during (iii) may produce loss of prefs.js.
Further, characteristics of some HDDs for PC like next may produce prefs.js corruption upon power failure.
  Some HDDs for PC return "write complete" when data is written to HDS's
  write cache, before data is physically written to hard disk platter.

As seen in meta bug 193638(see dependency tree with "Show Resolved"), "Safe file writing" drastically reduced prefs.js loss/corruption problem.
However, no one can perfectly protect from prefs.js loss/corruption due to power failure during (iii).

In Firefox, main problem due to loss/corruption of file was loss of bookmarks.html. So, after transfer of bookmark data to SQLite DB, loss/corruption of file is not so big issue in Firefox.

"ideally recovering automatically" is better but I think backup of prefs.js is sufficient for simple PC's application.
(A) Manual backup by script/batch.
  Delete prefs.js.bkup-10; 
  Do MM=1 to 9; NN=MM+1; Rename prefs.js.bkup-MM prefs.js.bkup-NN
  Copy prefs.js prefs.js.bkup-1;
  Start thunderbird.exe 
(B) Automatic backup by Thunderbird himself.
  See bug 246722 comment #2, please.
  The reason of WONTFIX of the bug was "there is no need for the backups
  anymore because Bug 250092(=the safe file writing) was fixed".

Primitive "ideally recovering automatically" example.
(C) Safer file writing than current.
(C-1) Write process
  indicate prefs-new.js is BAD (for example, write 0x00 at offset=0)
  write data to prefs-new.js, with 0x00 as first byte
  indicate prefs-new.js is GOOD (for example, replace 0x00 by correct byte)
  rename prefs.js to prefs-old.js
  rename prefs-new.js to prefs.js
  delete prefs-old.js
(C-2) Restart process
  If prefs.js only, continue.
  If prefs-new.js is GOOD, rename prefs-new.js prefs.js, delete prefs-old.js.
  If prefs-old.js only, rename prefs-old.js prefs.js.

cedric@gn.apc.org, what do you think about best solution for us?

Comment 3

7 years ago
Thanks for the detailed explanation.  I didn't realise there had been so much work on it already.  On the machine I'm using now (the same as with the problems described above), Firefox has a prefs-1.js over a month old as has Thunderbird, presumably from a crash.  How does that affect the safe writing?

I would favour solution (C), as it includes the case I describe, where for whatever reason prefs.js is accidentally deleted or corrupted through a file system error outside Thunderbird.  Provided the delay between (ii) and (iii) is only momentary, I would not think that power loss should cause many failures, but again (c) could recover if there is no prefs.js found.  

Losing a single session's changes is not as bad as losing the file altogether, so I was initially thinking:
On exit or prefs save:
(i) copy existing prefs.js to prefs-1.js (possibly repeating for 1..3, possibly depending on timestamp)
(ii) overwrite prefs.js
On launch:
(i) attempt to load pref.js
(ii) if file is absent or any syntax error, attempt to load prefs-1.js (or higher number if present)

Given the existing process and suggestions, however, how about something like:
On exit or prefs save:
(i) move prefs-old.js (if present) to prefs-older.js (warn on any disk error)
(ii) copy prefs.js to prefs-old.js (and ensure file handles are closed and then copy timestamp)
continue as at present:
(iii) write out to prefs-new.js (with some marker at the end that confirms it is complete).  (truncate it if it is already present)
(iv) delete prefs.js
(v) move prefs-new.js to prefs.js

On launch:
(i) attempt to load pref.js
(ii) if file is absent or any syntax error or final marker is not present, attempt to load prefs-old.js in the same way

Comment 4

6 years ago
My GOD! This bug (in various forms for different Mozilla products, including Thunderbird) dates back to 2004 at least! What is so freaking hard about simply backing up a file before making a change to it? Any freshman computer science student knows how to encapsulate writes to a file so that it is backed up safely before making changes to it, and how to restore it if and when necessary! How many annoyed users does it take before this bug gets fixed? This should NOT be an enhancement request, this is a serious problem that can lead to major data loss and therefore it is critical that it gets fixed ASAP. Please upgrade the priority on this to Major or Critical.


5 years ago
Severity: enhancement → major
Priority: -- → P3
(In reply to Marc Chamberlin from comment #4)
> My GOD! (snip)
> this is a serious problem that can lead to major data loss
> and therefore it is critical that it gets fixed ASAP.

Sigh... Shouting by looking string of "data loss"...

Try following.
(1) While Tb is running, "open prefs.js with write exclusive mode" by other
    program such as text editor(except notepad.exe because notepad.exe doesn't
(2) Do something on Tb
    => upon each trial of prefs.js update by Tb, prefs-N.js is created.
This is because "prefs.js writing" currently consists of;
  (a) write data to prefs-N.js
  (b) rename prefs-N.js to prefs.js, with requesting to OS "replace if exists"
  Note: Old one was "open prefs.js with replace mode", "write data", "close".
And, on MS Win, (b) is requested to MS Win via single API call.
So, potential data loss of prefs.js currently exists only in the single API call execution by OS.
The "single API call" is pretty popular one and normal request on a file.

Because "rename file A to B with replacing A by B if A already exists by OS" consists of many updates of many directory entries on HDD, problem such as power failure can cause inconsistency among the many many directory entries on HDD.
So, "absolute data loss of a file" can occur when both "file related directory data for A on HDD" and "file related directory data for B" are lost or corrupted at mid of OS's updating of many directory entries on HDD.
The "absolute data loss of a file" can occur on any file if such problem occurs on the file. 

Because probability of such problem is never ZERO or negative value, is surely greater than 1/(+Infinity), problem of "potential data loss of prefs.js" surely exists.
And, because (i) (importance of prefs.js)*(probability of such problem on prefs.js) is far greater than (ii) (importance of other than prefs.js)*(probability of such problem on a non-prefs.js file), I also think (i) is never negligible.

However, Thunderbird is not a software used on Space Shuttle.
I believe priority of this bug is far low than other important bugs.
This bug is request of "backup by Tb" in case of "prefs.js loss" by Tb or someone, in case of "logical data corruption" by Tb or someone and so on. i.e. Enhancement request.
Changigin back to Enhancement.
Severity: major → enhancement
OS: Linux → All
Priority: P3 → P5
Hardware: x86 → All

Comment 7

3 years ago
As a small note I must mention, that while Firefox is really secured by using Mozilla Sync, Thunderbird is really at the mercy of the system running it.

Not that it happens that frequently, but once a year I encounter a "missing prefs.js" error - be it Thunderbird or Firefox. And the worst thing is there is no way of finding out what caused the file loss. In a corporate environment you can count on the users using frequently their apps, and reporting every problem almost immediately. But home users tend to sit on the problem ("it started with this error message for some months, but I thought it will go away with time" approach) and by the time I get in front of the computer it is impossible to recover.

I would suggest the easiest and almost perfect solution would be - as mentioned above - to rename and keep the old prefs.js file, possibly deleting everything older than the last 10.
The automatic detection and recovery of a lost prefs.js situation is not needed - the machine would need some attention anyway if it keeps losing files.
You need to log in before you can comment on or make changes to this bug.