221843 - easily reproducible bookmarks corrupted and lost scenario

Status: RESOLVED FIXED

(SeaMonkey :: Bookmarks & History, defect)

Description

[Loren Rosen]

User-Agent:       / (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6a) Gecko/20031010
Build Identifier: / (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6a) Gecko/20031010

On my system at least the simple procedure below will cause the bookmarks to be
lost. It doesn't seem to match any of the bugs listed in meta-bug 203343
	

Reproducible: Always

Steps to Reproduce:
1. create and use a new profile
2. drag and drop the little mozilla icon (for the default "getting started with
mozilla" page) from the location box onto the personal toolbar
3. quit and re-start mozilla

Actual Results:  
when re-started the bookmark saved onto the toolbar, and all the default
bookmarks that come with a new profile will be gone. attempts to save anything
new onto the personal toolbar will just cause mozilla to beep. You can recover
bookmarks from a saved bookmark file, but this doesn't recover the personal toolbar.


running MacOS X 10.1.5

Comment 1

[Boris Zbarsky [:bzbarsky]]

Confirming this based on the sheer number of duplicates (which I'm all marking
dependent on this bug for now).  This seems to be the #2 issue users on OSX 10.1
are having (right after "crashes all the time").  Could we please try to avoid
shipping 1.6b (and then 1.6) with this bug?

Ben, you were saying you would look into this... have you had a chance to yet?

The steps to reproduce in this bug and bug 221115 both implicate bookmark D&D;
ccing Ian from bug 221115 because he said he's kept around a 10.1 install to
test with.  Some of the other bugs say the bookmark file just disappears;
perhaps it's a problem with the OSX nsIFile impl that only bites us on 10.1? 
darin, wasn't there something with moveTo() failing if the target already
existed?  When did you fix that, exactly?  This bug is supposedly still an issue
in 1.6a per bug 221037...

Comment 2

[Stefan [:stefanh]]

CC: ing myself - I'm on 10.1.5 and can repro this (2003102905).

Comment 3

[u99]

*** Bug 226903 has been marked as a duplicate of this bug. ***

Comment 4

[Jonathan]

Yes I've had this same problem in OS 10.1.5 on Mozilla 1.5 and 1.5.1. It seems
the bookmarks are erased completely after the first relaunch of the browser.

Comment 5

[Asa Dotzler [:asa]]

Tested with today's trunk build on OS X 10.1.5 and was able to reproduce with
the simple steps given in the initial comment. 

Here's what I saw (and was able to reproduce identicall on several tests).

I created a new profile and launched the browser. Then I opened the JS console
and the finder set to the profile folder to see if anything interesting could be
observed. I dragged the proxy icon from the addressfield and dropped it on the
personal toolbar. After that, the finder showed the bookmarks.html file
disappear and this message showed in the JS console:

Error: uncaught exception: [Exception... "Component returned failure code:
0x80520008 (NS_ERROR_FILE_ALREADY_EXISTS) [nsIRDFRemoteDataSource.Flush]" 
nsresult: "0x80520008 (NS_ERROR_FILE_ALREADY_EXISTS)"  location: "JS frame ::
chrome://communicator/content/bookmarks/bookmarks.js :: anonymous :: line 1499"
 data: no]

On one of my attempts to reproduce, I continued to use the browser and performed
a second drag and drop of a bookmark and I saw the bookmarks.html file restored
in the finder. It seems like every even numbered write attempt succeeds. I also
only see the javascript error on the failed writes. 

Further testing suggests that it's not just drag and drop (though I haven't
looked closely here so I could be wrong). I think I can reprooduce the lost
bookmarks file with other bookmark operations like adding a bookmark (though I
don't see the corresponding JS error). 

Comment 6

[Asa Dotzler [:asa]]

http://bugzilla.mozilla.org/show_bug.cgi?id=219738#c5 mentions that he gets a
successful bookmarks write (bookmarks.html exists after browser exit) if he
starts out with no bookmarks file, in his case the result of the file being
obliterated by this bug. I tested by manually deleting the bookmarks.html file
and then starting up.  In this case, the odd number writes succeed and the even
numbers fail. 

Comment 7

[[e-mail]]

Nice detective work, Asa! I don't have an OSX system but it might be helpful to
determine what happens if you export your bookmarks to the same file twice.

Comment 8

[Brendan Eich [:brendan]]

So if FSMoveRenameObjectUnicode can fail when the target exists, and as an
effect of that failure, remove the target, the OS is severely broken.  Is that
really the case?  A reduced testcase would tell.

A workaround would be to check target existence, move it aside if it's there,
rename the temporary bookmarks file to bookmarks.html, and only if that move
succeeds, remove the set-aside backup.  If the temp-to-final move fails, restore
from backup.  Not multi-process safe, but we don't care.  More important,
crashing may leave a backup but no bookmarks.html -- but the next run could cope.

/be

Comment 9

[Asa Dotzler [:asa]]

With the exception of bug 222339, which doesn't have enough information, I'm
pretty sure that all of these dependencies are indeed duplicates. 222339 may
very well be a dupe too.  All of these dependencies describe basically the same
thing, that the bookmarks file is lost, not corrupted, lost. 

Comment 10

[Asa Dotzler [:asa]]

OK! I think I got it. This looks like it was broken by bryner's checkin for bug
215089, an update to /mozilla/lib/mac/MoreFiles/MoreFilesX taking it from vesion
1.0 to version 1.0.1, and the symptoms on OS X 10.1.5 first appeared in the next
nightly Mac build 2003090903.  

Comment 11

[Asa Dotzler [:asa]]

Oops, that was darin's patch (an Apple file, actually) not bryner's. 

Comment 12

[Asa Dotzler [:asa]]

I can't say with any certainty, but if I had to guess, I'd say that this bug
impacts more people (all pre OS X 10.2 users?) than 215089 (users with home
directories on mounted NFS disks). If we can't get this fixed quickly, then we
should consider reverting the update to MoreFilesX and re-breaking users with
home dirs on NFS disks. 

Darin, Bryner, what do you guys think? Is this fixable on our end or by making
modifications to MoreFilesX? 

Comment 13

[Jonathan]

I would also like to note that Mozilla 1.4.1 runs fine (actually I'm using it
right now to type this) and I haven't had any trouble with my bookmarks on it.
It was just the switch from 1.4.1 to 1.5 that messed everything up for me.

Comment 14

[Patrick McDonough]

hi,
   this is _not_ new to mozilla 1.6*. I had this bookmarks-get-deleted problem
since mozilla 1.4-alpha with macosX 10.1.5 and higher on an old wallstreet,
_but_not_ with 10.2.*.

good luck
--pat

Comment 15

[Pascal Oberndörfer]

Mozilla 1.4.1 ran fine for me as well (MacOS X 10.1.5). The switch to 1.5
started the trouble. The very same problem occurs with Firebird 0.7.1 as well,
of course. Camino 0.7 as well as its nightly builds work fine though!

Comment 16

[Asa Dotzler [:asa]]

bryner, can you back out the revision to morefilesx since it doesn't look like
we're gonna have a fix for this for 1.6?

Comment 17

[DeWaun Simmons]

In my situation, this problem was solved by upgrading my Operating System to Mac
OS X 10.2+, or "Jaguar." Two bugs that the upgrade to "Jaguar" fixed was:
Bookmarks no longer disappear and downloads fully complete and are visible in
the download destination. I'm not sure if this is an OS X 10.1.5 (Puma) problem
or some code in Firebird that is broken. Either way, that is how I solved my
problems with Firebird.

Comment 18

[Brian Ryner (not reading)]

Attachment: patch

This works around a difference in behavior of MoreFiles between 10.1 and 10.2
(I'm hesitant to call it a bug).  If the file /foo/bar exists, and you call
FSMoveObject() on it, with a destination directory of /foo, 10.1 will return
dupFNErr.  10.2 and higher will return noErr.

This patch simply skips that step if we're not moving the file to a different
directory.  I tested on 10.1 and confirmed that the bookmarks problem is gone
(we were actually hitting the same problem with writing xpti.dat and
compreg.dat, which is bad).

I also fixed the dependencies for libxpcom.dylib so it will rebuild when
morefiles is changed.

Comment 19

[Darin Fisher]

Comment on attachment 137484 [details] [diff] [review]
patch

makes sense to me.  is there a way to file a bug against MoreFiles? ;-)

sr=darin

Comment 20

[Brendan Eich [:brendan]]

bryner: any Unix rename(2) like primitive would be handed "/foo/bar", "/foo/bar"
and no-op nicely.  Errors shouldn't be raised for situations where no harm could
be done (wasting cycles not considered harmful).  My 2 cents, shutting up now
(but I'll approve the patch as soon as ccarlen blesses it).

/be

Comment 21

[Conrad Carlen (not reading bugmail)]

Comment on attachment 137484 [details] [diff] [review]
patch

r=ccarlen

Comment 22

[Brendan Eich [:brendan]]

Comment on attachment 137484 [details] [diff] [review]
patch

I thought maybe you'd avoid the extra system call, and the race condition, by
just ignoring the "dup" error.	Is that not unambiguous?

Anyway, go ahead and get this or a better patch in for 1.6.

/be

Comment 23

[Brian Ryner (not reading)]

There's a race condition, to be sure, since there is no sort of filesystem lock
protecting the generation of the unique temporary filename and the renaming of
the file to that filename.  But ignoring the 'dup' error doesn't help there,
since you still wouldn't know whether you got the error because the file is
already in the target directory, or because some other process/thread raced with
you and got its copy of a file into the directory, using that temporary name,
before you did.  This seems to be a problem fundamental to MoreFiles, and
perhaps a reason to abandon it at some point (I have a hard time believing that
better system functions don't exist, as brendan mentions).

I think in the majority of cases this will be faster, as well, since the common
case for us seems to be renaming files into the same directory (and
FSCompareFSRefs is just going to be comparing a few bytes in memory, rather than
hitting the disk to see that the file is already there).

Comment 24

[Brian Ryner (not reading)]

Note that we _could_ probably avoid the temporary file altogether if the source
and target directories are the same, but this patch seems safest for 1.6.

Comment 25

[Brian Ryner (not reading)]

checked in.

Comment 26

[Stefan [:stefanh]]

*** Bug 223231 has been marked as a duplicate of this bug. ***

Comment 27

[Mark Lakata]

NOTE: This bug affects Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5)
Gecko/20031007 Firebird/0.7 as well. This is not just a OSX problem.

In a particular session, I could not move particular bookmarks around. When I
restarted Firebird, most (but not all) of the bookmarks were lost, and new
bookmarks could not be added. I deleted the bookmarks.html file. When I
restarted, I could add new bookmarks, but I could not drag any of them to the
Bookmarks Toolbar.  I do not have a work around to restore the functionality of
the Bookmark Toolbar.

In the meantime, I have found this workaround to back up the bookmarks.html file
nightly:

http://texturizer.net/firebird/extensions/#bookmarkbackup

Comment 28

[Stefan [:stefanh]]

*** Bug 228517 has been marked as a duplicate of this bug. ***

Comment 29

[Stefan [:stefanh]]

*** Bug 221037 has been marked as a duplicate of this bug. ***