215089 - nsLocalFileOSX::MoveTo fails if no "Temporary Items" folder on volume [was: Bookmarks.html consistantly lost]

Status: RESOLVED FIXED

(SeaMonkey :: Bookmarks & History, defect)

Description

[Len Laughridge]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.5b) Gecko/20030804
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.5b) Gecko/20030804

Mozilla deletes the bookmarks.html file resulting in data loss. 

Bookmarking a page does not cause data to be written to the bookmarks.html file
right away.  It appears to be cached elsewhere.  After bookmarking a page,
Mozilla will delete bookmarks.html when the browser window is closed, and will
not re-write the file until another browser window is opened.  Quitting Mozilla
when the bookmarks.html file is not present results in data loss.



Reproducible: Always

Steps to Reproduce:
1. Assume a bookmarks.html file exists in *.slt
2. Launch Mozilla - check that bookmarks.html exists
3. Add a bookmark.
4. Verify that bookmarks.html exists, and is the same size as before (the
bookmark has not been written to the file).
5. Close the browser window
6. Verify that bookmarks.html no longer exists.
7. Check the Bookmarks menu - the entry still exists (cached?)
8. If you quit Mozilla now, the bookmarks will be lost!
9. Create a new browser window.
10. Verify that bookmarks.html has now been written, AND is larger than before
reflecting the additional data from (3) above.
11. Repeat steps (4) to (6) to observe the same behaviour again.
12. If you quit Mozilla WITH the bookmarks.html file present, you have about a
2/3 chance that it will not be immediately deleted upon subsequent launch.

Actual Results:  
Consistant and repeatable loss of bookmarks.html

Expected Results:  
Mozilla should have handled bookmarks.html in a robust manner.  The app should
never be in a situation where bookmarks data is unwritten and only exists in cache. 

Both existing and new, clean profiles have been tested, with the same results.

This has been occuring with clean installs of 10.2.[3|4|5|6] on G4 notebooks
(12") and workstations (2x1.25GHz).  It has happened with every build of mozilla
from 1.4b --> the 3 August nightly.  

This occurs only on Mac OS X builds where the users $HOME is on a mounted NFS
volume.  It does not occur with profiles on HFS+ volumes.  It does not occur
with Solaris/SPARC or Linux/x86 builds with profiles on NFS volumes.  I'm not
saying that this is related to any specific Mozilla bug involving NFS. I'm
simply reporting facts that I can observe and reproduce.

Comment 1

[Asa Dotzler [:asa]]

unable to reproduce with latest trunk builds on OS X

Comment 2

[elijahlofgren]

I have the same problem. All recently added bookmarks are lost when mozilla is
shut down improperly. Any way to fix this? 

Comment 3

[Len Laughridge]

If 'trunk builds' == the 20 August nightly, then it is absolutely reproducible,
still.  Sorry.

Comment 4

[Rolf Braun]

This also happens with profiles in AFS (we symlink ~/Library into AFS), both
netscape 7.1 and moz trunk.

This is really annoying, and it didn't happen with CFM versions of moz.

Comment 5

[Darin Fisher]

hmm... i have no mac, and am therefore not able to test this bug out, but the
following code snipet from nsBookmarksService.cpp confuses me:

NS_IMETHODIMP nsBookmarksService::Observe(...)
{
...
  if (!nsCRT::strcmp(aTopic, "profile-before-change"))
  {
    // The profile has not changed yet.
    rv = Flush();                                                              
                                                                   
    if (!nsCRT::strcmp(someData, NS_LITERAL_STRING("shutdown-cleanse").get()))
    {
      nsCOMPtr<nsIFile> bookmarksFile;
                                                                               
                                                        rv =
GetBookmarksFile(getter_AddRefs(bookmarksFile));
                                                                               
                                                        if (bookmarksFile)
      {
        bookmarksFile->Remove(PR_FALSE);
      }
    }
  }
...

why in the world do we remove the bookmarks file here?

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

From what I see, shutdown-cleanse only occurs in unusual situations, in
particular in response to a user requester in CProfileManager::DoLogout() in
embedding/browser/powerplant/source/CProfileManager.cpp

   * "shutdown-persist"
   *   The user is logging out and whatever data the observer stores
   *   for the current profile should be released from memory and
   *   saved to disk.
   *
   * "shutdown-cleanse"
   *   The user is logging out and whatever data the observer stores
   *   for the current profile should be released from memory and
   *   deleted from disk.

Without knowing exactly the content of the requester, I'm not sure why it would
be invoked anyways.  It certainly could use some more commenting.  Not sure if
this is the cause, but it's possible.

I suspect this isn't the issue, and it's instead one with the logic to write new
bookmarks files atomically, but it's worth checking.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

I got an email from a user (Mike Breeze) with Win2000 and Netscape 7.1 that they
have this problem too (or if not, a similar problem).

OS->All
Hardware->All

Comment 8

[Hermann Schwab]

re comment 5
I don´t know, if this is relevant to this bug, but I made some observations:
1. switch to another profile or create a new one, open bookmarkmanager, and you
see as name: Bookmarks for [old profilename]
This is easily reproducible, don´t know, if a bug exists.

2. I´m using nightlies with my default profile, including mail. My default
profile  has NOT the standard Mozilla/Netscape name. Normally each of my older
Mozillas is started with it´s own profile.
I started Mozilla 1.0.2 with my default profile, tested a bug, then ended it.
I started Mozilla Nightly, I think from a shortcut with profile name supplied,
but I had a fresh profile, no bookmarks in the toolbar, no mail.
While still running, I opened windows explorer, and made a copy of my default
profile.
I think, at the second restart of mozilla my old profile was in use again,
no corrections necessary.
I didn´t file a bug, as I´ve got to investigate this better.

I never lost my bookmarks, since using 0.9.x

Comment 9

[Hermann Schwab]

part 2 of my previous post is irrelevant, the shortcut had option -P"Profile-Name"
instead of -P "Profile-Name", there was a space missing, so the profile wasn´t
changed. Sorry for the spam.

Comment 10

[Darin Fisher]

using Len's machine, i'm able to reproduce this bug (thanks Len!)... now to
figure this out...

Comment 11

[Darin Fisher]

i cannot seem to repro this bug using my debug seamonkey build under linux.

Comment 12

[Darin Fisher]

ok, i learned some more about what the problem is here:

at shutdown, nsBookmarksService::Flush is getting called, and WriteBookmarks is
getting called.  the temp bookmarks file (bookmarks-1.html) is getting created;
however, when we go to move bookmarks-1.html to bookmarks.html, we fail.  we
fail deep inside MoreFilesX.c in FSMoveRenameObjectUnicode (line 1389) with the
error that there exists no "Temporary Items" folder on the specified volume. 
simply creating a folder in the user's home directory called "Temporary Items"
makes this bug go away.

now, i have no idea why we need to depend on the existence of the "Temporary
Items" folder in order to simply move files around the file system.  maybe we
should "fix" this bug by having nsLocalFileOSX.cpp catch this error and do the
fallback copy method of moving/renaming files.  or maybe we can use some other
method of moving/renaming files under OSX.

conrad: do you have any suggestions?  (btw: this bug results from folks storing
their profiles on a NFS drive, and this particular bug is OSX only.)

rjesup: since this bug has been isolated as a Mac OSX specific problem, i think
we should really file a new bug for the WIN32 version.

Comment 13

[Darin Fisher]

ok, more info...

  FindFolders(..., kTemporaryFolder, ...);

is failing with the error -5000 (afpAccessDenied), which might just be another
way of saying that the directory doesn't exist.

might be wierd/out-of-place to catch this error in nsLocalFileOSX.cpp:MoveTo. 
maybe we should just always failover to the copy case?

Comment 14

[Darin Fisher]

ok, more info:

looks like newer versions of MoreFiles actually fixes this bug.  in the
changelog for MoreFilesX.c:

 <4>	 8/22/02	JL		[3016251] 
   Changed FSMoveRenameObjectUnicode to not use
   the Temporary folder because it isn't available on
   NFS volumes.

does anyone know where to get a more recent copy of MoreFiles?

Comment 15

[Darin Fisher]

ok, thanks to bryner for pointing me at the latest copy of MoreFilesX on apple's
website.  i'm going to verify that it fixes this bug.

Comment 16

[Darin Fisher]

Attachment: v1 patch : update to latest version of MoreFilesX

ok, this patch fixes the problem (without requiring folks to create a
"Temporary Items" folder).

Comment 17

[Conrad Carlen (not reading bugmail)]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

Good find. r=ccarlen

Comment 18

[Darin Fisher]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

requesting approval for 1.5 final.  patch is simply to update our copy of
MoreFilesX.

Comment 19

[Asa Dotzler [:asa]]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

a=asa (on behalf of drivers) for checkin to Mozilla 1.5

Comment 20

[Len Laughridge]

On behalf of myself, the (now 20 and counting) users at my company, and
enterprise users everywhere:

Thank You All Very Much, especially Darin.

Now, on to 166369 (hopefully already fixed by this same patch), and 201291!

I'll provide remote access to a G4 2x1.25 for anyone willing and able to get
these knocked out before 1.5 final.

Thanks again,

Len
IT Director
Kitchen & Associates Architectural Services

Comment 21

[Darin Fisher]

fixed-on-trunk (for mozilla 1.5)

Comment 22

[Darin Fisher]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

we probably want this on the 1.4 branch as well.

Comment 23

[Asa Dotzler [:asa]]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

too late for 1.4.1. if you'd like to see this in the next 1.4.x release please
set the approval1.4.2? flag.

Comment 24

[Matthias Versen [:Matti]]

*** Bug 219243 has been marked as a duplicate of this bug. ***

Comment 25

[Mike Kaply [:mkaply]]

Comment on attachment 131085 [details] [diff] [review]
v1 patch : update to latest version of MoreFilesX

a=mkaply for 1.4.2

Comment 26

[Darin Fisher]

fixed1.4.2