101319 - save bookmarks only when bookmarks are updated.

Status: RESOLVED WONTFIX

(Toolkit :: Places, defect)

Description

[matteo porta]

From Bugzilla Helper:
User-Agent: Mozilla/4.76 [en] (X11; U; Linux 2.4.9 i686)
BuildID:    2001091712

I have to use Linux with the home mounted remotely using nfs.
there is a problem with mozilla: every time I close a window
it saves the bookmarks, and since the home isn't local this
saving is slow (and I have a big bookmarks file). When I try at
home, where the home is local, there's no problem.
I suggest to add an option to save the bookmarks only when the
last browser window is closed, or when the user chooses quit
from the menu. Instead when the user chooses close the bookmarks
aren't saved to disk. I know that i risk loosing bookmarks in
case the browser crashes, however I prefer to take this risk
rather than having to wait too much everytime i close a window.
Since this problem is rather unusual (i think) you can add an
hidden option in the configuration file, without a gui setting,
so this option isn't selected "by accident" (just like you have 
done for the popup window setting).
please add this option in the next release!


Reproducible: Always
Steps to Reproduce:
this problem is related to our linux setup,
to reproduce you have to come here in italy, obtain an account from our
university, bookmark a lot of sites (about 4000 entries) and then close the
browser :-)

Actual Results:  the window closes after a irritating long delay


Expected Results:  the browser window should close almost instantly

Comment 1

[Chris Lyon]

Related to Bug 77133?

Comment 2

[matteo porta]

yes, it's almost the same as 77133, but in that bug nobody proposed a solution.here i propose to delay the saving of bookmarks until nmozilla is shut down.since last visited informations is important, mozilla should keep them in memory, and write only before terminating.

Comment 3

[xyzzy]

Roaming access should have similar issues because of remote bookmark location.  
See bug 17048.

Also, suggest adding 'NFS' and 'performance' to subject line for greater 
visibility and to keep it distinct from bug 77133.

Comment 4

[Jesse Ruderman]

I'm against adding a "lose more data than we need to on crash" pref.  Let's fix 
the problem, which is that bookmarks.html is written to even when you haven't 
changed your bookmarks.

Comment 5

[matteo porta]

to Jesse Ruderman:
adding an option to save the bookmarks only when last browser window is closed
is simpler than fixing the problem like you want to. I know that really fixing
the problem is the correct solution, however I fear I'll have to wait many
months for a fix like that. instead my solution is much faster. implement this
as a quick ugly hack, then fix the real problem, if you have time. moreover I'm
sceptical about totally fixing the problem like you propose: you surely have to
save the last visited informations, so writing to the bookmarks file is
inevitable, and it is slow if done via nfs. at this time keeping the bookmarks
information in memory seems the best thing to me. instead what do you propose? 

Comment 6

[Josh Kirby]

Here's a little suggestion for a "fix": when the user closes a window, the first
thing we do is to hide that window (we make it invisible). So the user has the
feeling that the window has been instantly closed, and he thinks mozilla is very
fast... then, when the user can't see, we save the bookmarks as usual.
Obviously we must pay attention not to freeze the other browser windows: in this
case we would gain nothing if the user cannot interact with the remaining
windows while mozilla is saving the bookmarks.

Comment 7

[Marco Lamberto]

*** This bug has been confirmed by popular vote. ***

Comment 8

[Ben Goodger (use ben at mozilla dot org for email)]

Paul Chen is now taking Bookmarks bugs. For your convenience, you can filter 
email notifications caused by this by searching for 'ilikegoats'.

Comment 9

[Paul Chen]

Mass move Ben's bugs dumped on me marked future with p5 to get off my untriaged
radar. You can filter out this email by looking for "ironstomachaussie"

Comment 10

[Peter Trudelle]

mass-reassign bookmarks & open pref perf bugs from pchen to ben

Comment 11

[Andreas Kunz]

I think this is a duplicate of 131130 (or vice versa).

Comment 12

[Boris Zbarsky [:bzbarsky]]

*** Bug 131130 has been marked as a duplicate of this bug. ***

Comment 13

[Boris Zbarsky [:bzbarsky]]

Setting severity/milestone based on duplicate, copying relevant keywords.

Comment 14

[Blu3]

In my opinion, this isn't a good thing.  Instead of saving the bookmarks on
every window close like we do now, we should have a pref option for this.

( ) Save bookmarks on change
( ) Save bookmarks at exit of program
( ) Save bookmarks every [ ] minutes

Comments?

Comment 15

[Blu3]

Let me change that to:

( ) Save bookmarks on change, but not more frequently than every [ ] minutes
(default is immediate)
( ) Save bookmarks at exit of program

Comment 16

[Jesse Ruderman]

From talking to bz, it seems like there are three problems:

1. Bookmarks are saved on window close even if not changed.  Instead, they
should be saved when a change is made or several seconds later.  (Need a "dirty"
flag.)

2. Changing the "last visited" times should not make bookmarks "dirty", since
losing that information is not much dataloss and most users don't see the "last
visited" column anyway.  But bookmarks.html should be saved when you exit
Mozilla if the "last visited" times have changed. (Need a "dirty but no real
dataloss" flag.)

3. Bookmarks should be written asynchronously (don't freeze the UI while writing
bookmarks.html).

I don't think we need a pref for how often bookmarks.html is saved.  This bug
should depend on the three above because it will be wontfixed if the pref is not
necessary.

Comment 17

[Blu3]

I prefer immediate[1] update to disk, asynchronously of course.  Others who are
on slow links to network mounted disk etc, want to delay the saving of bookmarks
to reduce the impact.  Thus the prefs I suggested.  If you're on a dialup and
saving a rather large bookmarks file, that currently means moz will stall for a
measurable period of time.  Nonsense in my opinion.

Fixing the bug part of this is actually independant of my suggestion.  That
being said, the fix for this is to (a) make this asynchronous and (b) only save
to disk when the data has actually changed.

[1] built-in delay of a dozen seconds or so to merge updates etc into one write.

Comment 18

[Alois Treindl]

I think this bug should have higher priority.

I am on a 64 kbit link, and my bookmarks file is mounted via a NFS through
this link from a central server; browser etc is local.

On each close of a Mozilla or Netscape6 window, the bookmark file which contains
about 100 kbyte is rewritten through the slow NFS link. This implies wait times
of about 20 sec, for each window close.

because of advertising popups these window close operations are frequent, and
the browser is frozen during 20 seconds each time.

Netscape 4.x did not have this problem.

Bookmark files should be updated on change, but not otherwise. A thread/process
should be forked to do the update, so that the browser does not hang while it is
done.

Comment 19

[Paul S. Kilroy]

I just wanted to point out something that has bitten myself and a co-worker. We
run linux and rarley shut mozilla down cleanly. Mozilla gets shutdown uncleanly
by random reboots/crashes/etc. This causes a loss of all new bookmarks since
startup (sometimes weeks).

I like the additional options mentioned already; namely the ability to save
bookmarks as they are added.

I'd like to throw out another option as well. Always save when the bookmarks are
moddified, but save in a "log" format. I.e. write a log of changes as they
happen to the file (appending each time). Compress this log during shutdown (or
startup if the app did not shutdown cleanly). This is similar to how some log
based filesystems work.

Comment 20

[Andrei Boros]

I'd like to add the suggestion to
Save bookmarks upon Bookmark manager window close.
(Netscape 3&4 do this and I use this as a safeguard if I add important bookmarks
and the browser might crash later as it sometimes happens)

Comment 21

[Andrew Sterling Hanenkamp]

I want bookmarks to save on change immediately. Mozilla 1.1 isn't the most
stable app in the world and when it crashes and I lose the half-dozen BMs I just
entered, it's a little frustrating. Whether this is an option or hard-coded, I
don't care. I can definitely see how saving infrequently is also desireable in
some circumstances, so probably an option.

Comment 22

[R.K.Aa.]

*** Bug 182435 has been marked as a duplicate of this bug. ***

Comment 23

[Jürgen Weber]

This bug is very annoying for me as I work "open in new window" and "close
window" very often, and close takes some seconds now for my 500K bookmark file.
I have the impression that this bug did not exist in the versions before 1.2

Comment 24

[R.K.Aa.]

*** Bug 182472 has been marked as a duplicate of this bug. ***

Comment 25

[Peter Bergt]

I upgraded to rv:1.2/Gecko/20021126 (linux) and it's really annoying that every
time a window is closed the bookmark file is written, especially if this has a
size of 1.8MB and no changes have been made to it.
Before the upgrade I used rv:1.1 and it wasn't that annoying and unusable.

Comment 26

[Laurens Blankers]

I agree with Peter Bergt the performance of 1.2-release is a lot worse than that
of 1.1. I have logged 129 write and 621 read operations to bookmark* alone when
closing a window.

I think the bug owner should consider upgrading the priority to P3. At the very
least the target milestone should be changed.

Comment 27

[R.K.Aa.]

*** Bug 182619 has been marked as a duplicate of this bug. ***

Comment 28

[R.K.Aa.]

there may have been a recent regression; see bug 182842

Comment 29

[Andrei Boros]

Bookmarks MUST definitely be saved when Bookmark manager window is closed.

Bookmarks should be saved when a bookmark has been manually added.
Bookmarks should be saved when a bookmark has beed added by a control related to 
browser window/tab X and that window/tab is closed. 
(ie: bookmark currently displayed page by keyb, context menu;
ie: boorkmark link via context menu)

bookmarks might be saved at intervals, but in a separate, low priority thread.

Comment 30

[Andrew Schultz]

FYI: if you are seeing degraded performance here in 1.2 with respect to 1.1,
please help to figure out why in bug 182373.

Comment 31

[Christian Hamacher]

CC->self

Comment 32

[Wayne Shumaker]

I had a large bookmark file, 600kB, on win2k system and every little pop-up window I closed would take forever. A bookmark dirty flag would be very appropriate. Update less important information during idle times.

ws

Comment 33

[Boris Zbarsky [:bzbarsky]]

*** Bug 191641 has been marked as a duplicate of this bug. ***

Comment 34

[Gareth Randall]

[I've just filed a duplicate bug for this. An important aspect is this:]

The delay problem is made MUCH WORSE by the fact that someone insists on calling
fsync after EVERY 4096 bytes! When was the last time the world stopped when an
app wrote ~500K in a single write? Not often. An "strace" shows hundreds of
delay-causing fsync's after each lump.

write(24, "<!DOCTYPE NETSCAPE-Bookmark-file"..., 4096) = 4096
fsync(24)                               = 0
write(24, "AST_VISIT=\"970339720\" LAST_MODIF"..., 4096) = 4096
fsync(24)
...
(repeat > 100 times)


Someone just get rid of that fsync and the visible delay will go. The OS will
then finish the write in the background. You'll even be okay on NFS - the OS
will cache it for you.


(And to be honest isn't nearly 1.5 years for this bug getting a bit long?)

Comment 35

[Jo Hermans]

That fsync was supposed to be fixed in Mozilla 1.3a by bug 180516 (see bug
182373 comment 23 and bug 182373 comment 38).

Comment 36

[Martin Dougiamas]

I've never had a problem with this before, but just in the last few builds (eg
2003040808 on Windows XP) I"m getting several-second delays every time I close a
window.  I can see that Mozilla is saving a bookmark-1.html file to my remote
volume, which it then renames to replace the bookmark.html file.

Nothing's changed about my setup, and my bookmarks file is the same, at about
175K  ... is there a new regression caused by the bookmarks work being done
recently?

Comment 37

[Martin Dougiamas]

Seems fixed!  See bug 202477

Comment 38

[Jason M. Abels]

This bug is not fixed. While one of the problems is addressed in bug 202477,
that doesn't solve the base problem, that bookmarks.html saves on close but not
on update. This is a more general problem, in that as you change the bookmark
file, this isn't getting saved to disk, so crashes can be serious. Also, as the
original bug report, an opposite problem comes when a bookmark file that isn't
changed gets re-written every time the browser is closed.

In my opinion, the bookmark file should ONLY be changed when bookmarks
themselves are changed. Here are the times when I think the bookmarks.html file
should save.

1. When a bookmark is added/filed.
2. When the bookmark manager is closed.

I changed the summary line to indicate what the root problem is. Many of the
comments address this root problem anyway.

Comment 39

[Christian Flatscher]

Hi,

I suggest to add a feature to Mozilla / Firebird that allows to create backups
of files such as bookmarks.html, Personal toolbar, mail prefs, news prefs, etc.

How about adding a menu entry to the "Edit / Preferences..." menu underneath the
"Navigator" branch called "Settings backup"?

This should contain tick boxes of what somebody wants to backup, to where and
how often (time interval) this should happen. Also a restore feature should be
introduced with a selection option what and which backup should be restored.

This may solve some of the issues where an OS freezes or crashes unrecoverably.

--Christian

Comment 40

[zug_treno]

*** Bug 330072 has been marked as a duplicate of this bug. ***

Comment 41

[Cees T.]

https://bugzilla.mozilla.org/show_bug.cgi?id=251456#c13 mentions a fix. Shouldn't this bug block that one?

Comment 42

[zack pryor]

This bug seems to have been around for a while, yet every time I get my update of Mozilla/Firefox, I have to go back and create all new bookmarks, because they all erase every time.  I am tired of it.  someone should be able to fix it.


thanks,

zack

Comment 43

[Jesse Ruderman]

That's not this bug.  Maybe it's bug 319196.

Comment 44

[Phoenix]

Looks like still valid, even opening new windows causes writes to places.sqlite, and I think that this bug should be moved to Toolkit

Comment 45

[Marco Bonardo [:mak]]

we are already going to optimize writes when we find possible improvements there, that said, I don't think we need this bug for Places.