809457 - Unexpected slow-down when bookmarking a page after switching Sync on

Status: RESOLVED DUPLICATE of bug 697649

(Firefox :: Sync, defect)

Description

[David Bourguignon]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Build ID: 20121024073032

Steps to reproduce:

I subscribed to Firefox Sync recently.


Actual results:

After this, I noticed unexpected slow-downs when bookmarking pages (and selecting specific bookmark folders). The browser appeared to be frozen sometimes.


Expected results:

No slow-downs! My configuration is quite old, therefore I suspect than what I'm experiencing is only a side effect of the algorithm behind Sync. Are you performing sync each time a new bookmark is added? This would explain the issue, I think.

Comment 1

[David Bourguignon]

Since supporting non-top-notch computers is also part of Firefox' mission I think, a workaround would be simply to add a setup flag disabling "instant sync" and allowing only manual sync through the Tools > Sync now menu entry. In any case, thanks in advance for your help!

Comment 2

[Richard Newman [:rnewman]]

Yes, Sync does some work every time you add a bookmark. Some of this is synchronous IO, some is spinning the event loop, and some is just work. We're working on eliminating some of these things, but there will always be more work happening than if Sync were disabled.

A workaround you can use is to set the timing prefs in about:config to larger values:

  services.sync.scheduler.activeInterval
  services.sync.scheduler.immediateInterval

and bump the score threshold:

  services.sync.syncThreshold

Increasing the latter to more than 301 will cause a single bookmark addition to *not* be enough to provoke a sync. You can choose how many multiples of 300 you want to set it to!

Thanks for the report!

Comment 3

[David Bourguignon]

Thanks a lot Richard for the explanations. I will try the workarounds you suggested.

Another idea: what about giving the user the option to only sync its bookmarks when closing Firefox? In my humble opinion, this is the usual behavior people expect when using software applications.

Thanks in advance for any improvements in Firefox sync. Have a good week-end!

Comment 5

[Salvo]

I recently switched from xmarks to sync and noticed the same behaviour.

Comment 6

[Salvo]

Richard, can you give us recommended values for the parameters?

services.sync.scheduler.activeInterval
services.sync.scheduler.immediateInterval
services.sync.syncThreshold

Thanks a lot!

Comment 7

[Richard Newman [:rnewman]]

(In reply to Salvo from comment #6)
> Richard, can you give us recommended values for the parameters?

The recommended ones are the ones they are set to by default. I only recommend changing them if you want to make syncs occur less frequently. If you want to change them, you need to experiment to see which values make sense to you.


(In reply to David Bourguignon from comment #3)

> Another idea: what about giving the user the option to only sync its
> bookmarks when closing Firefox? In my humble opinion, this is the usual
> behavior people expect when using software applications.

Unfortunately, by the time we know the user is wanting to quit, it's a really bad time to do a whole bunch of synchronous disk and network I/O.

Part of the reason why we sync fairly frequently, and also sync after the user performs high-expectation actions like bookmarking, is to meet user expectations of data synchronization without having to do things like sync-on-quit.

Comment 8

[Salvo]

Dear Richard,
Maybe the behaviour suggested by David it's not "ideal" for the reason you explained. However my opinion is that, if I choose (for whatever reason) this option, I'm aware of what it mean (slowdown on close). This is the way Xmarks works, a FF extension which I used (I have to say I was quite satisfied).

I believe that Firefox Sync choose a right and good path (to work trasparently behind the scenes) but I think that if this lowers the usability of bookmarks this is a serious issue... 

FF should take account of people having poor internet connections or huge bookmark list. Therefore, either it's possible to find a way to get the process run more smoothly by default, either it should be possible to choose...

I believe that setting a good default _and_ giving user the power to choose is unbeatable, expecially if the latter doesnt' require a lot of work

Thanks

Comment 9

[Richard Newman [:rnewman]]

(In reply to Salvo from comment #8)

> Maybe the behaviour suggested by David it's not "ideal" for the reason you
> explained. However my opinion is that, if I choose (for whatever reason)
> this option, I'm aware of what it mean (slowdown on close).

Alas, this is not a common user attitude; if it was, we'd all have an easier time building software! But regardless, attempting to do on quit the kinds of things that Sync does can cause crashes or odd behavior, and generally speaking that's not acceptable in a product used by half a billion people. (Even what we do now at runtime causes weird stuff like Bug 697649.)

If you want to try sync-on-exit, there's an (unsupported) add-on to do it:

  https://github.com/rnewman/sync-on-exit/blob/master/dist/xpi/sync-on-exit.xpi

but if it causes crashes or database corruption, consider yourself warned!


> FF should take account of people having poor internet connections or huge
> bookmark list.

That's actually one reason why we try to do frequent incremental syncs during operation, rather than one big sync on exit. Better to do 100msec of work now than hang for several seconds on quit. And you get the benefit of having your data now, rather than after you quit!


> Therefore, either it's possible to find a way to get the
> process run more smoothly by default, either it should be possible to
> choose...

I hope you see that when one of the choices is "subject some percentage of users to crashes, hangs, or data corruption", it's not really a choice. And "smoothly" is very much what we're aiming for: take a look at the two dependencies for this bug. Time and resource constraints are the only reason this bug is still open.


> I believe that setting a good default _and_ giving user the power to choose
> is unbeatable, expecially if the latter doesnt' require a lot of work

A few things:

* If it exposes UI, it's a lot of work. And if it doesn't expose UI… well, you can already change stuff in about:config, or use an add-on. That's where Firefox hides settings that we don't expect many users to change.

* Multiple code paths are costly to maintain, and end up lowering overall software quality (because they can't all be tested adequately), so there are very solid engineering reasons for resisting configuration options.

* Adding options for every choice like this -- and there are thousands of choices like this in a product like Firefox -- leads to a baffling and hostile user experience. Bear in mind that *you* might care about when Sync runs, but another user cares about which bookmarks get synced, and another user cares about whether print settings are saved, and …. Before you know it, you have dozens of settings tabs with confusing checkboxes, and the product can neither be tested for all of the combinations, nor grow in any meaningful way.

Rather than working on making Sync configurable like this, I'd rather just eliminate the issue that's prompting you to want a choice. If Sync doesn't spin the event loop in the bookmarks observer, and thus you don't get unpleasant behavior when you bookmark a page, *everyone* benefits, and you'll be happy, right?

Comment 10

[David Bourguignon]

Thanks Richard and Salvo for this supplementary information. In fact, I do observe a slow-down when bookmarking on either an old Windows PC or a recent Linux box (of course, the slow-down on the latter is less noticeable than on the former, which can sometimes freeze Firefox for ~10 seconds).

I am one of those users who has been collecting bookmarks over the last ten years or so... And I use this info everyday, this is not simply archiving for the sake of archiving. To give you an idea, my current bookmark backup file (the JSON version) is roughly 1.6MB, which is big, but not that big nowadays, I think.

Maybe the instant bookmark syncing algorithm could be made more tolerant to larger bookmark lists? Thanks in advance for your help.

Comment 11

[Richard Newman [:rnewman]]

(In reply to David Bourguignon from comment #10)
> Thanks Richard and Salvo for this supplementary information. In fact, I do
> observe a slow-down when bookmarking on either an old Windows PC or a recent
> Linux box (of course, the slow-down on the latter is less noticeable than on
> the former, which can sometimes freeze Firefox for ~10 seconds).

That's unpleasant!

Freezing suggests a slow disk or some other blocking, rather than just Firefox doing more work.


> Maybe the instant bookmark syncing algorithm could be made more tolerant to
> larger bookmark lists? Thanks in advance for your help.

There are parts of bookmark sync that are slow (e.g., Bug 655722). Bug 814801 might fix that.

I don't think there's a really good shorter-term improvement we can make here for Sync as a whole. We could delay instant syncs (either reducing frequency, or having it happen some seconds after triggered), but you're still going to get slowdown at some point. We can't rely on sync-on-idle here.

If syncing is simply too expensive to happen in the background for you, then any kind of automatic triggering during browsing is going to cause pain.

The good news is that you can work around some of the pain for the moment, and we'll be working on improving this stuff over the first half of this year.

If I were you, I'd do the following:

* Turn on the Add-on Bar (View > Toolbars > Add-on Bar), and add the Sync icon to it. Clicking that icon will trigger a sync.

* Set services.sync.syncThreshold to something large, say 6001. That'll require 20 big changes before a sync occurs.

* Increase activeInterval, too. That'll make Sync run less frequently when you're browsing.

* Keep an eye on syncing behavior: if you switch to your other device and frequently find that recent changes aren't present, you might need to manually activate a sync, rather than waiting for Firefox to figure out that you're idle.

If the idle timer is working, you'll sometimes see Firefox stall when you return to the machine after leaving it running. There's no way to avoid that, I'm afraid; that's Sync trying to make sure that your data is up-to-date before you start browsing.

Comment 12

[David Bourguignon]

Thanks again Richard for your precious help. I will definitely try the solutions you mentioned.

Another info, in case it could help you track this issue more precisely: in my configuration, the maximum slow-down observed occurs when, after opening the bookmarks window, I move or cut-paste a bookmark from one place to another in my bookmark list.

This bookmark window slow-down is way more unpleasant than the simple slow-down occurring after bookmarking a page.

Comment 13

[Richard Newman [:rnewman]]

(In reply to David Bourguignon from comment #12)
> Thanks again Richard for your precious help. I will definitely try the
> solutions you mentioned.
> 
> Another info, in case it could help you track this issue more precisely: in
> my configuration, the maximum slow-down observed occurs when, after opening
> the bookmarks window, I move or cut-paste a bookmark from one place to
> another in my bookmark list.

Thanks, David. Will bear these steps in mind when assessing performance impacts.

Comment 14

[tfpsly]

The issue still exists in 25.0.1. I've had it since I activated Sync months ago. Each time I edit a bookmark, Sync will immediately try to update the server bookmarks, stalling the bookmarks window. It doesn't seem to be disk I/O related.

Setting services.sync.syncThreshold to 3000 does fix the issue: Sync stop trying synching each time I edit a bookmark, making the situation much more comfortable.

Ideal fix: wait 15 second after last change before synching; if another edit occurs, reset the timer back to 15 seconds.

Comment 15

[Richard Newman [:rnewman]]

(In reply to tfpsly from comment #14)

> Ideal fix: wait 15 second after last change before synching; if another edit
> occurs, reset the timer back to 15 seconds.

That doesn't help users who spend more than 15 seconds naming/categorizing a new bookmark (probably many of them).

It also delays the sync, to the point that you might have quit Firefox or put your machine to sleep before the sync, which is confounding to users. That undoes the Instant Sync feature.

The right fix is to not screw with the input just because you're syncing in the background.

I'm going to dupe this bug back to the older bug I just found.