433915 - Trigger a sync event on local changes

Status: RESOLVED FIXED

(Cloud Services :: General, defect, P1)

Description

[Dan Mills [:thunder]]

We should monitor local data (bookmarks, history, cookies, etc) for changes, and try to be smart about syncing changes to the server.

Each engine may need to have their own flavor of the algorithm, but we should in general 1) use the idle timer to wait until there changes have stopped for a bit, and 2) have some limit per-hour or such to prevent flooding the server and establish a worst-case baseline.

We may want to bubble up these events from the engines to the sync service, so it can decide to sync multiple engines at the same time or not.

Comment 1

[Myk Melez [:myk] [@mykmelez]]

Is there a particular use case for this functionality?  It's problematic to design an API like this implementing a consumer of it, as we're apt to design something that turns out not to be useful for the use cases we later come up with.

Comment 2

[Dan Mills [:thunder]]

The idea is to sync 'on demand', to propagate changes quickly, but not spend unnecessary cycles syncing when nothing has changed.

We have largely done this already with the 'trigger' classes, which this bug should've depended on, but instead are under bug 434816 (which is to change the service to use the engine API this bug describes).

As such, either we can reparent all the trigger bugs (ugh), or we can close this bug as fixed (since all the triggers are checked in, except afaik the password engine one).

Comment 3

[Myk Melez [:myk] [@mykmelez]]

Sounds like closing this bug as fixed is the less-ughy path.

Comment 4

[Myk Melez [:myk] [@mykmelez]]

Per bug 434816, scheduled sync is now an engine-specific function of the amount of data changed and the last time we synced, so engines that want Weave to sync their data "immediately" can set their tracker score to 100, whereupon Weave will sync their data within one minute.

It's trivial to make that faster if it isn't "immediate" enough.  For example, we could set the SCHEDULED_SYNC_INTERVAL constant in service.js to 12 seconds and the THRESHOLD_DECREMENT_STEP to 1, which could cause Weave to run scheduled sync every 12 seconds (but still take 20 minutes to reduce the thresholds for syncing down to their minimum value).

One potential downside to a faster scheduled sync is if on-demand tracker score generation turns out to be expensive.  For example, the tab tracker calculates its score on demand by querying the session store for the current state (which I think causes the store to enumerate open windows and tabs) and then comparing that to the current snapshot.

If that turned out to be expensive, then running scheduled sync every 12 seconds could be problematic.  At this point, however, there's no indication that's the case.

Note that the fix for bug 434816 does not implement use of the idle service or a per-hour limit as described in comment 0, but presumably it implements enough of the intention of this bug to make further enhancements be of lower priority and schedulable to a later milestone, so I'm marking this bug fixed.  Let's open new bugs on further enhancements and issues as they crop up.