504788 - Handle different weave versions by wiping, upgrading, updating

Status: RESOLVED FIXED

(Firefox :: Sync, defect)

Description

[Ed Lee :Mardak]

After landing 504346, thunder and I discussed some more about versioning...


Current code: There's 3 versions floating around.. the weave version, storage version, and the remote storage version on the server.

The remote version is set when the server is wiped. When weave upgrades it will try to sync everything and any records that are different will be put on the server.

For a given storage version, nothing in the records can change. This means even if the new client uploads a new record, e.g., a bookmark as BookmarkQuery instead of Bookmark, the storage version would need to be bumped because otherwise, an old client with that same storage version wouldn't be able to process that record.


Proposal: We can avoid unnecessarily wipe the server in some cases by tracking 3 versions.. the weave version, compatible version, and newest version to access the server.

Weave version is the add-on version as before, and newest version to access the server is set when a newer weave version talks to the server. Compatible version is the earliest version that the current build of the add-on is able to read.

When a newest version of weave talks to the server, it will update the "newest version" value on the server, so any older client knows that it needs to update because potentially it can't access the data.

We can avoid server wipes by marking a weave version compatible with an older version, so it can leave most of the records untouched and only upload the new records that the new version uses.


Just to permute various versions to make sure I didn't miss anything..

Weave version
| Compatible version
| | Newest version
| | |
1 1 1 same version
1 1 2 upgrade local
1 1 3 upgrade local
1 2 1 invalid
1 2 2 invalid
1 2 3 invalid
1 3 1 invalid
1 3 2 invalid
1 3 3 invalid
2 1 1 compatible, update remote -> 2
2 1 2 compatible, same version
2 1 3 upgrade local
2 2 1 wipe server -> 2
2 2 2 same version
2 2 3 upgrade local
2 3 1 invalid
2 3 2 invalid
2 3 3 invalid
3 1 1 compatible, update remote -> 3
3 1 2 compatible, update remote -> 3
3 1 3 compatible, same version
3 2 1 wipe server -> 3
3 2 2 compatible, update remote -> 3
3 2 3 compatible, same version
3 3 1 wipe server -> 3
3 3 2 wipe server -> 3
3 3 3 same version


The main difference from what was implemented before bug 504346 is that now we'll need to make sure to update the "newest version" on upgrade.

Comment 1

[Dan Mills [:thunder]]

Great write-up.  We should get this in for 0.5.

Comment 2

[Ed Lee :Mardak]

The compatible version is always going to be less than or equal to the current weave number, so there's 5 possible relations for the "newest version" value stored on the server.

Assuming a given weave client is at version 4 and compatible to 2, five example "newest version" relations are below and what the client should do:

 v C - W ^
 1 2 3 4 5 <-- "newest version" value
 | | | | \- newest is newer, upgrade weave
 | | | \- is newest, nothing to do
 | | \- is compatible, update latest version
 | \- is almost incompatible, update latest version
 \- incompatible, wipe server

Comment 3

[Ed Lee :Mardak]

http://hg.mozilla.org/labs/weave/rev/9f7b464c9199
Store the most newest Weave version on the server and update it if necessary on each remoteSetup. Make a concept of a compatible version where this version of Weave can read but might change records with its full sync.