506768 - Use relative bookmark positioning to not require depthindex sort (and mini records to update depth/index)

Status: VERIFIED FIXED

(Firefox :: Sync, defect, P1)

Description

[Ed Lee :Mardak]

Right now bookmarks require the server to give bookmarks in the right depthindex order so that parents and predecessors are correct. This also causes us to send the server a bunch of depth/index mini records for every affected sibling, and this can cause a lot of server writes for each bookmark.

We can switch to a relative positioning by tracking the parent and predecessor GUIDs of a particular bookmark, and this basically makes a linked-list structure.

The general idea is if an incoming bookmark doesn't have its parent/predecessor ready, it sits in a "waiting folder" and we add annotations to persist the relations.

Comment 1

[Ed Lee :Mardak]

relative bookmarks branch here:
http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/

Some interesting changesets so far..

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/e8095d816c90
Remove meta/mini records. Bye! 

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/f3344304f254
Store the parent GUID as an annotation if the item is missing a parent and share some parent lookup code. 

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/60617a65eb6e
Reparent orphans when creating the parent folder.

Comment 2

[Ed Lee :Mardak]

Bookmarks will be inserted after the predecessor, but the predecessor annotation isn't used yet to fix things up after-the-fact:

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/7f3df30d9648
Set the predecessorid when creating records for upload. 

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/fb42b4205805
Use the predecessorid to figure out where to put the item or save the predecessor as an annotation.

Comment 3

[Ed Lee :Mardak]

Repositioning a successor to after a predecessor was more complicated than I expected ;) Because it's repositioning a whole chain of followers! :p

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/86210dcdabc1
For items that have the correct parent, see if there's an item following it and reposition the chain of items to after the correct parented item. A chain of followers starts at the item with the predecessor annotation and goes until the end of the folder, an item that is looking for a predecessor, or ends up at where we started (the predecessor).

Comment 4

[Ed Lee :Mardak]

Attachment: super patch so far

Everything seems to work so far to recreate the bookmark structure on after a wipe local and sync. Same ordering of stuff if I switch to sort=index/newest/oldest. The recently bookmarked smart folder is different, but that's expected.

I'll check if things are working for updates and stuff in the morning.

Comment 5

[Ed Lee :Mardak]

Lots of interesting changes to get updates working, and that should be it for relative bookmarks!

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/623bba8283c4
When updating the position of an item, move all of its followers with it. Share chain moving logic used for fixing followers on create and updating positions. 

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/b059f2fc784c
Add the successor ids when adding/removing items as well as both old and new successors when moving. 

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/fb651af14fea
Attach followers for both created items and updated items (call from applyIncoming) when it's under the right parent. Share logic for attaching followers when creating a folder then fixing the reparented children and when creating/updating an item in the right parent.

Comment 6

[Ed Lee :Mardak]

Attachment: super patch

Comment 7

[Ed Lee :Mardak]

http://hg.mozilla.org/users/edward.lee_engineering.uiuc.edu/weave-relative/rev/e7cf228ec3d4
Add tests to make sure bookmarks are in the right position after creating/updating. 

Woohoo! tests :)

Comment 8

[Dan Mills [:thunder]]

Comment on attachment 395226 [details] [diff] [review]
super patch

looks great!
r=thunder

Comment 9

[Ed Lee :Mardak]

http://hg.mozilla.org/labs/weave/pushloghtml?fromchange=3eb68a7b4be8&tochange=0d9fefc7eaa4

Comment 10

[Tracy Walker [:tracy]]

moving manual test cases to moztrap: manual test case for bookmarking present