745059 - memcached backend fails to increment total size

Status: VERIFIED FIXED

(Cloud Services Graveyard :: Server: Sync, defect)

Description

[Ryan Kelly [:rfkelly]]

Attachment: patch to correctly increment storage size in memcached

The MemcachedSQLStorage backend fails to increment the cached "total size" measurement when the user uploads an item, due to a quirk of memcached/pylibmc.

The "user_id:size" is currently stored into memcached as a float.  When storing a new item, the MemcachedSQLStorage backend uses the "incr" command to increase this by the size of the new item.  Unfortunately, "incr" silently fails to work on floats and the size does not get updated.

Attached patch fixes the problem by explicitly coercing to an integer.

This could well explain the strange behaviour witnessed in Bug 687108.  There is a small value stored in memcache from an earlier quota viewing, and it is not updated as the user syncs lots and lots of stuff.  This leads to a growing discrepancy between the information in /info/quota (which is calculated from memcached) and /info/collection_usage (which is calculated from the database).

The eventual call to /info/collection_usage forces an update of the cached data and results in subsequent /info/quota calls returning the correct(ish) value.

Comment 1

[Toby Elliott [:telliott]]

Comment on attachment 614653 [details] [diff] [review]
patch to correctly increment storage size in memcached

This sort of thing is not endearing me to python :P

Comment 2

[Ryan Kelly [:rfkelly]]

https://github.com/mozilla-services/server-syncstorage/commit/b3a5f073d070a3625ca8877a98dec3dbcfe8366f

Comment 3

[James Bonacci [:jbonacci]]

This is no longer happening.
Tested on qa1.

Comment 4

[Ryan Kelly [:rfkelly]]

Backported to Sync1.1 trunk: http://hg.mozilla.org/services/server-storage/rev/505649a33610

Comment 5

[James Bonacci [:jbonacci]]

And, again, Verified as fixed with 4/16 make build and make test.

Comment 6

[Ryan Kelly [:rfkelly]]

Backported to 1.10 release branch: http://hg.mozilla.org/services/server-storage/rev/406ea814d079