781993 - Sanity-check maximum key and value lengths in memcache backend

Status: VERIFIED FIXED

(Cloud Services Graveyard :: Server: Sync, defect)

Description

[Ryan Kelly [:rfkelly]]

Memcache imposes maximum key length and maximum data size limits, and I'm pretty sure couchbase as similar limits.  We should have some sanity-checking in the memcache backend to be sure we don't go over them, or at least to ensure that things break safely if we do.  :rfkelly to audit this in current setup.

I am hoping that the underlying memcache client library would report this as an error rather than, say, silently truncating any data...

Comment 1

[Ryan Kelly [:rfkelly]]

:atoll reports that couchbase sets the following limits:

Max key length: 250 bytes
Max value size: 20 Mbytes

Comment 2

[Ryan Kelly [:rfkelly]]

Quick test:  pylibmc handles this properly, umemcache passes through the underlying server errors and then gets confused about the state of the connection.  So we need to add an extra level of sanity-checking on top of umemcache.

Comment 3

[Ryan Kelly [:rfkelly]]

This will need to happen inside mozsvc for sync2.0:

  https://github.com/mozilla-services/mozservices/pull/26

Comment 4

[Tarek Ziadé (:tarek)]

FWIW ultramecache had a hardcoded 1MB limit for the item size, and I made it configurable a while ago at https://github.com/esnme/ultramemcache/commit/1ac1a0e02c78710fa99899d86481cb8a02e986a9

You should get a 'clean' error with that backend at least for the value.

Comment 5

[Ryan Kelly [:rfkelly]]

old pr is old, but landed in mozsvc

Comment 6

[James Bonacci [:jbonacci]]

Quick verification in code.