567882 - Memcached server errors on support-stage-new

Status: RESOLVED DUPLICATE of bug 568175

(mozilla.org Graveyard :: Server Operations, task)

Description

[James Socol [:jsocol, :james]]

In the past 15 minutes, I've seen 5 of this error (and it's the first time I ever recall seeing it, but we had occasional, vaguer issues with the old memcache client):

======
  File "/data/virtualenvs/kitsune/src/django-pylibmc/django_pylibmc/memcached.py", line 57, in add
    return self._cache.add(smart_str(key), value, self._get_memcache_timeout(timeout))

ServerError: error 10 from memcached_add: SERVER ERROR
======

It's not the most verbose or helpful error message--the memcached docs don't actually list the error codes or their meanings. I'm hoping someone can help us figure out what's going on here.

Comment 1

[James Socol [:jsocol, :james]]

Jeff suggests flushing memcached after the switch to pylibmc but also says he has "no idea" if it will make the error go away.

Comment 2

[James Socol [:jsocol, :james]]

This is happening quite a lot and making stage-new pretty unusable now.

Comment 3

[chizu]

I flushed the cache and it seems to have helped. Are you still seeing the error at all?

Comment 4

[James Socol [:jsocol, :james]]

(In reply to comment #3)
> I flushed the cache and it seems to have helped. Are you still seeing the error
> at all?

I can create new threads, again. Looks like that helped.

Comment 5

[Stephen Donner [:stephend] Not actively reading bugmail]

Not sure why I'm one of the few (only?) still seeing this, but I'm constantly triggering 500 Internal Service Errors that generate the memcached tracebacks.

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

I'm still seeing this, and it's blocking me from testing -> blocker (mrz said we can if it truly blocks testing, and it does).

Comment 7

[Jeff Balogh (:jbalogh)]

This is an application bug.  The app is trying to store more data than memcached allows.