749335 - Investigate what is using redis and if it would be easy to use something else

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Code Quality, defect, P1)

Description

[Wil Clouser [:clouserw]]

Redis doesn't know much about replication across data centers and it might just be easier to quit using it.  For the most part we use it for things that could use memcache instead.  The persistent stuff could live in the db easily enough I think.

Regarding this bug:  We should differentiate between reads and writes as well as the temporary and persistent clusters.  Let's make a list of everything that is using redis, what it could be replaced with, estimate how long it would take, and any other concerns when doing it.

Comment 1

[Wil Clouser [:clouserw]]

- Caches the FeaturedManager stuff in overly complicated code in apps/addons/utils.py.  Could be replaced with mysql with the normal memcache pass through.  Any additional caching in memcache would be fine.

- apps/perf/tasks.py uses it to keep track of perf numbers.  This can be deleted

- apps/zadmin/views.py and apps/compat/cron.py uses it to show compatibility of add-ons.  I guess redis stores this in a special format, but I suppose we could do this with mysql queries

- apps/compat uses it to print the info it gets from ^^ that.  

- apps/zadmin/models.py the ValidationJobTally.  Used for bulk validation reporting.  I think this could easily use memcache instead.

- apps/reviews/models.py uses it for rating graphs (can use memcache instead) and spam (can also use memcache I think)

- the blacklist uses it as a cache.  No reason not to use memcache

- jetpack repackaging uses redis to store log messages.  upgrading jetpacks is something I'd be comfortable using memcache for - the jobs usually don't take long enough to be worried about losing data

- we do some paypal caching in apps/market/tasks.py.  I'm not sure why it's using redis though.  Could just as easily use the db and have memcache caching like everything else

- maybe the one legitimate use of redis is in apps/amo/mail.py where we are using it as a fake email backend.  We should replace it with something, but it won't be as elegant.

Overall it seems like redis was used in a bunch of places as an experiment but it wasn't required.

Comment 2

[Rob Hudson [:robhudson]]

I had an uncommitted patch using Redis hashes that I changed to use namespaced keys for memcache. I committed this as a utility to help do that...
https://github.com/mozilla/zamboni/commit/7f2a4b16

Feel free to re-use if it's useful.

Comment 3

[Allen Short [:ashort]]

I've done most of this here:
https://github.com/washort/zamboni/tree/no-redis

the bits in addons.utils remain untouched so far. FeaturedManager.build is called by a cron job to do a bunch of db queries and build some sets in redis. FeaturedManager.featured_ids then does some set intersection lookups there. It's not clear to me whether this should be replaced by just doing some self-joins when featured_ids is requested, or if a table should be built in the db by the cron job. ReverseNameLookup faces a similar question. The information's all there, I just don't know what the best simplicity/performance tradeoff will look like.

Comment 4

[Wraithan (Chris McDonald) [:wraithan]]

I am continuing work on this here:
https://github.com/wraithan/zamboni/tree/no-redis

I have the FeaturedManager using MySQL and just caching (which it was caching before after querying redis so not a big change.) Next step is to get tests passing again, then work on ReverseNameLookup and anything else that is depending on redis.

Comment 5

[Wraithan (Chris McDonald) [:wraithan]]

https://github.com/mozilla/zamboni/commit/25c2ffff5151f6fcace1037137806a95c56941f8

This has most everything other than the ReverseNameLookup. It is merged in and working on getting the last bit done.

Comment 6

[Wraithan (Chris McDonald) [:wraithan]]

https://github.com/mozilla/zamboni/commit/9dba89f71505c0831534d965cb21ce3e759b7233

RNL.get() no longer uses redis and is behind the no-redis waffle. I just noticed I missed a spot in the performance stats. Need to get that and this issue will be closed

Comment 7

[Wraithan (Chris McDonald) [:wraithan]]

https://github.com/mozilla/zamboni/commit/19f9ab72a7908bc4142f1844813d44fbf9ac7012

This should be the last thing. Closing. Will open a ticket and assign it to myself to remove the waffles in a week or two.

Comment 8

[Wraithan (Chris McDonald) [:wraithan]]

reopening to keep track of other redis related stuff.

Comment 9

[Wraithan (Chris McDonald) [:wraithan]]

ES backend for cache-machine was a deadend, closing this as it is just tracking for two bugs one of which is in question at this point.