991841 - [tracking] dev/stage: reduce database size for dev/stage instances

Status: RESOLVED FIXED

(Cloud Services :: Operations: Marketplace, task)

Description

[Jeremy Orem [:oremj]]

We need to figure out how to reduce the size of our dev/stage databases, so we can host more instances and bring them up faster.

Comment 1

[Wil Clouser [:clouserw]]

marketplace dbs are about to shrink like crazy when we remove the AMO stuff from them

Comment 2

[Jason Thomas [:jason]]

Can we do anything to reduce the size of amo database? At first look I think we can prune *_counts tables on dev and stage.

Comment 3

[Wil Clouser [:clouserw]]

yeah, I think there is precedent for that.  Can we reduce dev and not stage?  If you want to delete anything older than Jan 1 2014 in those tables on -dev, I'd say go for it.  CCing krupa so she knows.

Comment 4

[Jeremy Orem [:oremj]]

(In reply to Wil Clouser [:clouserw] from comment #1)
> marketplace dbs are about to shrink like crazy when we remove the AMO stuff
> from them

Are there any bugs open for removing data from the marketplace database?

Comment 5

[Wil Clouser [:clouserw]]

Yessir.

Bug 999123 - [AMO] Clean up the FS after guillotine
Bug 999125 - [Mkt] Clean up the FS after guillotine
Bug 999117 - [AMO] Clean up the database after guillotine
Bug 999119 - [Mkt] Clean up the database after guillotine

Comment 6

[Jeremy Orem [:oremj]]

I've truncated update_counts, download_counts and theme_user_counts in dev/stage.

Comment 8

[Jeremy Orem [:oremj]]

Wil, can we add a migration, so these are truncated everywhere?

Comment 9

[Rob Hudson [:robhudson]]

fwiw I just committed this which drops those tables (plus a bunch of others) b/c the code that uses them is now gone:

https://github.com/mozilla/zamboni/commit/a16cb6c3