790749 - db consistency checks for addons, support

Status: RESOLVED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Sheeri Cabral [:sheeri]]

Just like bug 774162 for Bugzilla, let's put consistency checks on addons and sumo for q4.

Comment 1

[Sheeri Cabral [:sheeri]]

Currently running a checksum on addons with:

 pt-table-checksum  --databases addons_mozilla_org --lock-wait-timeout=50 --user checksum --ask-pass --quiet --chunk-size-limit=0  --no-check-plan --replicate percona.checksums --no-check-replication-filters

Comment 2

[Sheeri Cabral [:sheeri]]

The checksum completed - VAMO is fine, but it will need to only check the following tables - 

 select * from percona.checksums  where this_crc!=master_crc and tbl in ('applications','addons','incompatible_versions','applications_versions','tmp','files','appversions','versions');

The checksum is perfect on all the slaves except addons5, which has some problems with synced_collections. I will work on fixing that, but as soon as I do, we can put this into a cron script - the checksum took 28 minutes, and there was no replication lag (once we fixed VAMO not having the right table structures).

Comment 3

[Sheeri Cabral [:sheeri]]

looks like it's count values that are wrong on synced_collections on addons5 only. Will resync from addons7 now.

Comment 4

[Sheeri Cabral [:sheeri]]

Added dependent bug 802283 for nagios checking addons.

Comment 5

[Sheeri Cabral [:sheeri]]

putting the cron in puppet for addons7 now. Once the dependent bug 802283 is resolved, that's half this goal met!

Comment 6

[Sheeri Cabral [:sheeri]]

Consistency check for support has been run, there are no warnings, and the only difference in in the mysql database, so we're good to go on having a nagios check turned on for this. Once dependent bug 802793 is resolved, this goal can be CLOSED.

Comment 7

[Sheeri Cabral [:sheeri]]

Well, this goal has been met. Whee!