We don't actually get physically paged for the backups not running...for example: [13:31:49] <nagios-phx1>  backup1.db.phx1.mozilla.com:Database Backups is CRITICAL: CRITICAL: dev1.db.phx1 is not replicating, sumo is not replicating, addons:marketplace_mozilla_org sql 1.33 days, addons:mysql sql 1.33 days, addons:percona sql 1.33 days, addons:pfs2_mozilla_org sql 1.33 days, addons:plugins_mozilla_org sql 1.33 days, dev1.db.phx1:addons_dev sql 1.33 days, dev1.db.phx1:addons_forums_allizom_org sql 1.33 days, dev1.db.phx1: Please page us. backups are important! (there are 3 backup servers.....)
The 3 backup serves are: * backup1.db.phx1 * backup2.db.phx1 * backup1.db.scl3 This needed some cleanup. There were two services with the same check but with different names, hosts and options. I've consolidated them now. For clarity - "MySQL DB Backups" will page DBAs like all other DB checks. In addition, it will also alert if the check output differs within the same state .  So first alert for "CRITICAL: dev1.db.phx1 is not replicating", second alert for "CRITICAL: dev1.db.phx1 is not replicating, addons:marketplace_mozilla_org sql 1.33 days" but not if the output remains the same. Excellent? :)
Assignee: server-ops → ashish
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Mucho excelente! Not verified just yet, but I believe in your work .....
Status: RESOLVED → VERIFIED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.