tm-amo01-webdev02 no longer replicating

RESOLVED FIXED

Status

mozilla.org Graveyard
Server Operations
--
major
RESOLVED FIXED
7 years ago
3 years ago

People

(Reporter: clouserw, Assigned: justdave)

Tracking

Details

(Reporter)

Description

7 years ago
tm-amo01-webdev01 is more out of date than usual.  Fligtar says we should be using tm-amo01-webdev02 instead, but it's not replicating either.  I didn't know -01 was old news, but whichever one we are using, please make it work again. thx. :)

Updated

7 years ago
Assignee: server-ops → tellis
This has been broken since Tuesday night's database work, and all of the daily reports that are run off of it have failed. Can it please be fixed tomorrow morning?
Severity: normal → major

Comment 2

7 years ago
I'm doing tm-amo01-webdev02, then.

Can one of you comment on the bug whether or not we need to do tm-amo01-webdev01? If it isn't necessary to rebuild it, I'd rather not.
We were told early this year (or last year?) to stop using webdev01 after it couldn't keep up because of performance problems. I haven't used it since then.

Comment 4

7 years ago
Got it. tm-amo01-webdev02 has been rebuilt and is currently up-to-date! Use it. Love it. Reopen this bug if either (1) you see problems with webdev02 or (2) you realise webdev01 needs also to be rebuilt.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Great, thanks!
(Reporter)

Comment 6

7 years ago
can we get rid of webdev01 if it's not going to have valid data?

Comment 7

7 years ago
I've sent Dave Miller a note about it. We may be reclaiming the machine as a spare, but it may have other responsibilities besides the database.
This stopped replicating again 2 days ago.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: tm-amo01-webdev01 no longer replicating → tm-amo01-webdev02 no longer replicating
I corrected the replication error and resumed it.  It's *WAY* behind, but making decent progress, so I'll let it catch up on its own instead of reloading it.  I expect it'll probably be caught up in 2 or 3 hours at the rate it's going.
Assignee: tellis → justdave
Status: REOPENED → RESOLVED
Last Resolved: 7 years ago7 years ago
Resolution: --- → FIXED
I just added a nagios check for this as well, so we'll get a better heads-up if this happens again.  Since it's not a production db server it'll only alert on IRC and won't actually page anyone, but with it alerting on IRC every half hour it's sure to still get attention eventually.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.