Closed Bug 759957 Opened 12 years ago Closed 12 years ago

intranet2.db.phx1 needs to be reseeded from intranet1.db.phx1

Categories

(Data & BI Services Team :: DB: MySQL, task)

Product:
Data & BI Services Team
Component:
DB: MySQL
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mpressman, Assigned: mpressman)

Details

intranet2.db.phx1 was blocked in replication running an optimize on puppet_dashboard.resource_statuses for around 7 hours. Either an over-aggressive logrotate removed the logs necessary, or more likely some config change and a related daemon restart on intranet1.db.phx1 caused it's binlog naming to be changed and intranet2 no longer has state for the necessary binlogs and needs to be reseeded.
Assignee: server-ops-database → mpressman
intranet2.db.phx1 has been reseeded and is now replicating.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
For the same reason, the backup for intranet has to be re-seeded. In progress now:

on the backup server, killed the mysql instance for intranet, deleted the old db (but kept the conf and the master.info file) and then:

cd /data/intranet
nc -l 9999 | tar xfi -

and on intranet:

[root@intranet1.db.phx1 ~]# time innobackupex --stream=tar /var/lib/mysql | nc backup2.db.phx1.mozilla.com 9999
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
That took 77 minutes, now I did this on the backup server:

innobackupex --apply-log . --ibbackup xtrabackup_51


And applied the logs. I'm going to chown -R mysql:mysql . and get replication going again.
backup slave is replicating; closing.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Product: mozilla.org → Data & BI Services Team
You need to log in before you can comment on or make changes to this bug.