933455 - Archiving appears to be misconfigured on stage

Status: RESOLVED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Selena Deckelmann :selenamarie :selena]

Just noticed this today:

2013-10-31 00:02:31.023 GMT,,,11119,,526e9c3c.2b6f,16371,,2013-10-28 17:17:48 GMT,,0,LOG,00000,"archive command failed with exit code 1","The failed archive command was: /var/lib/pgsql/scripts/wal_archive ""pg_xlog/0000001D000015D9000000D4"" ""0000001D000015D9000000D4""",,,,,,,,""

I'm debugging another issue, and all these error messages are making it difficult to sort though. 

If we could get the archiving fixed before the database refresh this weekend, that would be helpful for future debugging.

Comment 1

[Matt Pressman [:mpressman]]

it looks like the param archive_mode is set to on. I've changed it back to off, but we'll need to restart the daemon to have the change take effect.

Comment 2

[Selena Deckelmann :selenamarie :selena]

(In reply to Matt Pressman [:mpressman] from comment #1)
> it looks like the param archive_mode is set to on. I've changed it back to
> off, but we'll need to restart the daemon to have the change take effect.

Great! Thanks, Matt. Is there a commit in puppet to reflect the change?

Comment 3

[Matt Pressman [:mpressman]]

This will be restarted at 5pm PT so as to not interrupt any work being done in the meantime. Additionally, I've confirmed the config files used for the refresh have the archive_mode param set to off so this will not come up again after the refresh.


This change was committed to puppet in revision 74696

Comment 4

[Matt Pressman [:mpressman]]

selenamarie said that a restart after 5pm PT would be acceptable. I've sent an email out notifying that I'll give it another 30 mins just in case there is any work this will affect. So, I'll restart at 5:30pm

Comment 5

[Matt Pressman [:mpressman]]

selenamarie and stephend gave the go-ahead to restart now. stage was restarted at 5:10pm PT

Comment 6

[Matt Pressman [:mpressman]]

With archive_mode set to off, there are no more "failed archive command was: /var/lib/pgsql/scripts/wal_archive" error messages