892840 - Socorro stage failing QA data seems out of sync

Status: RESOLVED WORKSFORME

(Data & BI Services Team :: DB: MySQL, task)

Description

[Matt Pressman [:mpressman]]

[20:53:31] <stephend> dev is passing, where stage/staging is horrible wrong :-(

The last few days, alerts like this have come in:
socorro-processor1.stage.metrics.phx1.mozilla.com:Socorro Processors - procs is CRITICAL: PROCS CRITICAL: 2 processes with regex args processor_app

On Wednesday, this alert and exchange in #socorro-alerts
<nagios-phx1> Tue 10:38:57 PDT [175] sp-admin01.phx1.mozilla.com:Socorro Admin - cron_submitter-crash-reports.allizom.org.log is CRITICAL: FILE_AGE CRITICAL: /var/log/socorro/cron_submitter-crash-reports.allizom.org.log is 646 seconds old and 5202162 bytes (http://m.allizom.org/Socorro+Admin+-+cron_submitter-crash-reports.allizom.org.log)
[11:39:27] <laura> solarce: and that one reads from prod hbase and writes to staging

Comment 1

[Matt Pressman [:mpressman]]

Without fully knowing why stage doesn't have a full set of data, but with seeing issues regarding services responsible for sending data to stage, I have kicked off a full stage refresh.

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

Is this related to bug 892897?

Comment 3

[Stephen Donner [:stephend] Not actively reading bugmail]

This was indeed "fixed" by the refresh finishing.