929039 - http file age - /buildjson/builds-4hr.js.gz is stale

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Phil Ringnalda (:philor)]

***** Nagios  *****

Notification Type: PROBLEM

Service: http file age - /buildjson/builds-4hr.js.gz
Host: builddata.pub.build.mozilla.org
Address: 63.245.215.57
State: CRITICAL

Date/Time: 10-21-2013 10:18:00

Additional Info:
HTTP CRITICAL: HTTP/1.1 200 OK - Last modified 0:11:35 ago - 738193 bytes in 0.057 second response time

All trees closed at 10:28.

Comment 1

[Justin Wood (:Callek)]

The underlying cause here was builders that the DB was pointing at was not present in the DB.

This itself was caused by :sheeri bringing buildbot1.db back into the pool as the slave and checksum verifying in the wrong direction.

:sheeri failed over back to buildbot2 as the slave db and is running consistency checks on buildbot1 again.

*this* bug is fixed.

Comment 2

[David Burns :automatedtester]

Callek,

What is the bug that details the work in comment 1 so we have references?

thanks!

Comment 3

[Justin Wood (:Callek)]

(In reply to David Burns :automatedtester from comment #2)
> Callek,
> 
> What is the bug that details the work in comment 1 so we have references?

not certain, but :sheeri would know for sure

Comment 4

[Sheeri Cabral [:sheeri]]

bug 928521