Closed Bug 1124705 Opened 10 years ago Closed 10 years ago

Trees closed due to builds-4hr.js.gz Problem

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

Product:
Infrastructure & Operations Graveyard
Component:
CIDuty
Platform:
x86
All
Type:
task
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: cbook, Unassigned)

Details

nagios alert: ***** Nagios ***** Notification Type: PROBLEM Service: http file age - /buildjson/builds-4hr.js.gz Host: builddata.pub.build.mozilla.org Address: 63.245.215.57 State: CRITICAL Date/Time: 01-22-2015 07:45:06 Additional Info: HTTP CRITICAL: HTTP/1.1 200 OK - Last modified 0:10:29 ago - 8191 bytes in 0.013 second response time
We did an intentional buildbot db failover this morning, which cause in-progress scripts to timeout. I've cleaned up after them and we should be running again within 15min.
db failover is bug 1121516.
(In reply to Chris Cooper [:coop] from comment #2) > db failover is bug 1121516. Hasn't happened yet apparently, just slowdown in db replication. Sheeri flushed some tables, and builds-4hr is up-to-date again.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Trees are mostly reopened. Inbound has some new failures that need addressing before it can be reopened, though.
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.