Disk - All on mozdefes1.private.scl3.mozilla.com is WARNING: DISK WARNING - free space: /data 459420 MB (10% inode=99%):

RESOLVED FIXED

Status

mozilla.org Graveyard
Server Operations: MOC
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: MOC Nagios API, Unassigned)

Tracking

Details

(Whiteboard: [id=nagios1.private.scl3.mozilla.com:371780], URL)

(Reporter)

Description

3 years ago
Automated alert report from nagios1.private.scl3.mozilla.com:

Hostname: mozdefes1.private.scl3.mozilla.com
Service:  Disk - All
State:    WARNING
Output:   DISK WARNING - free space: /data 459420 MB (10% inode=99%):

Runbook:  http://m.allizom.org/Disk+-+All
CC'ing :jeff - please prune some indexes. Thanks!
See Also: → bug 997560
(Reporter)

Comment 2

3 years ago
Automated alert acknowledgement: (ashish)Bug 1026839
Status: NEW → ASSIGNED
Status: ASSIGNED → NEW
Flags: needinfo?(jbryner)
Status: NEW → ASSIGNED
(Reporter)

Comment 3

3 years ago
Automated alert recovery:

Hostname: mozdefes1.private.scl3.mozilla.com
Service:  Disk - All
State:    OK
Output:   DISK OK
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Some of this is due to the timing of backups vs pruning. I'll see about re-arranging things.
Flags: needinfo?(jbryner)
Any update on this?
We are still getting alerts (flapping; alerts, recovers, alerts etc).
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Updated the timings of backup/purge. Some of this was due to sudden surge in errant data resulting in logging loads of requests. Fix is in progress and we are working new backup/purge/maintenance scripts, and estimates for additional disk.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.