Closed Bug 1025413 Opened 10 years ago Closed 10 years ago

Disk - All on mozdefes4.private.scl3.mozilla.com is WARNING: DISK WARNING - free space: /data 460578 MB (10% inode=99%):

Categories

(mozilla.org Graveyard :: Server Operations: MOC, task)

Product:
mozilla.org Graveyard
Component:
Server Operations: MOC
Platform:
Other
Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: nagiosapi, Unassigned)

References

(
URL
)

Details

(Whiteboard: [id=nagios1.private.scl3.mozilla.com:370457]) 

Automated alert report from nagios1.private.scl3.mozilla.com:

Hostname: mozdefes4.private.scl3.mozilla.com
Service:  Disk - All
State:    WARNING
Output:   DISK WARNING - free space: /data 460578 MB (10% inode=99%):

Runbook:  http://m.allizom.org/Disk+-+All
It looks like all these boxes are sharing the same nfs mount as they all went into alarm at the same time? I don't know how you handle the multiple write conditions with nfs however? Got cluster or just writing in diff locations?

mozdefes5.private.scl3.mozilla.com
mozdefes2.private.scl3.mozilla.com

[dgarvey@mozdefes4.private.scl3 ~]$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/sda3                           37G   16G   19G  46% /
tmpfs                               16G     0   16G   0% /dev/shm
/dev/sda1                          985M  100M  836M  11% /boot
10.22.75.11:/mozdef_elasticsearch  4.0T  3.6T  445G  90% /data
[dgarvey@mozdefes4.private.scl3 ~]$ mount
/dev/sda3 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
10.22.75.11:/mozdef_elasticsearch on /data type nfs (rw,noatime,hard,nointr,rsize=65536,wsize=65536,bg,proto=tcp,vers=3,nolock,addr=10.22.75.11)
nfsd on /proc/fs/nfsd type nfsd (rw)
[dgarvey@mozdefes4.private.scl3 ~]$
Automated alert acknowledgement: (dgarvey)updated bug 1025413
Status: NEW → ASSIGNED
Automated alert recovery:

Hostname: mozdefes4.private.scl3.mozilla.com
Service:  Disk - All
State:    OK
Output:   DISK OK
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.