Closed Bug 652335 Opened 13 years ago Closed 13 years ago

cm-metricsapp02 is out of disk space

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dietrich, Unassigned)

Details

Now that's no way to celebrate 100 million downloads!
Assignee: nobody → server-ops
Severity: normal → major
Component: glow.mozilla.org → Server Operations
Product: Websites → mozilla.org
QA Contact: glow-mozilla-org → mrz
Version: unspecified → other
cm-metricsapp02 is out of disk space.

[jbalogh@cm-metricsapp02 home]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      100G   95G  154M 100% /
/dev/cciss/c0d0p1      99M   26M   68M  28% /boot
tmpfs                  16G     0   16G   0% /dev/shm
10.253.0.121:/data    8.1T  7.8T  299G  97% /mnt/stats_im-log02
10.253.0.65:/data     774G  421G  315G  58% /mnt/stats_im-log03
10.253.0.121:/data-2/stats/
                      7.2T  3.3T  3.9T  46% /data-2/stats
10.253.0.11:/vol/stats_nfs
                      600G  496G  105G  83% /mnt/stats
10.253.0.11:/vol/mpt_app_cluster/glow.mozilla.org
                      288G  251G   38G  87% /mnt/netapp/glow.mozilla.org
I got glow going again by deleting my 250M log file, but it's going to crash again soon.
Severity: major → critical
Summary: Glow.m.o is empty. No numbers at all. → cm-metricsapp02 is out of disk space
The rotate logs cron was disabled during some troubleshooting a while back and wasn't restarted.
Also, there was a rsyslog filter statement missing which caused all the dmo log data to be copied to /var/log/messages.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.