Figure out what to do with zamboni's timing logs



9 years ago
4 years ago


(Reporter: jbalogh, Assigned: oremj)





9 years ago
In we started logging how long each request takes along with some other metrics.  The data is very interesting, but today's log file is already over 500M and we're only halfway through.  I don't want to fill up our disks with this.

Here's what we want:
1. A different place to store timing data.  Logging can be configured to send timer logs to a different place than the normal syslog.  Or they can go to syslog and get filtered out later.  Timing data can be deleted once #2 is going.

2. A way to get this into ganglia.  I'd like to write a script that aggregates and analyzes the data and throw it into a few ganglia graphs.  The stats in ganglia can be our long-term storage, we don't need to keep all the logs.  My ganglia script needs a way to read the logs; right now it doesn't have permissions.


9 years ago
Assignee: server-ops → jeremy.orem+bugs

Comment 1

9 years ago
Looks like it compresses down to about 21M a day. I vote we just periodically delete them.

2. I added average time to ganglia.

Do we want anything else?
Last Resolved: 9 years ago
Resolution: --- → FIXED

Comment 2

9 years ago
I want to be able to break down the times into sections of the site like /extensions and /search.  I think we need to be able to parse the timing logs for that.
Product: → Graveyard
You need to log in before you can comment on or make changes to this bug.