1142159 - Send relengapi logs to log-aggregator

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This should include what's currently going to relengapi.log as well as the Apache access logs.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

I set up a SysLogHandler in relengapi.wsgi and deployed that.  Data is flowing!

Access logs remain TBD

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

On Amy's advice I reconfigured the SysLogHandler to log to localhost, and also fixed up the formatting so that it produces legit output.  I still need to land the puppet patch to make those hosts forward to our aggregators.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

# see http://help.papertrailapp.com/kb/configuration/configuring-centralized-logging-from-python-apps/
syslog_formatter = logging.Formatter(
    '%(asctime)s ' + socket.gethostname() + ' relengapi: %(message)s', datefmt='%b %d %H:%M:%S')
sys_log = logging.handlers.SysLogHandler(socktype=socket.SOCK_DGRAM)
sys_log.setLevel(logging.INFO)
sys_log.setFormatter(syslog_formatter)

Apache logs are as simple as http://blog.papertrailapp.com/send-apache-access-logs-to-remote-syslog-in-1-line/

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

I enabled logging to syslog for all vhosts in the releng cluster, too, in r103928.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

OK, looking good!

The access logs are pretty voluminous, so we may need to filter those back at some point.