725041 - Generate configured logstash environment to support sync metlog load testing

Status: VERIFIED FIXED

Description

[Rob Miller [:RaFromBRC :rmiller]]

To load test the full metlog tool chain, we'll need a correctly constructed and configured logstash environment. The environment should contain all of our required logstash input, filter, and output plug-ins, set up per load testing requirements:

- zeromq input plug-in to accept metlog-py's ZmqPubSender generated messages

- filter that routes all timer & counter message types to statsd output, all "oldstyle" message types to both the file system log file output and the HDFS bound file system output

- any required logrotate and accompanying cron config and scripts required to actually push the HDFS bound messages into the HDFS cluster provided by the metrics team

This bug represents getting this environment dialed in and working on a dev (or even local) box, a separate bug will be opened for handling the deployment of this to the staging environment.

Comment 1

[James Bonacci [:jbonacci]]

:RaFromBRC is there a deploy ticket open are resolved for Stage?

Comment 2

[Rob Miller [:RaFromBRC :rmiller]]

(In reply to James Bonacci [:jbonacci] from comment #1)
> :RaFromBRC is there a deploy ticket open are resolved for Stage?

This process from way back was not targeted for deployment, so this particular stack was never deployed to stage. This has been superseded by all of the various sync 1.1 deploy tickets that we've been using for our recent load testing.

Comment 3

[James Bonacci [:jbonacci]]

Well, in that case, marking as Verified.