814122 - logstash-metlog 0.8.8 deploy to sync staging servers

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Victor Ng [:vng] [:crankycoder]]

Attachment: updatd metrics_hdfs.ini file for sync staging

This revision of logstash-metlog includes an update to the python script which pushes JSON logs to the HDFS cluster.

We needed to add the ability to configure the use of SSH identity files to use alternate private keys when pushing to HDFS.

RPMs are available at : 

r6.build.mtv1.svc.mozilla.com:/home/vng/RPMS/logstash-metlog-0.8.7-1.x86_64.rpm

The metrics_hdfs.ini file needs to have a modification in it as the filename that is trying to be pushed into HDFS is incorrect.

The SRC_LOGFILE name has been modified in the attached INI file to drop the .gz extension as the real logfiles that are rotated are not gzipped.

Comment 1

[Victor Ng [:vng] [:crankycoder]]

I've updated the RPM in : 

r6.build.mtv1.svc.mozilla.com:/home/vng/RPMS/logstash-metlog-0.8.7-1.x86_64.rpm

To fix the incorrect month in the RPM changelog.  No other modifications were made to the RPM.

Comment 2

[Victor Ng [:vng] [:crankycoder]]

Ugh.  I missed some fully qualified paths.  RPMs for logstash-metlog 0.8.8 are in 

r6.build.mtv1.svc.mozilla.com:/home/vng/RPMS

Comment 3

[Wesley Dawson [:whd]]

Now that the snafu with logstash is hopefully cleared up, I will take this, and update stage (which was downgraded to prepare for the prod change) to 0.8.8 next week.

Comment 4

[Wesley Dawson [:whd]]

I went to rebuild this rpm with the new specfile and updated the logstash dependency again to 1.1.5-2svc. I noticed the version bump and spec updates for 0.8.8 occur after the git tag for 0.8.8.

:vng, do you want to tag the current head of the master branch as 0.8.8-2 or 0.8.9 to build against? I can also just build 0.8.8-2svc with the current head, assuming we won't need a 0.8.8-3.

Comment 5

[Victor Ng [:vng] [:crankycoder]]

:whd I've tagged the head to 0.8.9 and pushed the tag up.

Comment 6

[Wesley Dawson [:whd]]

I'm going ahead with putting the 0.8.9 rpm in mrepo and deploying to stage later today.

Comment 7

[Wesley Dawson [:whd]]

Deployed. We had already modified the hdfs configs to remove the .gz extension as a break-fix some time ago so only the rpms needed to be updated.

Comment 8

[James Bonacci [:jbonacci]]

Done long ago then...