833397 - deploy heka 0.1 into logstash servers for marketplace-dev

Status: RESOLVED FIXED

(Cloud Services :: Operations: Marketplace, task)

Description

[Victor Ng [:vng] [:crankycoder]]

Attachment: /etc/hekad.json

We'd like to deploy heka 0.1 onto the logstash servers (logstash1, logstash2) for marketplace-dev.

The RPM can be found here:

r6.build.mtv1.svc.mozilla.com:/home/vng/RPMS/hekad-0.1-1.x86_64.rpm

I've attached a sample configuration that should be placed into /etc/hekad.json.

We'll need to configure the StatsdOutput in hekad.json to write out to the proper host/port for statsd.

The FileOutput in hekad.json will also need to be configured to write out JSON logs to a proper filepath.

Comment 1

[Victor Ng [:vng] [:crankycoder]]

RPM built on r6 did not include heka-mozsvc-plugins, I'm rebuilding the RPM now.

Comment 3

[Victor Ng [:vng] [:crankycoder]]

Attachment: updated /etc/hekad.json

Comment 4

[Victor Ng [:vng] [:crankycoder]]

Attachment: rebuilt hekad with all plugins

Comment 5

[Victor Ng [:vng] [:crankycoder]]

:jason - this RPM should fix the missing CefOutput error you were seeing

Comment 6

[Jason Thomas [:jason]]

I install the new rpm but I am still seeing the error.

2013/02/04 15:51:08 Error reading config: Error reading outputs: No such plugin of that name: SentryOutput

[root@ip-10-248-118-214 ~]# rpm -qip hekad.rpm 
Name        : hekad                        Relocations: (not relocatable)
Version     : 0.1                               Vendor: [e-mail]
Release     : 1                             Build Date: Tue 22 Jan 2013 11:56:14 AM EST
Install Date: (not installed)               Build Host: r6.build.mtv1.svc.mozilla.com
Group       : default                       Source RPM: hekad-0.1-1.src.rpm
Size        : 7038926                          License: unknown
Signature   : (none)
Packager    : <[e-mail]>
URL         : http://example.com/no-uri-given
Summary     : no description given
Description :
no description given

Comment 7

[Victor Ng [:vng] [:crankycoder]]

Attachment: hekad with SentryOutput

$ rpm -qip ./hekad-0.1-1.x86_64.rpm 
Name        : hekad
Version     : 0.1
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : default
Size        : 7164089
License     : unknown
Signature   : (none)
Source RPM  : hekad-0.1-1.src.rpm
Build Date  : Mon Feb  4 16:56:58 2013
Build Host  : r6.build.mtv1.svc.mozilla.com
Relocations : (not relocatable)
Packager    : <[e-mail]>
Vendor      : [e-mail]
URL         : http://example.com/no-uri-given
Summary     : no description given
Description :
no description given

If you check the binary:


$ strings hekad |grep SentryOutput
SentryOutput
SentryOutputWriter
SentryOutputWriterConfig
*heka_mozsvc_plugins.SentryOutputWriter
heka_mozsvc_plugins.SentryOutputWriter
*heka_mozsvc_plugins.SentryOutputWriterConfig

Comment 8

[Jason Thomas [:jason]]

Deployed to logstash hosts.