966086 - Create module to send user activities to MakerMind

Status: RESOLVED FIXED

(Webmaker Graveyard :: General, defect)

Description

[Jon Buckley [:jbuck]]

We need to create a node module that we can easily integrate into our server apps to send user activities to the MakerMind.

It should easily support a "null" transport mechanism for local development, as well as the udp and redis transport mechanisms that logstash can use.

http://cookbook.logstash.net/recipes/logging-from-nodejs/ recommends a node module called "bucker" that we might be able to use.

I'm not to certain how we should store actions, so we should come up with a few sample actions, to make sure we're capturing the right data.

Comment 1

[JP Schneider :jp]

We have a ELK stack setup (Elasticsearch, Logstash, Kibana) 

ec2-23-20-40-4.compute-1.amazonaws.com



Here is the config for logstash we're using:

input { stdin { type => example } }
output { 
  stdout { codec => rubydebug }
  elasticsearch{
      type => "all"
      embedded => false
      host => "127.0.0.1"
      port => "9200"
      cluster => "jf"
      node_name => "logstash"
    }
}

redis {
        host => "127.0.0.1"
        type => "redis-channel"
        data_type => "channel"
        key => "bucker"  // bucker will only ever send to this key
}

-------------

We're thinking about using Redis pubsub for transport instead of UDP for reliability:http://cookbook.logstash.net/recipes/logging-from-nodejs/

Comment 2

[JP Schneider :jp]

NEW WORKING CONFIG:

/etc/logstash/conf.d/logstash_with_es.conf

redis {
        host => "elk-demo.4pyhmj.0001.use1.cache.amazonaws.com"
        type => "redis-channel"
        data_type => "channel"
        key => "bucker"
        }
}
output {
  elasticsearch_http{
      host => "127.0.0.1"
    }
}

Comment 3

[Jon Buckley [:jbuck]]

http://michael.bouvy.net/blog/en/2013/12/06/use-lumberjack-logstash-forwarder-to-forward-logs-logstash/

Comment 4

[Jon Buckley [:jbuck]]

https://github.com/jbuck/hatchet
https://www.npmjs.org/package/hatchet