1268485 - Add elasticsearch to local Vagrant environment

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[James Graham [:jgraham]]

This needs to be installed when the vagrant instance is provisioned.

emorley: Can you help here? I guess that some configuration is required under puppet/ but I haven't ever used puppet before, so I don't really know what I'm doing :) There seems to be an official module at [1], if that helps.

[1] https://github.com/elastic/puppet-elasticsearch

Comment 1

[Ed Morley [:emorley]]

Currently we don't use any third-party puppet modules, such as the one linked in comment 0. I've looked into how we can use these without having to vendor them into the repo, which turned up the concept of Puppetfiles:
https://github.com/rodjek/librarian-puppet

However, we end up in a catch-22 where we have to have installed the puppet modules prior to puppet running during the VM provision. This could be achieved by using a shell (ie bash script) provisioner prior to the existing puppet provisioner:
http://www.jeroenreijn.com/2014/11/using_vagrant_with_puppet.html

However this + the extra dependencies that the comment 0 puppet module requires makes this solution quite bloated, given that I imagine we only need an elasticsearch service running on localhost with mostly defaults.

There doesn't appear to be an Ubuntu package repository package for Elasticsearch for Ubuntu 14.04, only 15.10 onwards:
http://packages.ubuntu.com/wily/elasticsearch

However we can use the Debian package from the Elasticsearch website directly, per:
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-elasticsearch-on-ubuntu-14-04

We'll need to also install Java + OpenJDK.

Comment 2

[Ed Morley [:emorley]]

Following the instructions on:


...was giving unhelpful
E: Unable to locate package elasticsearch

Which it turns out is due to their not being 32bit packages of ES 2.x

I'll look at switching the Vagrant base Ubuntu image to x64.

Comment 3

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #2)
> Following the instructions on:

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html#_apt

Comment 4

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:vagrant-elasticsearch > mozilla:master

Comment 5

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:vagrant-elasticsearch > mozilla:master

Comment 6

[Ed Morley [:emorley]]

Comment on attachment 8747059 [details] [review]
[treeherder] mozilla:vagrant-elasticsearch > mozilla:master

vagrant ~/treeherder $ curl -X GET localhost:9200
{
  "name" : "Scorpion",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.3.2",
    "build_hash" : "b9e4a6acad4008027e4038f6abed7f7dba346f94",
    "build_timestamp" : "2016-04-21T16:03:47Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

Comment 7

[Ed Morley [:emorley]]

I've also filed this issue to improve the Elasticsearch docs re the unmentioned 64bit requirement:
https://github.com/elastic/docs/issues/85

Comment 8

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/1193b2d08f5f20670eacde8fbba74b05e583240e
Bug 1268485 - Vagrant: Switch to a 64bit Ubuntu base image

Since no packages for Elasticsearch 2.x+ are available for 32bit.
This also makes Vagrant consistent with stage/prod/Heroku/Travis.

https://github.com/mozilla/treeherder/commit/11fdc7feb939e305d679d5bef113e71468d0fe4a
Bug 1268485 - Vagrant: Install/run an Elasticsearch instance

Since it's required for fuzzy auto-classification in bug 1268484.

Follows the steps here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

Test using:
`curl -X GET localhost:9200`