1026503 - vagrantify fjord

Status: RESOLVED FIXED

(Input :: General, defect, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

We need a better way for contributors to get Fjord working on their local machine so that they can contribute. Currently, the expertise and work involved in doing that are probably too high. Further, our current system (do it all by hand) restricts the set of possible contributors to people using Linux and Mac OSX.

I spent some time last week looking at what other Mozilla projects were doing these days. I had looked at Vagrant a year or two ago and disliked it, but it seems like they've fixed a lot of the issues I had since then.

This bug covers finishing up my vagrantify branch so that a new contributor can install a few things (git, virtualbox, vagrant) and be up and running with an instance of Fjord.

Specifically, this requires the following work:

1. finish up bin/vagrant_provision.sh

2. overhaul the documentation folding the useful bits of the hacking howto into the getting started guide

3. testing all of this on OSX and Windows

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this a mentored bug for now. It's something we need to fix soon, so if no one claims it before I get a chance to continue working on it, then I'll take it.

The branch in question is here:

https://github.com/willkg/fjord/tree/vagrantify

If you're interested in working on it, let me know in the bug comments or on IRC (I'm willkg).

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Also, here are some probably useful links:

* I heavily based what's there on this: https://github.com/mozilla/dxr/blob/master/vagrant_provision.sh
* I started rewriting the Getting Started chapter based on this: https://dxr.readthedocs.org/en/latest/getting-started.html
* Basic information on provisioners: http://docs.vagrantup.com/v2/provisioning/basic_usage.html
* We're using the Vagrant shell provisioner: http://docs.vagrantup.com/v2/provisioning/shell.html

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

De-mentoring this and grabbing it to work on.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Adding whiteboard data.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I got everything working on Linux and did a first-pass overhaul on the docs.

Still outstanding:

* "./manage.py <anything>" is **crazy slow**. Dean is having similar
  problems with vagrant and kitsune and says it's from the way we do
  vendor/. The current thinking is to switch from vendor/ to peep-based
  virtual environments. I'll probably create a new bug for this later.

* Everything needs to be tested on Mac OSX and Windows and anything that
  doesn't work should get fixed.

Will land as soon as the gengo human translation stuff works and I can land whatever in master again.

Comment 6

[Ian Kronquist (:muricula)]

Hey, I just used the Vagrantfile on a mac, here is what I found:
You really need to install the vagrant virtualbox add ons.
The lxml python package may fail to compile if the VM does not have enough memory. I bumped it to 1024 MB with the virtualbox GUI, but I need to look and see if there is a way to specify minimum VM memory size in a vagrant file.

Mike also pointed out that the version of node installed with apt-get is ancient and may not be acceptable for development. He recommended using an alternate installation method. I'm going to take a look at this and perhaps open another bug based on this.

Comment 7

[Ian Kronquist (:muricula)]

Without the virtualbox add ons vagrant hangs (at least on my osx system). In the file bin/vagrant_provision.sh there are suggestions on how to fix this. These are not, however, on the read the docs page, and should be added there as part of the bug.

lxml compiled correctly during the initial provision, but when I later tried to install it to a virtual environment the compilation failed. Upon changing the size of the VM's memory lxml compiled successfully. I have encountered this previously at the Open Source Lab when I was having chef compile the python mysql bindings and gcc was killed. Perhaps a note in the troubleshooting section would be appropriate.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Landed in master in:

https://github.com/mozilla/fjord/commit/74a409e
https://github.com/mozilla/fjord/commit/a7a440f
https://github.com/mozilla/fjord/commit/a7e958e

Ian was able to get a vm working. I've gotten a vm working. Given that, I think this is probably 80% of the way there and we should spin off issues with it in new bugs.

This doesn't affect our servers, so I'm marking this FIXED now.