1201523 - Run `grunt build` as part of the Heroku buildpack compile

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Infrastructure, defect, P3)

Description

[Ed Morley [:emorley]]

When I'd looked at this previously it looked like it would be more of a pain, since many of the Stack Overflow posts talk about using a third party multi-buildpack buildpack that let's you stack up several other buildpacks - but has a few issues. (The other alternative was to create your own custom buildpack, which would be added maintenance hassle).

Heroku now have native support for multiple buildpacks:
https://devcenter.heroku.com/changelog-items/653
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app

As such, we can use this to combine both the Python and nodejs buildpacks, so we can use the latter for our `grunt build`.

That said, we'll need to make sure that the Python buildpack bug around playing nicely with other buildpacks doesn't affect us:
https://github.com/heroku/heroku-buildpack-python/issues/223
https://github.com/heroku/heroku-buildpack-nodejs/issues/230
(though if it does, there is a workaround of adding `export PATH=/app/.heroku/node/bin:$PATH` to post_compile).

Rough steps:
1) Get a nodejs environment working as part of compile
2) Add grunt build to the steps run during compile
3) Make sure the dependencies in package.json are pinned properly, so the build is deterministic
4) Figure out if we want to land this yet (and diverge from stage/prod), or also switch stage/prod to running grunt build as part of deploy too. (Though if the latter, we'll need to get nodejs installed on the admin node first).

Comment 1

[Ed Morley [:emorley]]

5) Figure out how much of a hassle it would be to strip out node/node_modules after running the grunt build, so we don't bloat the size of the slug unnecessarily (and thus increase the time taken to transfer when spinning up each Dyno).

Comment 2

[Ed Morley [:emorley]]

I've added the nodejs buildpack prior to the python buildpack, so the latter can use it:

[~/src/treeherder]$ heroku buildpacks
=== treeherder-heroku Buildpack URL
https://github.com/heroku/heroku-buildpack-python

[~/src/treeherder]$ heroku buildpacks:add --index 1 https://github.com/heroku/heroku-buildpack-nodejs
Buildpack added. Next release on treeherder-heroku will use:
  1. https://github.com/heroku/heroku-buildpack-nodejs
  2. https://github.com/heroku/heroku-buildpack-python
Run `git push heroku master` to create a new release using these buildpacks.

Comment 3

[Ed Morley [:emorley]]

Attachment: Run grunt build during the Heroku deploy

Comment 4

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #1)
> 5) Figure out how much of a hassle it would be to strip out
> node/node_modules after running the grunt build, so we don't bloat the size
> of the slug unnecessarily (and thus increase the time taken to transfer when
> spinning up each Dyno).

The slug only increased from 55MB to 70MB so let's worry about this another time.

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8667995 [details] [review]
Run grunt build during the Heroku deploy

I'm not really that familiar with heroku yet, but this looks fine to me. I feel comfortable giving it an r+.

Comment 6

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/f4e915bdbe43d40302f75854af7b589e74696db4
Bug 1201523 - Add grunt-cli to local dependencies list

Normally grunt-cli (a wrapper around grunt to allow it to be easily run
from the command line) would be installed globally, so that it is on the
PATH. However to save having to do this on prod/Heroku/Travis, we
install it locally - and it will then be available via:
`./node_modules/.bin/grunt`
...or else via just `grunt`, if referenced inside a package.json scripts
section.

https://github.com/mozilla/treeherder/commit/0c79292d682c8ee6d61fbc29e49d99d7e0b1c1ef
Bug 1201523 - Run grunt build during the Heroku deploy

This commit relies on the nodejs buildpack being added to the list of
buildpacks for the app, and prior to the Python buildpack. See:
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app

The nodejs buildpack will automatically install the packages listed in
`dependencies` in package.json, so that we have the requirements for
the grunt build. We don't actually need node or all of the files in
node_modules after we've run the grunt build, so in the future could try
and remove them to reduce the resultant slug size (though it only
increased from 55MB to 70MB, so it's not urgent).

The dist directory has been added to `.slugignore` to prevent the
in-repo directory from being uploaded, since we'll be generating a new
one as part of the deploy. Once `dist/` is deleted from master, that
entry can be removed.