1206108 - Split up devDependencies packages into those for testing vs required for `grunt build`

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

To run grunt build as part of deploy, we're going to need to npm install the dependencies required for it. However we do not need to install the testing related packages, so by splitting the two between `dependencies` and `devDependencies` we'll have more control. eg:
`npm install --production`
...will only install those under `dependencies`.

Comment 1

[Ed Morley [:emorley]]

Few notes:

1) We're currently using "load-grunt-tasks" which is supposed to reduce the Gruntfile.js `grunt.loadNpmTasks()` boilerplate - but we haven't actually removed the old `grunt.loadNpmTasks()` entries. 

2) Once we start splitting the packages into `dependencies` vs `devDependencies`, we'll get errors/warnings when using `grunt build` if we've only done a `npm install --production`, since load-grunt-tasks tries to load all the tasks, even those in `devDependencies`. Production needs to use:
    require('load-grunt-tasks')(grunt, {scope: 'dependencies'});
And locally:
    require('load-grunt-tasks')(grunt);
...ideally load-grunt-tasks would toggle between those depending on the value of `NODE_ENV` or the use of `grunt build --production`, I've filed a ticket against them for this:
https://github.com/sindresorhus/load-grunt-tasks/issues/39
...though I guess we can add conditionals to Gruntfile.js ourselves if needs be.

3) jit-grunt might be a better alternative compared to load-grunt-tasks - it makes the packages be lazy loaded for faster grunt initialisation (https://www.npmjs.com/package/jit-grunt).

Comment 2

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #1)
> ...though I guess we can add conditionals to Gruntfile.js ourselves if needs be.

eg using:
http://gruntjs.com/api/grunt.option

Comment 3

[Ed Morley [:emorley]]

Attachment: Split up devDependencies packages into those for testing vs `grunt build`

Comment 4

[William Lachance (:wlach)]

Comment on attachment 8663007 [details] [review]
Split up devDependencies packages into those for testing vs `grunt build`

lgtm

Comment 5

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/d5e7b5ca49f213e3ae1ebb6c167303887b052fa4
Bug 1206108 - Separate `grunt build` npm packages from those for testing

To run grunt build as part of deploy, we're going to need to npm install
the dependencies required for it. However we do not need to install the
testing related packages, so by splitting the two between `dependencies`
and `devDependencies` we'll have more control. eg:
`npm install --production` will only install those under `dependencies`.

See:
https://docs.npmjs.com/files/package.json#dependencies
https://docs.npmjs.com/files/package.json#devdependencies
https://docs.npmjs.com/misc/config#production

https://github.com/mozilla/treeherder/commit/89d11c115fad9a5693e7a9d24830bf2235155af9
Bug 1206108 - Remove unnecessary loadNpmTasks() calls

Since they're duplicating what the `require('load-grunt-tasks')(grunt)`
call is already doing. See:
https://github.com/sindresorhus/load-grunt-tasks/blob/master/readme.md

https://github.com/mozilla/treeherder/commit/97d42b862bbed92e52f4679b397657876b040884
Bug 1206108 - Add grunt option to skip loading `devDependencies` tasks

In production we plan to only install the packages listed under
`dependencies` in package.json. However by default load-grunt-tasks
loads tasks from packages from `devDependencies` too, which results in
warnings if those packages are not installed locally.

To ensure that the local development grunt tasks are still loaded, but
that warnings are not shown for missing packages on production, a new
option is added to select which packages are loaded. eg:
`grunt build --production`

See:
http://gruntjs.com/api/grunt.option
https://github.com/sindresorhus/load-grunt-tasks/blob/master/readme.md#only-load-from-devdependencies

https://github.com/mozilla/treeherder/commit/52a99d7a786533d5b37a3542b329253a2aec8209
Bug 1206108 - Pin load-grunt-tasks to v3.2.0

The previous value of "^3.2.0" is equivalent to "4 > version >= 3.2.0".
We should pin to a specific version to stop deployments breaking if
a new release doesn't behave as expected. See:
https://nodesource.com/blog/semver-tilde-and-caret

We should also do the same for devDependencies, but let's do that in
another bug, since it doesn't affect production.