Bug 1187958 (node-support)

[META] Add node buildsystem support

NEW
Unassigned

Status

task
4 years ago
3 months ago

People

(Reporter: selenamarie, Unassigned)

Tracking

(Depends on 3 bugs, Blocks 1 bug, {meta})

Firefox Tracking Flags

(Not tracked)

Details

This is meant to be a tracker for adding npm and node support. 

Collecting notes here: https://etherpad.mozilla.org/npm-and-node-in-build-system
Motivation and goal would be better pasted here than referenced through an etherpad link.
Keep in mind build machines don't have access to the outside world. Also, we want builds and test results to be reproducible over time. We'll almost certainly need to run our own internal package mirror so all packages referenced by the tree are available for all of time. Or, we can vendor dependencies in mozilla-central like we do for Python. The latter is preferred, as it doesn't introduce an external dependency beyond the base `node` and `npm` executables and there is no potential for "drift" over time.
Here's a straw-man take on a user story here:

As a Firefox module owner or peer, I should be able to easily use key high-quality npm packages (such eslint, react-tools, various css linters, etc) in the build process.  

Motivation: most of the JavaScript and web tooling energy these days is happening in the node/npm ecosystem.  There's an opportunity to help take our build-tooling to the next level using well-maintained, industry-tested packages where those are already available.

This probably needs more fleshing out in terms of what the acceptance criteria would be, but it's a start.
Depends on: 1211914
Any progress on this? This might become a critical part of building the Firefox developer tools. We are looking at processing our JS files on the fly, so we need to run packages on node to do this.
(In reply to Gregory Szorc [:gps] from comment #2)
> Keep in mind build machines don't have access to the outside world. Also, we
> want builds and test results to be reproducible over time. We'll almost
> certainly need to run our own internal package mirror so all packages
> referenced by the tree are available for all of time. Or, we can vendor
> dependencies in mozilla-central like we do for Python. The latter is
> preferred, as it doesn't introduce an external dependency beyond the base
> `node` and `npm` executables and there is no potential for "drift" over time.

Vendoring deps seems far easier and we'd be fine with that. I don't think we plan on using too many build-time node dependencies, so it shouldn't be hard to get what we need vendored.
One thing that has moved forward slightly is that it's now part of the Ubuntu testing images used in TaskCluster (https://bugzilla.mozilla.org/show_bug.cgi?id=1273695#c4) (thanks Selena for the heads-up on this).
Notes from the old etherpad, mostly the stuff from Ted is still relevant:

From dmose: 
10:47 <dmose> the base requirement is probably being able to "npm install" something and then execute it with node
10:47 <dmose> eslint is probably a fine choice for that base
10:47 <dmose> since both Hello, Mobile, and maybe DevTools use it

on existance of npm/node on build machines: 

10:48 <dmose> because there are SPDY tests that run using it
10:49 <dmose> here's another request, which might well already be fulfilled by the existing stuff on the test machines: https://bugzilla.mozilla.org/show_bug.cgi?id=1083249
10:50 <dmose> npm depends on having git around, i believe
10:50 <dmose> there is a bug related to git
10:50 <dmose> https://bugzilla.mozilla.org/show_bug.cgi?id=590921
10:50 <dmose> for windows

From Ted:
07:57 <ted> 1) get it into mozillabuild for windows users
07:57 <ted> 2) make sure the package is easily installable on linux/mac, get it into the bootstrap scripts in-tree
07:57 <ted> 3) get it onto the build machines, maybe using tooltool
07:58 <ted> 4) decide if it's a hard requirement or if you can --disable-foo your way out of it, if it's the former that's going to put extra burden on tier 3 platforms (weird stuff like Solaris/AIX/whatever)
07:58 <ted> 5) add configure checks for it, error if it's missing
From Ted's list:

1) is done.
2) appears to done in service of making eslint work after './mach bootstrap' has been run

3-5 are still TBD.
Note, there's a lot of work going on in bug 1401189 that aid or affect aspects affect this.

Also for 3), you basically just need to make sure you choose the right docker image for your task, or set one up with npm/node on. There's already a linux based taskcluster docker image called "lint" that has the necessary items.
See Also: → 1401189

Updated

Last year
Product: Core → Firefox Build System
Duplicate of this bug: buildnode

Updated

Last year
See Also: → 1457223
Summary: Add npm and node support → [META] Add npm and node support
Depends on: 1482433
Depends on: 1482435
Depends on: 1482676
Alias: node-support
Summary: [META] Add npm and node support → [META] Add node buildsystem support
Blocks: 1520362
Type: defect → task
You need to log in before you can comment on or make changes to this bug.