Open Bug 1547823 Opened 9 months ago Updated Last month

Upgrade Node build/test environment to v10.x

Categories

(Firefox Build System :: General, task, P3)

Tracking

(Not tracked)

People

(Reporter: loganfsmyth, Assigned: dmose)

References

(Blocks 2 open bugs)

Details

Attachments

(1 obsolete file)

Currently we use 8.x , but it went into maintenance mode in January 1st and will be end-of-lifed in December, so we should upgrade Node to a newer version. 12.x came came out last week, but presumably that would be a bit rushed and may life hard for distro maintainers, so 10.x seems like a reasonable choice.

Several projects that use Node for building, bundling, and unit testing are beginning to run into frustrations with 8.x due to missing ECMAScript and Node APIs, and the fact that some dependencies are beginning to drop support for it. For example https://bugzilla.mozilla.org/show_bug.cgi?id=1546370

I asked this in https://www.mail-archive.com/dev-builds@lists.mozilla.org/msg01363.html, with no answer, but the question still applies: There are two consumers of node: builds and developer workflow things like eslint. Is it necessary to bump the version required for the former?

It would be nice to upgrade for two reasons.

  1. we have node based unit tests for the debugger that have a Promise polyfill
  2. we would like to use v8's new snapshots in the build and that requires a newer version

(In reply to Jason Laster [:jlast] from comment #3)

It would be nice to upgrade for two reasons.

  1. we have node based unit tests for the debugger that have a Promise polyfill
  2. we would like to use v8's new snapshots in the build and that requires a newer version

None of these seem related to building Firefox.

I think my question would mostly be about how to make CI handle 2 versions. Is there much additional complexity to having build logic that runs in an environment that is different from the test logic?

The build and test environments are already different.

(In reply to Mike Hommey [:glandium] from comment #4)

(In reply to Jason Laster [:jlast] from comment #3)

It would be nice to upgrade for two reasons.

  1. we have node based unit tests for the debugger that have a Promise polyfill
  2. we would like to use v8's new snapshots in the build and that requires a newer version

None of these seem related to building Firefox.

The motivation for number 2 is to speed up build time, at least by a bit. Presumably we'd need to benchmark a bit to know by how much. That said, having two different versions of node for those different things seems like it adds complexity and fragility, so we'd want to think carefully about that.

Note also even if we were to separate build and test versions, Node v8 (what we're currently running) will hit end-of-life at the end of 2019, so it doesn't seem wise to keep them separated longer than that.

Are you saying that newer versions of node can't run code that runs on older versions?

The concern once Node 8 is end-of-lifed is that build tooling in the Node ecosystem will start dropping support for it quickly, which would prevent us from adopting newer versions of tools.

Additionally, once it is end-of-lifed, it will no longer get security fixes.

(In reply to Logan Smyth [:loganfsmyth] from comment #10)

The concern once Node 8 is end-of-lifed is that build tooling in the Node ecosystem will start dropping support for it quickly, which would prevent us from adopting newer versions of tools.

What tool we use to build would be impacted by that?

(In reply to Dan Mosedale (:dmose, :dmosedale) from comment #11)

Additionally, once it is end-of-lifed, it will no longer get security fixes.

That's not our concern to have. It's downstream's. Downstreams that use desupported versions of software is not unheard of, neither is downstreams fixing security issues in those.

What tool we use to build would be impacted by that?

I could certainly see that happening for Webpack, and I'm sure other things.

Once Node 8 is end-of-lifed, new major versions of libraries published to npm will begin to assume that Node 10 is the minimum version anyone will be using and start relying on new functionality introduced in Node 10. For instance, we've already accidentally tried to introduce dependencies in the past (because the dev was using Node 12 locally) that required async generator support in Node , which only landed as part of Node 10. Those patches had to be undone because we were still required to support Node 8.

Blocks: 1490802

(In reply to Logan Smyth [:loganfsmyth] from comment #13)

What tool we use to build would be impacted by that?

I could certainly see that happening for Webpack, and I'm sure other things.

Where do we use Webpack in the current build?

(In reply to Nathan Froyd [:froydnj] from comment #14)

Where do we use Webpack in the current build?

In retrospect probably not the best example. I don't think it runs in CI at the moment, we just run it manually locally which is ugly. Though again if we had people running newer node locally that increases the chance of accidental breakage because something could work locally by chance. Babel is the biggest thing we run in CI, which at the moment supports older Node versions, but again will drop it in future versions, making it harder to upgrade.

(In reply to Nathan Froyd [:froydnj] from comment #14)

(In reply to Logan Smyth [:loganfsmyth] from comment #13)

What tool we use to build would be impacted by that?

I could certainly see that happening for Webpack, and I'm sure other things.

Where do we use Webpack in the current build?

We do not use Webpack but we do use node to build the DevTools Debugger frontend.
It starts with declaration like this:
https://searchfox.org/mozilla-central/source/devtools/client/debugger/src/moz.build#16-20

CompiledModules(
    'main.development.js',
    'main.js',
    'vendors.js',
)

Which is being translated into a nodejs call around here:
https://searchfox.org/mozilla-central/source/devtools/client/shared/build/node-templates.mozbuild#38-41
To finally execute this JS file on nodejs:
https://searchfox.org/mozilla-central/source/devtools/client/shared/build/build.js

It is only there to convert "JSX" files to pain javascript that gecko can evaluate as well as stripping Flow types annotations.
To do that it uses Babel.

All of this is ran to build any Firefox. Full builds as well as artifact builds. So on CI but also local builds.

For tests, we use node in toolchain artifact instead of node in tooltool after Bug 1571573.
I believe it's the right move since it's easier to update and we can see the provenance of the tools we are using

However, we figure out v8.11.3 is not new enough for some TLS proxy testing, which is important at this moment
node crashes after client ends a TLS session (detail in bug 1580976 comment 4)
v8.13.0 and v8.16.1 don't work neither.

Upgrading to node v10 for testing seems a good step to move forward.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bb8da28aa9d80ff0d23a3a2d46323550038558dc
node doesn't crash for v10 under current test.

Assignee: loganfsmyth → nobody
Status: ASSIGNED → NEW

I'm not working on JS parts of the codebase right now so I won't be moving this along, but reminder that Node 8 is reaching end-of-life at the end of December, so it really would be good for us to upgrade to 10 so we can stay on a supported release.

Attachment #9061486 - Attachment is obsolete: true

Thanks for kicking things off, Logan, and I'm gonna iterate on your patch and run with it...

Assignee: nobody → dmose
You need to log in before you can comment on or make changes to this bug.