1482433 - always require NodeJS by removing --disable-nodejs switch

Status: NEW

(Firefox Build System :: General, enhancement)

Description

[Dan Mosedale (:dmosedale, :dmose)]

Note that we'll first need to figure out another way to make the L10n CI builds work (right now they simply set --disable-nodejs in their mozconfig files).

This is likely to happen early in the 64 nightly timeline.

Comment 1

[Tom Prince [:tomprince]]

The source tarball jobs are also using that flag.

Comment 2

[Gregory Szorc [:gps]]

I think we'll want to logically still keep the ability to run the build system without Node.js to support the special projects and build configurations that don't need it.

What this means is we'll likely want a configure function somewhere for determining whether Node.js is required. We'll internally disable the requirement for things like source tarball "builds."

TBH I'm not sure the best way to implement this. We do have moz.configure files that are per-"project" (e.g. "browser"). But I'm not sure if we can easily have a "shared" moz.configure check that pulls in the results of a function that is defined by each separate project/app/unit/build config. The closest analog we have for this is --disable-compile-environment. But that requires individual builds to add a configure flag to opt in, which is annoying.

We could perhaps still retain the --disable-nodejs flag but make that a fatal error for build configurations that require it (like Firefox).

glandium: do you have any thoughts on how this should be done?

Comment 3

[Mike Hommey [:glandium]]

node.configure is included from toolkit/moz.configure, which is only included for projects that use gecko, so there's no problem here.

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

If the problem is individual builds opting in, is there any existing mechanism for attaching this to classes of builds (eg all L10n builds)?

Comment 5

[(not currently active) Ted Mielczarek]

Yeah, we have plenty of things that respect `--disable-compile-environment`, for example.

Comment 6

[Gregory Szorc [:gps]]

I think a fundamental problem here is that we don't explicitly tell configure what we're building: callers can pass various configure flags and then proceed to build firefox, source packages, l10n packages, etc.

Because configure lacks the knowledge to know what the caller is actually requesting, we have to be conservative in rejecting things and/or we end up exposing possibly foot-gunning flags, like --disable-compile-environment and --disable-nodejs.

I think a more user-friendly experience would be to tell configure explicitly what things you intend to build. Then configure can imply options from that. e.g. --with-build=sourcepackage,l10nrepack would imply --disable-compile-environment/--disable-nodejs.

Fixing this seems like a tall order. Yet, doing nothing means `mach build` would encounter an error on the first build action that attempts to use Node.

Perhaps we should add a "verification" step at the top of `mach build` - before the backend is invoked - that inspects the build targets and verifies config.status state is sane?

Comment 7

[Gregory Szorc [:gps]]

To be clear, I don't think we should scope bloat this specific bug to tackle larger, long-outstanding issues.

The easiest thing is to do nothing and continue to allow --disable-nodejs to turn off the configure checks for people with workflows that don't require node and don't want to install it. But that will result in some people specifying it and running into build-time errors - possibly minutes into the build - if they do. I wish we didn't have that footgun. But until we have evidence of people falling into this trap, I think it is OK to keep --disable-nodejs around.

i.e. we can start landing build system actions that use Node. If --disable-nodejs is used, they will hit build system errors. As long as any error messages related to missing Node instruct the user to remove --disable-nodejs, I think we'll be fine.

Does this sound reasonable?

Comment 8

[Dan Mosedale (:dmosedale, :dmose)]

That sounds like a good choice to me.  I'll add --disable-nodejs verbiage into the appropriate action error message.

Comment 9

[John Paul Adrian Glaubitz]

I think it should still be possible to build without NodeJS and just be able to point to the directory with the compiled JavaScript files.

The reason is that NodeJS does not support as many architectures as Firefox itself, so making NodeJS a hard-requirement makes Firefox less portable.

At least in Debian, there is the possibility to build architecture-specific and architecture-agnostic artifacts on different architectures. For example, it would make sense to compile all the JavaScript files on x86_64 and put the build artifacts into a package called "firefox-common" with the architecture "all". Then have the various architectures build the architecture-specific artifacts.

When the user installs Firefox on Debian, they would for example install a package called "firefox-75.0-1_amd64.deb" plus a package called "firefox-common_75.0-1_all.deb".

This way, we could also easily provide Firefox for RISC-V as this architecture already has a fully working Rust compiler.

Comment 10

[Dan Mosedale (:dmosedale, :dmose)]

John, that is an interesting point. Whether or not that sort of patch would be accepted into the tree would depend on how much maintenance it would require (ie how complicated the code was), as well sign off from a build peer. I suspect this work would need to be driven by someone from a downstream distribution, as it's hard for me to imagine how this would become high enough priority for someone else.

Comment 11

[RRogalski]

re: building with precompiled javascript files for architectures unsupported by node

Oracle has been doing this for quite some time now to package firefox for SPARC64 on Solaris11. You can see that here: https://github.com/oracle/solaris-userland/tree/master/components/desktop/firefox

Comment 12

[RRogalski]

Some more info on that, for anyone else who has found this in an effort to build Firefox on a nodeless system:

On that link above, there is a wrapper-node folder. It contains a script called node, and acts as a fake node of sorts. Adding this wrapper-node folder to your $PATH will cover the node.js requirement for building Firefox. It will correctly output pre-generated js files for what the build system is asking.
Note that you will probably need to edit the script to change mach to uname -p, as this script was intended for Solaris. Those two commands are exactly identical though :D

You might also be interested in some of the patches in the above link in the patches dir, especially if you are on a big endian system.

Besides one tiny patch I wrote myself, the patches and node wrapper above were enough to get Firefox to build on Linux/SPARC for me. It would be nice if this was integrated in the Firefox build system, but personally I'm just glad I got it to work at all. So good luck, next brave soul to land here. :)