Closed Bug 1129924 Opened 7 years ago Closed 5 years ago

Create moz.build files for NSPR/NSS/ICU instead of using their build systems

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ted, Assigned: ted)

References

(Blocks 2 open bugs)

Details

We currently treat (and historically have) NSPR and NSS as third-party projects with their own build systems. We invoke NSPR's configure and then run make for it, and we jump through hoops to invoke make to build NSS.

However, we have lots of other third-party projects in the tree whose build systems we *don't* use. We either generate moz.build files for them on import (Breakpad, Skia) or import a gyp file via moz.build (WebRTC). I don't think there's any real compelling reason we need to use NSPR and NSS's build systems directly. In fact, if you look at the ridiculousness we use to build NSS I would argue that we're barely using their build system anyway:
https://dxr.mozilla.org/mozilla-central/source/config/external/nss/Makefile.in

The benefits of this would be:
1. One less configure script to invoke (NSPR's), which saves time at the beginning of a build.
2. NSS' build system is not parallel-safe, so we have to invoke it with make -j1 which makes the build slower. With moz.build we could build it in parallel.
3. Better integration into our build graph so things like 'mach build binaries' will be faster.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #0)
> 2. NSS' build system is not parallel-safe, so we have to invoke it with make
> -j1 which makes the build slower. With moz.build we could build it in
> parallel.

That hasn't been true in a while, but the rules to build nss in parallel are horrible and hard to maintain.
Now, for the downside:
- NSPR and NSS "upstreams" will likely make those moz.build files stale pretty quickly.

One idea I had is to parse the manifest files in nss directories like we parse the gyp stuff.
NSPR doesn't change all that much, and especially the set of files that get built probably haven't changed in ages.

NSS might be a different story. Parsing the manifest.mn files sounds clever, as long as they don't do anything funny in there. Would you do that at build-time or at import time and generate moz.build?
Depends on: 1230117
Depends on: 1237872
Summary: Create moz.build files for NSPR/NSS instead of using their build systems → Create moz.build files for NSPR/NSS/ICU instead of using their build systems
Depends on: 1239083
I don't think we talked about them, but we're also running subconfigures for libffi, jemalloc, and sometimes freetype2:
https://dxr.mozilla.org/mozilla-central/rev/9a358be6fa798f24deecac1b502742b2c37cd6bd/configure.in#9095

I suspect jemalloc is simple enough that we could probably just write a moz.build file for it. libffi is more complicated, but I will take a look at it when I get NSPR/NSS/ICU sorted. freetype2 looks pretty complicated too, and we're only building it on Android right now, so I think I'll punt on that for the time being.
Depends on: 1262155
Depends on: 1262163
Depends on: 1262164
Depends on: 1295937
No longer depends on: 1237872
In the intervening time, I fixed bug 1230117 (stop using NSPR's build system) and bug 1239083 (stop using ICU's build system).

What's left:
bug 1262155 covers replacing the libffi build system, which chmanchester is actively working on.
bug 1295937 covers replacing the NSS build system, which I'm actively working on.
bug 1262163 covers replacing the freetype2 build system, which nobody is actively working on, but only impacts Android builds
bug 1262164 covers replacing the jemalloc configure. We don't use the jemalloc build system, just configure, so this is not super impactful (but might make a difference in configure times on Windows).
I'm going to close this bug, since we fixed all the really big-ticket items (hooray!)

bug 1262163 only impacts Android builds. It'd still be nice to fix it, and it's probably not a huge effort, but someone will have to step up to own that.

bug 1262164 is only part of configure, not the build step, so it's not fully in-scope here.
Assignee: nobody → ted
Status: NEW → RESOLVED
Closed: 5 years ago
No longer depends on: 1262163, 1262164
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.