gmake build rework and cleanup
Categories
(NSS :: Build, enhancement, P1)
Tracking
(Not tracked)
People
(Reporter: glogow, Assigned: glogow)
References
Details
Attachments
(9 files, 1 obsolete file)
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review |
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0
Steps to reproduce:
Try working on the NSS gmake build system to fix the parallel build issue with NSS experienced in the LibreOffice CI, forcefully building NSS with -j1.
Actual results:
The system is split into too many separate files per directory. It is hard to follow the actual workflow of the finally parsed / merged Makefile and parts of it are definitely not working as intended, where later includes override some previous settings.
There are multiple parts, where different defaults can reduce the size of the Makefiles in general. And it seems there are quite a bit of unused rules.
Then there is the whole gigantic freebl Makefile, where I thing all the separate libraries should probably be build in different directories, or some better way to build multiple libraries from one Makefile.
Expected results:
Makefiles are smaller due to better defaults. In the end they probably just include config and rules from coreconf and are otherwise self contained. Some templating is use to reduce boilerplate Makefile code. Unused rules and variables are dropped. It's actually possible to read the freebl Makefiles without total confusion ;-)
Maybe even someone will implement and merge a non-recursive build system, or the new gyp+ninja one becomes more ubiquitous and usable on more platforms, so this gmake one can be finally dropped.
Assignee | ||
Comment 1•4 years ago
|
||
In the manifest.mn the LIBRARY_VERSION is normally used to define
the major version of the build shared library. This ust works for
the pk11wrap case, because pk11wrap is a static library. But it's
still very confusing when reading the manifest.mn. Also the
referenced define in the code is just named SHLIB_VERSION.
So this prefixes the defines and the variables with NSS_, because
it tries to load the NSS library, just as the SOFTOKEN_.*_VERSION
is used to load the versioned softokn library.
Assignee | ||
Comment 2•4 years ago
|
||
Gets rid of a lot of boilerplate Makefile code.
Assignee | ||
Comment 3•4 years ago
|
||
- Drop the WIN% "32" default DLL suffix
- Add default export script handling => drop default MAPFILE
- Add default resource file handling => drop default RES
- Generate IMPORT_LIBRARY based on IMPORT_LIB_SUFFIX and
SHARED_LIBRARY, so we can drop all the explicit empty
IMPORT_LIBRARY lines
Assignee | ||
Comment 4•4 years ago
|
||
Depends on D70369
Assignee | ||
Comment 5•4 years ago
|
||
There is really no good reason to explicitly change the TARGET
variable. And the empty SHARED_LIBRARY variable should also be
in the manifest.mn to begin with.
All the other empty variables start empty or undefined, so there
is also no need to explicitly set them empty.
Depends on D70690
Updated•4 years ago
|
Assignee | ||
Comment 6•4 years ago
|
||
Depends on D70689
Updated•4 years ago
|
Assignee | ||
Comment 7•4 years ago
|
||
These pattern rules already had a comment to keep both in sync, so
just use an eval template to enforce this.
Depends on D69022
Assignee | ||
Comment 8•4 years ago
|
||
There aren't an Java sources in NSS, so just drop all the stuff
referencing java, jars, jni, etc.
I didn't try to remove it from tests.
Depends on D70691
Assignee | ||
Comment 9•4 years ago
|
||
Depends on D70986
Assignee | ||
Comment 10•4 years ago
|
||
Introduces a simple "%/d" rule to create directories using
$(MAKE_OBJDIR) and replace all explicit $(MAKE_OBJDIR) calls
with an order-only-prerequisites.
To expand the $(@D) prerequisite, this needs .SECONDEXPANSION.
Depends on D70988
Comment 11•4 years ago
|
||
Resetting severity to default of --
.
Comment 12•4 years ago
|
||
I'll take this series in NSS 3.53, so we'll land them later this week / early next week after we branch.
Comment 13•4 years ago
|
||
Try push failed for Windows make builds: https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=59e1b1970611b2ed08a3c7d2b0e270aa43584038
WIN954.0_DBG.OBJ/nssutil.lib
nsinstall -m 664 WIN954.0_DBG.OBJ/nssutil.lib ../../../dist/WIN954.0_DBG.OBJ/lib
rc.exe -DDEBUG -DWIN32 -D_X86_ -D_WINDOWS -DWIN95 -DNSS_DISABLE_AVX2 -DNSS_NO_INIT_SUPPORT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -I../../../dist/WIN954.0_DBG.OBJ/include -I../../../dist/public/nss -I../../../dist/private/nss -FoWIN954.0_DBG.OBJ/nssutil.res nssutil.rc
Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384
Copyright (C) Microsoft Corporation. All rights reserved.
WIN954.0_DBG.OBJ/nssutil.res finished
cp nssutil.def WIN954.0_DBG.OBJ/nssutil.def
rm -f WIN954.0_DBG.OBJ/nssutil3.dll
link -nologo -DLL -SUBSYSTEM:WINDOWS -DYNAMICBASE -DEBUG -OUT:WIN954.0_DBG.OBJ/nssutil3.dll -MAP WIN954.0_DBG.OBJ\\quickder.obj WIN954.0_DBG.OBJ\\secdig.obj WIN954.0_DBG.OBJ\\derdec.obj WIN954.0_DBG.OBJ\\derenc.obj WIN954.0_DBG.OBJ\\dersubr.obj WIN954.0_DBG.OBJ\\dertime.obj WIN954.0_DBG.OBJ\\errstrs.obj WIN954.0_DBG.OBJ\\nssb64d.obj WIN954.0_DBG.OBJ\\nssb64e.obj WIN954.0_DBG.OBJ\\nssrwlk.obj WIN954.0_DBG.OBJ\\nssilock.obj WIN954.0_DBG.OBJ\\oidstring.obj WIN954.0_DBG.OBJ\\pkcs1sig.obj WIN954.0_DBG.OBJ\\portreg.obj WIN954.0_DBG.OBJ\\secalgid.obj WIN954.0_DBG.OBJ\\secasn1d.obj WIN954.0_DBG.OBJ\\secasn1e.obj WIN954.0_DBG.OBJ\\secasn1u.obj WIN954.0_DBG.OBJ\\secitem.obj WIN954.0_DBG.OBJ\\secload.obj WIN954.0_DBG.OBJ\\secoid.obj WIN954.0_DBG.OBJ\\sectime.obj WIN954.0_DBG.OBJ\\secport.obj WIN954.0_DBG.OBJ\\templates.obj WIN954.0_DBG.OBJ\\utf8.obj WIN954.0_DBG.OBJ\\utilmod.obj WIN954.0_DBG.OBJ\\utilpars.obj WIN954.0_DBG.OBJ\\pkcs11uri.obj ..\\..\\..\\dist\\WIN954.0_DBG.OBJ\\lib\\plc4.lib ..\\..\\..\\dist\\WIN954.0_DBG.OBJ\\lib\\plds4.lib ..\\..\\..\\dist\\WIN954.0_DBG.OBJ\\lib\\nspr4.lib WIN954.0_DBG.OBJ\\nssutil.res
if test -f WIN954.0_DBG.OBJ/nssutil3.dll.manifest; then \
mt.exe -NOLOGO -MANIFEST WIN954.0_DBG.OBJ/nssutil3.dll.manifest -OUTPUTRESOURCE:WIN954.0_DBG.OBJ/nssutil3.dll\;2; \
rm -f WIN954.0_DBG.OBJ/nssutil3.dll.manifest; \
fi
nsinstall -m 775 WIN954.0_DBG.OBJ/nssutil3.dll ../../../dist/WIN954.0_DBG.OBJ/lib
nsinstall -m 775 WIN954.0_DBG.OBJ/nssutil3.lib ../../../dist/WIN954.0_DBG.OBJ/lib
nsinstall: WIN954.0_DBG.OBJ\nssutil3.lib: No such file or directory
make[4]: *** [WIN954.0_DBG.OBJ/nssutil3.lib] Error 3
Maybe the script is assuming it can make a symlink from nssutil.lib to nssutil3.lib but it actually has to copy on Windows?
Assignee | ||
Comment 14•4 years ago
|
||
No. I made a mistake when handling the patches between my LibreOffice and the mozilla-build NSS setup. The linker command is missing the export script (-DEF:$(MAPFILE)). So I can't generate a default MAPFILE in the ruleset.mk, because that variable is already used earlier in arch.mk to modify the linker command. And I don't see an easy way to fix that, so I reverted all the MAPFILE cleanup changes and just left them in manifest.mn.
Then I saw the broken incremental build... and had to move more stuff ... and gave up on using the LibreOffice automatic CI to test my changes in a much more parallel environment then my laptop, as I always have to patch the old NSS release. I'm missing something like git rerere when transfering the patches from hg to git...
Comment 15•4 years ago
|
||
https://hg.mozilla.org/projects/nss/rev/cbb737bc6c0ccda559c9e7b34fd5d5004dbe3316
https://hg.mozilla.org/projects/nss/rev/45db681898be760e53c70206912456dd296bb629
https://hg.mozilla.org/projects/nss/rev/71dd05b782e4554b1e3b42ee3cde9028114fd3b8
https://hg.mozilla.org/projects/nss/rev/9b628d9c57e5c490a5bcc90005f13f366026ebb6
https://hg.mozilla.org/projects/nss/rev/877d721d93cdb151eb0ded41a1dc6de9e94ecad6
https://hg.mozilla.org/projects/nss/rev/dc1ef0faf4a6ca181486b6fe45bf63cd01e166bf
https://hg.mozilla.org/projects/nss/rev/7d285fe69c8c06a0cab4b97f40d16e4321594a72
https://hg.mozilla.org/projects/nss/rev/d1f9546272601278cf34a41815537db964313fef
https://hg.mozilla.org/projects/nss/rev/c3f11da5acfc4d7fb2bbf042bb74072b95b47b7f
Description
•