Closed Bug 1054034 Opened 11 years ago Closed 7 years ago

build fails due to "NSModules are not ordered appropriately"

Categories

(Toolkit Graveyard :: Build Config, defect)

Product:
Toolkit Graveyard
Component:
Build Config
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla63

People

(Reporter: aceman, Unassigned)

References

Details

(Whiteboard: [fixed by bug 1470127])

Build fails for me on Linux, compiling for 32bit on a 64bit system. I tried the normal script for full build, but it failed without any visible error message. After running 'make -C objdir/mozilla/toolkit/library' exclusively the error message was unveiled. 1st problem: after linking I get this error: test "$(nm -g libxul.so | grep _NSModule$ | sort | sed -n 's/^.* _*\([^ ]*\)$/\1/;1p;$p' | xargs echo)" != "start_kPStaticModules_NSModule end_kPStaticModules_NSModule" && echo "NSModules are not ordered appropriately" && exit 1 || exit 0 ; test "$(readelf -l libxul.so | awk 'libxul.so == "LOAD" { t += 1 } END { print t }')" -le 1 && echo "Only one PT_LOAD segment" && exit 1 || exit 0 NSModules are not ordered appropriately make[1]: *** [libxul.so] Error 1 make[1]: *** Deleting file `libxul.so' 2nd problem: ld.gold is always used even though /usr/bin/ld points to ld.bfd (the original ld linker). But replacing objdir/mozilla/build/unix/gold/ld with ld.bfd DOES use the correct linker now, but does not fix the 1st problem. It just takes much longer.
Are you building with lto?
I think lto is the default, but I specifically disabled it in .mozconfig. I use gcc 4.8.3 (cross-compiling to x86 32bit).
May this be caused by config options in .mozconfig? I have some unneded stuff disabled there. But I tried to enable everything (except some stuff for which I do not have libs) but it doesn't seem to help.
My gcc is 4.8.2 and compiling 64bit on a 64bit system works. Why do you need to crosscompile to 32bit? For a while options from .mozconfig were totally ignored, but that bug is fixed now.
Isn't this meant to be in thunderbird : build config?
(In reply to Magnus Melin from comment #4) > My gcc is 4.8.2 and compiling 64bit on a 64bit system works. Why do you need > to crosscompile to 32bit? Because I have choosen to have only 32bit apps/libs for now. Only the kernel is 64bit. (In reply to Magnus Melin from comment #5) > Isn't this meant to be in thunderbird : build config? That is not sure yet. It fails after linking libxul.so. So there is a chance it comes from m-c. I don't know if we take some parts of the build code from m-c or if c-c's build code is duplicated from m-c but separate.
(In reply to Magnus Melin from comment #5) > Isn't this meant to be in thunderbird : build config? This failure is purely related to libxul magic, most likely.
The failing check is doing this: nm -g libxul.so | grep '_NSModule$' | grep -vw refptr | sort | sed -n 's/^.* _*\([^ ]*\)$/\1/;1p;$p' | xargs echo The result of this on my machine is: necko_NSModule end_kPStaticModules_NSModule while the check expects: start_kPStaticModules_NSModule end_kPStaticModules_NSModule There is no "start_kPStaticModules_NSModule" symbol in the output of "nm -g libxul.so" as far as I can see. This symbol should come from the file StaticXULComponentsStart.cpp, which is there and I see no obvious problems in moz.build that would somehow skip it on my platform. I have temporarily disabled this check in mozilla/toolkit/library/libxul.mk and the build finished fine and the application is now running. But I can't say much about the correctness of it, just that most of our testsuite (xpcshell and mozmill) passes.
Having a similar issue, I'm able use: --- toolkit/library/libxul.mk.orig 2015-06-24 21:28:03.000000000 +0000 +++ toolkit/library/libxul.mk @@ -55,7 +55,7 @@ endif ifdef _MSC_VER get_first_and_last = dumpbin -exports $1 | grep _NSModule@@ | sort -k 3 | sed -n 's/^.*?\([^@]*\)@@.*$$/\1/;1p;$$p' else -get_first_and_last = $(TOOLCHAIN_PREFIX)nm -g $1 | grep _NSModule$$ | grep -vw refptr | sort | sed -n 's/^.* _*\([^ ]*\)$$/\1/;1p;$$p' +get_first_and_last = $(TOOLCHAIN_PREFIX)nm -Pgv $1 | grep _NSModule | egrep 'start|end' | cut -f1 -d' ' endif LOCAL_CHECKS = test "$$($(get_first_and_last) | xargs echo)" != "start_kPStaticModules_NSModule end_kPStaticModules_NSModule" && echo "NSModules are not ordered appropriately" && exit 1 || exit 0
(In reply to Richard PALO from comment #9) > Having a similar issue, I'm able use: > > --- toolkit/library/libxul.mk.orig 2015-06-24 21:28:03.000000000 +0000 > +++ toolkit/library/libxul.mk > @@ -55,7 +55,7 @@ endif > ifdef _MSC_VER > get_first_and_last = dumpbin -exports $1 | grep _NSModule@@ | sort -k 3 | > sed -n 's/^.*?\([^@]*\)@@.*$$/\1/;1p;$$p' > else > -get_first_and_last = $(TOOLCHAIN_PREFIX)nm -g $1 | grep _NSModule$$ | grep > -vw refptr | sort | sed -n 's/^.* _*\([^ ]*\)$$/\1/;1p;$$p' > +get_first_and_last = $(TOOLCHAIN_PREFIX)nm -Pgv $1 | grep _NSModule | egrep > 'start|end' | cut -f1 -d' ' This breaks the guarantee that there aren't other NSModules before start or after end, and that start appears before end. Also there is a reason for the refptr exclusion. See bug 940208. Also, your own symptoms are different than what this bug was filed for. Please file a separate one.
I believe it is the same as I had somewhat similar output...beans, that is. $ gnm -g libxul.so | grep '_NSModule$' | grep -vw refptr | sort | nbsed -n 's/^.* _*\([^ ]*\)$/\1/;1p;$p' start_kPStaticModules_NSModule nsGIOModule_NSModule nm '-v' means to sort external symbols by value before they are printed sort, by consequence, is unnecessary. with respect to 'grep -vw refptr', being a noop (I believe) on non windows platforms was an omission on my part to put back. It appears to me that the key, on ELF platforms anyway, is that these come from the read-only sections 'R' so perhaps to select things correctly from: $ gnm -Pgv libxul.so | grep '_NSModule'| grep -vw refptr start_kPStaticModules_NSModule R 0383cb24 00000004 end_kPStaticModules_NSModule R 0383cb28 00000004 nsPrefModule_NSModule D 03c8cfe0 00000004 nsUConvModule_NSModule D 03c8cfe4 00000004 nsI18nModule_NSModule D 03c8cfe8 00000004 necko_NSModule D 03c8cfec 00000004 nsAuthModule_NSModule D 03c8cff0 00000004 nsChardetModule_NSModule D 03c8cff4 00000004 nsJarModule_NSModule D 03c8cff8 00000004 ZipWriterModule_NSModule D 03c8cffc 00000004 mozStorageModule_NSModule D 03c8d000 00000004 nsCookieModule_NSModule D 03c8d004 00000004 nsPermissionsModule_NSModule D 03c8d008 00000004 nsRDFModule_NSModule D 03c8d00c 00000004 nsParserModule_NSModule D 03c8d010 00000004 nsGfxModule_NSModule D 03c8d014 00000004 nsIconDecoderModule_NSModule D 03c8d018 00000004 nsImageLib2Module_NSModule D 03c8d01c 00000004 nsPluginModule_NSModule D 03c8d020 00000004 nsContentProcessWidgetModule_NSModule D 03c8d024 00000004 nsWidgetGtk2Module_NSModule D 03c8d028 00000004 nsTransactionManagerModule_NSModule D 03c8d02c 00000004 nsComposerModule_NSModule D 03c8d030 00000004 nsLayoutModule_NSModule D 03c8d034 00000004 docshell_provider_NSModule D 03c8d038 00000004 embedcomponents_NSModule D 03c8d03c 00000004 Browser_Embedding_Module_NSModule D 03c8d040 00000004 appshell_NSModule D 03c8d044 00000004 nsUniversalCharDetModule_NSModule D 03c8d048 00000004 nsWindowDataSourceModule_NSModule D 03c8d04c 00000004 application_NSModule D 03c8d050 00000004 mozSpellCheckerModule_NSModule D 03c8d054 00000004 BOOT_NSModule D 03c8d058 00000004 NSS_NSModule D 03c8d05c 00000004 PKI_NSModule D 03c8d060 00000004 RemoteServiceModule_NSModule D 03c8d064 00000004 CommandLineModule_NSModule D 03c8d068 00000004 DiskSpaceWatcherModule_NSModule D 03c8d06c 00000004 nsFileViewModule_NSModule D 03c8d070 00000004 nsMediaSnifferModule_NSModule D 03c8d074 00000004 jsperf_NSModule D 03c8d078 00000004 nsPlacesModule_NSModule D 03c8d07c 00000004 jsreflect_NSModule D 03c8d080 00000004 nsTelemetryModule_NSModule D 03c8d084 00000004 jsctypes_NSModule D 03c8d088 00000004 tkAutoCompleteModule_NSModule D 03c8d08c 00000004 satchel_NSModule D 03c8d090 00000004 nsToolkitCompsModule_NSModule D 03c8d094 00000004 _ZN7mozilla31LocalCertServiceModule_NSModuleE D 03c8d098 00000004 jsinspector_NSModule D 03c8d09c 00000004 identity_NSModule D 03c8d0a0 00000004 Apprunner_NSModule D 03c8d0a4 00000004 nsUnixProxyModule_NSModule D 03c8d0a8 00000004 nsAutoConfigModule_NSModule D 03c8d0ac 00000004 nsServicesCryptoModule_NSModule D 03c8d0b0 00000004 StartupCacheModule_NSModule D 03c8d0b4 00000004 jsdebugger_NSModule D 03c8d0b8 00000004 nsGIOModule_NSModule D 03c8d0bc 00000004 $ gnm -Pgv libxul.so | grep '_NSModule'| grep -vw refptr |grep -vw D |cut -f1 -d' ' |xargs echo start_kPStaticModules_NSModule end_kPStaticModules_NSModule unfortunately, this is gnu nm specific and may not work with native nm such as solaris nm which doesn't seem to differentiate here read-only data from read-write data.
(In reply to Richard PALO from comment #11) > I believe it is the same as I had somewhat similar output...beans, that is. comment 8 says there was no start symbol *at all*. That's a different issue. > $ gnm -g libxul.so | grep '_NSModule$' | grep -vw refptr | sort | nbsed -n > 's/^.* _*\([^ ]*\)$/\1/;1p;$p' > start_kPStaticModules_NSModule > nsGIOModule_NSModule > > nm '-v' means to sort external symbols by value before they are printed > sort, by consequence, is unnecessary. > > with respect to 'grep -vw refptr', being a noop (I believe) on non windows > platforms was an omission on > my part to put back. > > It appears to me that the key, on ELF platforms anyway, is that these come > from the read-only sections 'R' > so perhaps to select things correctly from: > $ gnm -Pgv libxul.so | grep '_NSModule'| grep -vw refptr > start_kPStaticModules_NSModule R 0383cb24 00000004 > end_kPStaticModules_NSModule R 0383cb28 00000004 > nsPrefModule_NSModule D 03c8cfe0 00000004 > nsUConvModule_NSModule D 03c8cfe4 00000004 > nsI18nModule_NSModule D 03c8cfe8 00000004 > necko_NSModule D 03c8cfec 00000004 > nsAuthModule_NSModule D 03c8cff0 00000004 > nsChardetModule_NSModule D 03c8cff4 00000004 > nsJarModule_NSModule D 03c8cff8 00000004 > ZipWriterModule_NSModule D 03c8cffc 00000004 > mozStorageModule_NSModule D 03c8d000 00000004 > nsCookieModule_NSModule D 03c8d004 00000004 > nsPermissionsModule_NSModule D 03c8d008 00000004 > nsRDFModule_NSModule D 03c8d00c 00000004 > nsParserModule_NSModule D 03c8d010 00000004 > nsGfxModule_NSModule D 03c8d014 00000004 > nsIconDecoderModule_NSModule D 03c8d018 00000004 > nsImageLib2Module_NSModule D 03c8d01c 00000004 > nsPluginModule_NSModule D 03c8d020 00000004 > nsContentProcessWidgetModule_NSModule D 03c8d024 00000004 > nsWidgetGtk2Module_NSModule D 03c8d028 00000004 > nsTransactionManagerModule_NSModule D 03c8d02c 00000004 > nsComposerModule_NSModule D 03c8d030 00000004 > nsLayoutModule_NSModule D 03c8d034 00000004 > docshell_provider_NSModule D 03c8d038 00000004 > embedcomponents_NSModule D 03c8d03c 00000004 > Browser_Embedding_Module_NSModule D 03c8d040 00000004 > appshell_NSModule D 03c8d044 00000004 > nsUniversalCharDetModule_NSModule D 03c8d048 00000004 > nsWindowDataSourceModule_NSModule D 03c8d04c 00000004 > application_NSModule D 03c8d050 00000004 > mozSpellCheckerModule_NSModule D 03c8d054 00000004 > BOOT_NSModule D 03c8d058 00000004 > NSS_NSModule D 03c8d05c 00000004 > PKI_NSModule D 03c8d060 00000004 > RemoteServiceModule_NSModule D 03c8d064 00000004 > CommandLineModule_NSModule D 03c8d068 00000004 > DiskSpaceWatcherModule_NSModule D 03c8d06c 00000004 > nsFileViewModule_NSModule D 03c8d070 00000004 > nsMediaSnifferModule_NSModule D 03c8d074 00000004 > jsperf_NSModule D 03c8d078 00000004 > nsPlacesModule_NSModule D 03c8d07c 00000004 > jsreflect_NSModule D 03c8d080 00000004 > nsTelemetryModule_NSModule D 03c8d084 00000004 > jsctypes_NSModule D 03c8d088 00000004 > tkAutoCompleteModule_NSModule D 03c8d08c 00000004 > satchel_NSModule D 03c8d090 00000004 > nsToolkitCompsModule_NSModule D 03c8d094 00000004 > _ZN7mozilla31LocalCertServiceModule_NSModuleE D 03c8d098 00000004 > jsinspector_NSModule D 03c8d09c 00000004 > identity_NSModule D 03c8d0a0 00000004 > Apprunner_NSModule D 03c8d0a4 00000004 > nsUnixProxyModule_NSModule D 03c8d0a8 00000004 > nsAutoConfigModule_NSModule D 03c8d0ac 00000004 > nsServicesCryptoModule_NSModule D 03c8d0b0 00000004 > StartupCacheModule_NSModule D 03c8d0b4 00000004 > jsdebugger_NSModule D 03c8d0b8 00000004 > nsGIOModule_NSModule D 03c8d0bc 00000004 > > $ gnm -Pgv libxul.so | grep '_NSModule'| grep -vw refptr |grep -vw D |cut > -f1 -d' ' |xargs echo > start_kPStaticModules_NSModule end_kPStaticModules_NSModule > > unfortunately, this is gnu nm specific and may not work with native nm such > as solaris nm > which doesn't seem to differentiate here read-only data from read-write data. You're looking at it the wrong way. Yes, the command needs fixing to work with solaris's nm, but the fact is, the NSModule symbols are not ordered properly in the first place. Look at the addresses. start and end are *not* surrounding the others.
$ gobjdump -h libxul.so |ggrep -A1 kPSt 20 .kPStaticModules 00000008 0383cb24 0383cb24 0383cb24 2**2 CONTENTS, ALLOC, LOAD, READONLY, DATA -- 220 .kPStaticModules 000000e0 03c8cfe0 03c8cfe0 03c7cfe0 2**2 CONTENTS, ALLOC, LOAD, DATA Seems a merge of these sections is expected whereas, if I recall correctly, an ABI-conforming object file may not depend on specific elements being merged, and an ABI-conforming link editor may choose not to merge specific elements.
I've hit this same issue trying to build mozilla-release on OpenSUSE 13.1 x86_64. Everything is stock - no modifications to the build environment, all libraries from the official OpenSUSE repos and versions higher than the listed required versions. Not cross-compiling - default 64-bit build on 64-bit. If I comment out the two LOCAL_CHECK lines in libxul.mk (mentioned above - thanks), then the build completes and I can run Firefox. Why has this been left broken for 6 months? That's ridiculous. Everything stock. gcc version: "cc (SUSE Linux) 4.8.1 20130909 [gcc-4_8-branch revision 202388]" nm version: "NU nm (GNU Binutils; openSUSE 13.1) 2.23.2"
I ran in to this today on Windows too.
It happens on my Windows 7 Desktop and Windows 10 VM too. Does anyone have any suggestions to resolve this?
I'm seeing this on my windows 10 box too. I don't use it much. Is there some dependency change?
Sorry, I don't remember what fixed it for me. Maybe a clobber build but I assume you have tried that.
This has happened to me permanently all the time, even when no longer cross-compiling, but compiling normal x86_64 build on Linux x86_64. But today it worked to build libxul without having to remove the modules order check. It seems bug 1470127 may have fixed the logic of the test.
Status: NEW → RESOLVED
Closed: 7 years ago
Depends on: 1470127
Resolution: --- → FIXED
Whiteboard: [fixed by bug 1470127]
Target Milestone: --- → mozilla63
Product: Toolkit → Toolkit Graveyard
You need to log in before you can comment on or make changes to this bug.