702179 - dtrace probes are not installed on Solaris with static js lib

Status: RESOLVED INACTIVE

(Firefox Build System :: General: Unsupported Platforms, defect, P5)

Description

[Ginn Chen]

Build Firefox with static js lib and enable-dtrace on Solaris.
Run Firefox.
Run dtrace -l | grep javascript

Actual Results: Get nothing.

The problem is mozjs-dtrace.o has symbols of dtrace_dof_init and dtrace_dof_fini.
But it is not linked into libxul.so.

It got lost during mozjs-dtrace.o->libjs_static.a->libxul.so.

By default, archive members are extracted to satisfy undefined references and to promote tentative definitions with data definitions.

So we need to use allextract for libjs_static.a, or add mozjs-dtrace.o to libxul.so.

I don't know if we need to similar fix it on BSD or Linux.
And I don't know what to do if we have another module with dtrace probe obj.

Comment 1

[Ginn Chen]

Attachment: patch

I know it is ugly, but it is what I have for now.

Comment 2

[(not currently active) Ted Mielczarek]

Note that we don't actually link the static library anymore, we create fake static libraries (libmozjs_static.a.desc) that just list the object files contained within. I think we can probably get rid of all the special-case handling for DTrace and just include the probes directly in the "static library" we create.

Comment 3

[Mike Hommey [:glandium] OoO until Dec 30]

We also use the actual static library when both the fake and the static lib exist. For consistency, in that case, we should add MKSHLIB_FORCE_ALL and MKSHLIB_UNFORCE_ALL, but it turns out this breaks things that do expect unused stuff to be magically eliminated (iirc, this happens mostly with the xpcom glue).

The "easy" fix is to stop building the real static library.

Comment 4

[Ginn Chen]

OK, that will fix it.

I tried MKSHLIB_FORCE_ALL and MKSHLIB_UNFORCE_ALL for js, the generated libxul is sightly larger than just adding mozjs-dtrace.o.

Comment 5

[Ginn Chen]

Attachment: patch v2

stop building real .a for js_static
also fix MOZ_JS_SHARED_LIBS

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 574538 [details] [diff] [review]
patch v2

Review of attachment 574538 [details] [diff] [review]:
-----------------------------------------------------------------

I'm going to pass this one to glandium.

Comment 7

[Mike Hommey [:glandium] OoO until Dec 30]

Comment on attachment 574538 [details] [diff] [review]
patch v2

Review of attachment 574538 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.in
@@ +1335,5 @@
>  MOZ_PNG_CFLAGS=
>  MOZ_PNG_LIBS='$(call EXPAND_LIBNAME_PATH,mozpng,$(DEPTH)/media/libpng)'
>  
> +MOZ_JS_STATIC_LIBS='$(call EXPAND_LIBNAME_PATH,js_static,$(DEPTH)/js/src)'
> +MOZ_JS_SHARED_LIBS='$(LIBXUL_DIST)/lib/$(DLL_PREFIX)mozjs$(DLL_SUFFIX)'

You don't need to change MOZ_JS_SHARED_LIBS.

Comment 8

[Ginn Chen]

(In reply to Mike Hommey [:glandium] from comment #7)

> You don't need to change MOZ_JS_SHARED_LIBS.

You're right.

I didn't know we have scripts to change it to libmozjs.so.

Comment 9

[Ginn Chen]

http://hg.mozilla.org/integration/mozilla-inbound/rev/281583d67072

Comment 10

[Mike Hommey [:glandium] OoO until Dec 30]

Attachment: Additional change

Bug 673616 made a (wrong) change that broke win64 when this bug landed. I think this additional patch should solve the issue. I'll push on try.

Comment 11

[Marco Bonardo [:mak]]

backed out for comment 10

Comment 12

[Mike Hommey [:glandium] OoO until Dec 30]

Try run looks good (with the typo fixed in the patch I attached)

https://tbpl.mozilla.org/?tree=Try&rev=cff48b989bcd

Comment 13

[Ginn Chen]

Reland with additional fix

http://hg.mozilla.org/integration/mozilla-inbound/rev/1d0ec7fa8b96

Comment 14

[Ed Morley [:emorley]]

Backed out for build PGO Linux build failures:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=f154f18b320a
https://tbpl.mozilla.org/php/getParsedLog.php?id=8087854&tree=Mozilla-Inbound
{
/tools/python/bin/python2.5 /builds/slave/m-in-lnx64-pgo/build/config/pythonpath.py -I../../config /builds/slave/m-in-lnx64-pgo/build/config/expandlibs_exec.py --uselist --  <<snip>>
g++: error trying to exec '/tools/gcc-4.5-0moz3/libexec/gcc/x86_64-unknown-linux-gnu/4.5.2/collect2': execv: Argument list too long
}

Comment 15

[Ed Morley [:emorley]]

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/856b3079c55c

Comment 16

[Mike Hommey [:glandium] OoO until Dec 30]

(In reply to Ed Morley [:edmorley] from comment #14)
> Backed out for build PGO Linux build failures:
> https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=f154f18b320a
> https://tbpl.mozilla.org/php/getParsedLog.php?id=8087854&tree=Mozilla-Inbound
> {
> /tools/python/bin/python2.5
> /builds/slave/m-in-lnx64-pgo/build/config/pythonpath.py -I../../config
> /builds/slave/m-in-lnx64-pgo/build/config/expandlibs_exec.py --uselist -- 
> <<snip>>
> g++: error trying to exec
> '/tools/gcc-4.5-0moz3/libexec/gcc/x86_64-unknown-linux-gnu/4.5.2/collect2':
> execv: Argument list too long
> }

O_o that's exactly the kind of error that expandlibs was designed to avoid

Comment 17

[Ginn Chen]

(In reply to Mike Hommey [:glandium] from comment #16)

> > g++: error trying to exec
> > '/tools/gcc-4.5-0moz3/libexec/gcc/x86_64-unknown-linux-gnu/4.5.2/collect2':
> > execv: Argument list too long
> > }
> 
> O_o that's exactly the kind of error that expandlibs was designed to avoid

I guess gcc extracts the list when it executes collect2.

Comment 18

[Mike Hommey [:glandium] OoO until Dec 30]

collect2 is not supposed to see the list, it should be in a linker script, thus seen by ld directly.

Comment 19

[Mike Hommey [:glandium] OoO until Dec 30]

even better, it just built on try...

Comment 20

[Ginn Chen]

Reland after Bug 719742 is fixed.

Comment 21

[Makoto Kato [:m_kato]]

backout
https://hg.mozilla.org/integration/mozilla-inbound/rev/5cb20ac871d0

Windows build uses shared js...

TestRacingServiceManager.exe : fatal error LNK1120: 1 unresolved externals
make[6]: *** [TestRacingServiceManager.exe] Error 96
make[5]: *** [tools] Error 2
make[4]: *** [tools_tier_platform] Error 2
make[3]: *** [tier_platform] Error 2
make[2]: *** [default] Error 2
make[1]: *** [realbuild] Error 2
make: *** [build] Error 2

MacOSX ...

make[7]: *** [TestGetURL] Error 1
make[7]: *** Waiting for unfinished jobs....
make[7]: *** [TestNativeXMLHttpRequest] Error 1
make[6]: *** [tools] Error 2
make[5]: *** [base_tools] Error 2
make[4]: *** [tools_tier_platform] Error 2
make[3]: *** [tier_platform] Error 2
make[2]: *** [default] Error 2
make[1]: *** [realbuild] Error 2
make: *** [build] Error 2

Comment 22

[Ginn Chen]

So we do require $(MOZ_JS_LIBS) for debug build now, even no JS_SHARED_LIBRARY or MOZ_SHARK. (for JS_Assert)

I can't remember why it broke win64.

Comment 23

[Ginn Chen]

Here's the error message with Win64 opt, without the additional fix.

c:/mozilla-build/python/python2.6.exe /e/builds/moz2_slave/try-w64/build/config/pythonpath.py -I../../../config /e/builds/moz2_slave/try-w64/build/config/expandlibs_exec.py --uselist -- link -nologo -out:TestCOM.exe -pdb:TestCOM.pdb TestCOM.obj  -LARGEADDRESSAWARE -NXCOMPAT -DYNAMICBASE  -DEBUG -DEBUGTYPE:CV -DEBUG -OPT:REF,ICF   -LIBPATH:../../../dist/lib -NODEFAULTLIB:msvcrt -NODEFAULTLIB:msvcrtd -NODEFAULTLIB:msvcprt -NODEFAULTLIB:msvcprtd -DEFAULTLIB:mozcrt e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcomglue_s.lib e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcom.lib e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/mozalloc.lib e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/nspr4.lib e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plc4.lib e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plds4.lib  ../../../js/src/js_static.lib  rpcrt4.lib uuid.lib advapi32.lib
   Creating library TestCOM.lib and object TestCOM.exp
prmjtime.obj : error LNK2019: unresolved external symbol __imp_timeEndPeriod referenced in function "void __cdecl NowCalibrate(void)" (?NowCalibrate@@YAXXZ)
prmjtime.obj : error LNK2019: unresolved external symbol __imp_timeBeginPeriod referenced in function "void __cdecl NowCalibrate(void)" (?NowCalibrate@@YAXXZ)
TestCOM.exe : fatal error LNK1120: 2 unresolved externals

So the problem is:
MOZ_ASSERT uses JS_Assert for debug builds, we need link js libs for that.
If we use js_static.lib on Win64, we need Winmm.lib.

Comment 24

[Mike Hommey [:glandium] OoO until Dec 30]

So it all boils down to bug 717540

Comment 25

[Makoto Kato [:m_kato]]

(In reply to Ginn Chen from comment #23)
> Here's the error message with Win64 opt, without the additional fix.
> 
> c:/mozilla-build/python/python2.6.exe
> /e/builds/moz2_slave/try-w64/build/config/pythonpath.py -I../../../config
> /e/builds/moz2_slave/try-w64/build/config/expandlibs_exec.py --uselist --
> link -nologo -out:TestCOM.exe -pdb:TestCOM.pdb TestCOM.obj 
> -LARGEADDRESSAWARE -NXCOMPAT -DYNAMICBASE  -DEBUG -DEBUGTYPE:CV -DEBUG
> -OPT:REF,ICF   -LIBPATH:../../../dist/lib -NODEFAULTLIB:msvcrt
> -NODEFAULTLIB:msvcrtd -NODEFAULTLIB:msvcprt -NODEFAULTLIB:msvcprtd
> -DEFAULTLIB:mozcrt
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcomglue_s.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcom.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/mozalloc.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/nspr4.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plc4.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plds4.lib 
> ../../../js/src/js_static.lib  rpcrt4.lib uuid.lib advapi32.lib
>    Creating library TestCOM.lib and object TestCOM.exp
> prmjtime.obj : error LNK2019: unresolved external symbol __imp_timeEndPeriod
> referenced in function "void __cdecl NowCalibrate(void)"
> (?NowCalibrate@@YAXXZ)
> prmjtime.obj : error LNK2019: unresolved external symbol
> __imp_timeBeginPeriod referenced in function "void __cdecl
> NowCalibrate(void)" (?NowCalibrate@@YAXXZ)
> TestCOM.exe : fatal error LNK1120: 2 unresolved externals
> 
> So the problem is:
> MOZ_ASSERT uses JS_Assert for debug builds, we need link js libs for that.
> If we use js_static.lib on Win64, we need Winmm.lib.

(In reply to Ginn Chen from comment #23)
> Here's the error message with Win64 opt, without the additional fix.
> 
> c:/mozilla-build/python/python2.6.exe
> /e/builds/moz2_slave/try-w64/build/config/pythonpath.py -I../../../config
> /e/builds/moz2_slave/try-w64/build/config/expandlibs_exec.py --uselist --
> link -nologo -out:TestCOM.exe -pdb:TestCOM.pdb TestCOM.obj 
> -LARGEADDRESSAWARE -NXCOMPAT -DYNAMICBASE  -DEBUG -DEBUGTYPE:CV -DEBUG
> -OPT:REF,ICF   -LIBPATH:../../../dist/lib -NODEFAULTLIB:msvcrt
> -NODEFAULTLIB:msvcrtd -NODEFAULTLIB:msvcprt -NODEFAULTLIB:msvcprtd
> -DEFAULTLIB:mozcrt
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcomglue_s.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/xpcom.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/mozalloc.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/nspr4.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plc4.lib
> e:/builds/moz2_slave/try-w64/build/obj-firefox/dist/lib/plds4.lib 
> ../../../js/src/js_static.lib  rpcrt4.lib uuid.lib advapi32.lib
>    Creating library TestCOM.lib and object TestCOM.exp
> prmjtime.obj : error LNK2019: unresolved external symbol __imp_timeEndPeriod
> referenced in function "void __cdecl NowCalibrate(void)"
> (?NowCalibrate@@YAXXZ)
> prmjtime.obj : error LNK2019: unresolved external symbol
> __imp_timeBeginPeriod referenced in function "void __cdecl
> NowCalibrate(void)" (?NowCalibrate@@YAXXZ)
> TestCOM.exe : fatal error LNK1120: 2 unresolved externals
> 
> So the problem is:
> MOZ_ASSERT uses JS_Assert for debug builds, we need link js libs for that.
> If we use js_static.lib on Win64, we need Winmm.lib.

Win64 is --disable-shared-js as default.  (Win32 is --enable-shared-js as default).  If using --disable-shared-js on Win32, this error will occurs.  Also, this option will turn on after switching to VS2010SP1.

Also, this error means that winmm.lib is requried for linking.

Comment 26

[Jan Beich]

(In reply to Ginn Chen from comment #0)
> I don't know if we need to similar fix it on BSD or Linux.

Confirming the issue for FreeBSD. Not sure in what shape userland DTrace is on NetBSD.

Comment 27

[Jan Beich]

comment 9 before/after bug 1036894. "after" version regresses bug 1041936 comment 10.

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=d163f60763e5
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=6bdb85ae34bf

Comment 28

[Jan Beich]

Attachment: Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

Review commit: https://reviewboard.mozilla.org/r/115138/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115138/

Comment 29

[Jan Beich]

Comment on attachment 8840706 [details]
Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

https://reviewboard.mozilla.org/r/115138/#review116614

Comment 30

[Jan Beich]

Comment on attachment 8840706 [details]
Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

https://reviewboard.mozilla.org/r/115138/#review116614

[PGO Try build](https://treeherder.mozilla.org/#/jobs?repo=try&revision=35741a4c871e1487d040a5e2aa81480e9a231c20)

Comment 31

[Jan Beich]

Comment on attachment 8840706 [details]
Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

https://reviewboard.mozilla.org/r/115136/#review116646

Comment 32

[Jan Beich]

Comment on attachment 8840706 [details]
Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

https://reviewboard.mozilla.org/r/115138/#review116648

`make check` on Linux and mozjs-crate debug build failed.

```
obj-firefox/_virtualenv/bin/python config/check_vanilla_allocations.py libjs_static.a
Traceback (most recent call last):
  File "config/check_vanilla_allocations.py", line 190, in <module>
    main()
  File "config/check_vanilla_allocations.py", line 81, in main
    stderr=subprocess.PIPE).split('\n')
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['nm', '-u', '-C', '-A', 'libjs_static.a']' returned non-zero exit status 1
Makefile:53: recipe for target 'check-vanilla-allocations' failed
make[1]: *** [check-vanilla-allocations] Error 1
make[1]: Target 'check' not remade because of errors.
```

```
+ cargo build --verbose --frozen --features debugmozjs
[...]
     Running `rustc --crate-name mozjs_sys lib.rs --crate-type lib -g --cfg feature=\"debugmozjs\" -C metadata=5b0cf026770f624a -C extra-filename=-5b0cf026770f624a --out-dir js/src/target/debug/deps --emit=dep-info,link -L dependency=js/src/target/debug/deps --extern libc=js/src/target/debug/deps/liblibc-83c2bd88b43ecde3.rlib --extern libz_sys=js/src/target/debug/deps/liblibz_sys-16ebe4b239f83619.rlib -L native=js/src/target/debug/build/mozjs_sys-f2c6bbd7d5d251d7/out/js/src -l stdc++ -l static=js_static -L native=/usr/lib64`
error: could not find native static library `js_static`, perhaps an -L flag is missing?

error: Could not compile `mozjs_sys`.
```

Comment 33

[Jan Beich]

Comment on attachment 8840706 [details]
Bug 702179 - Don't build static JS library to fix DTrace probes on Solaris and FreeBSD.

No longer builds after bug 1063728:

gmake[2]: Entering directory 'obj-x86_64-unknown-freebsd12.0/js/src/build'
gmake[2]: *** No rule to make target 'libjs_static.a', needed by 'spidermonkey_checks'.  Stop.

Comment 34

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:mhentges, could you have a look please?
For more information, please visit auto_nag documentation.