574403 - Build failure with --enable-dtrace

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Steve Fink [:sfink] [:s:]]

Attachment: Build log

I am trying to use SystemTap's dtrace compatibility tools to compile with --enable-dtrace on x86_64. The compile fails with numerous errors like

  undefined reference to `function__info_semaphore'

The symbol is defined in mozjs-dtrace.o, which is linked into libmozjs.so, which is included in the compile command line. The problem is that although the above symbol (and several others) are exported in mozjs-dtrace.o, they are converted to local symbols in libmozjs.so:

  % nm mozjs-dtrace.o| fgrep function__info_
  0000000000000002 D function__info_semaphore
  % nm libmozjs.so | fgrep function__info_   
  00000000004b871e d function__info_semaphore

I have posted a question about this to http://stackoverflow.com/questions/3106302/force-linker-to-export-a-resolved-symbol-when-creating-shared-library where someone suggested it was a linker bug and to try the latest version of ld. I did, but it didn't help. (I don't understand the linker well enough to know how this is supposed to work.)

Comment 1

[Steve Fink [:sfink] [:s:]]

Attachment: Hack to make it work

Something very odd is going on during linking. If I relink and specify mozjs-dtrace.o again, then everything works. (Except you need to do the equivalent for libjs_static.a as well, but I think it's just missing the file in the first place.)

The attached patch is a horrible hack that forces it to do the extra linking. It's still unclear to me whether this is a linker bug or what.

Comment 2

[Steve Fink [:sfink] [:s:]]

I lied. The attached patch makes the js directory build, because it uses libjs_static.a. But there seems to be no easy way to update a .so with a new object file; the hack in the patch just creates a new .so containing only the .o I wanted added.

So currently the only workaround is to add mozjs-dtrace.o to every link line that needs it.

Comment 3

[Steve Fink [:sfink] [:s:]]

Attachment: Part 1: alternative hack

This plus the next patch will make it work in the ugliest way possible.

Comment 4

[Steve Fink [:sfink] [:s:]]

Attachment: Part 2: alternative hack

Nasty workaround to link in the object to the static lib. I'm sure there are better ways.

NOT FOR COMMIT

Comment 5

[Steve Fink [:sfink] [:s:]]

There is also a compile failure due to underparenthesized macros in <system/sdt.h>. These problems all only apply on non-OSX systems, since on OSX dtrace doesn't require the "dtrace -G" step where you pull out the locations of all the probe points into a separate object file.

Comment 6

[Steve Fink [:sfink] [:s:]]

Attachment: Overparenthesize probes to compensate for stap's dtrace headers

Comment 7

[Steve Fink [:sfink] [:s:]]

I should also mention that in some version of the build scripts, firefox-bin fails to include the dtrace probe symbols because it links with both libxul.so and libmozjs.so. The symbols are present and exported in libmozjs.so, but it doesn't link them in because the symbols that cause the necessary object files to be linked in are also present in libxul.so, so only it gets linked in and (those portions of) libmozjs.so are unused. But in libxul.so, the symbols are present but not exported.

I'm actively working on getting a build system that works with both OSX dtrace and systemtap dtrace.

Comment 8

[Steve Fink [:sfink] [:s:]]

Attachment: Fix --enable-dtrace build for both OSX and Linux

Things seemed to have changed. The attached patch is all that I needed to get things compiling with both systemtap on Linux and dtrace on OSX.

With systemtap, you can check whether the probes have been compiled in with:
  stap -L 'process("./js").mark("*")'
I notice that the 'const char *' parameters are not set to stap's 'string' type, but are instead all 'long'. They can be successfully used with eg 'user_string($arg1)' though. (I am using systemtap 1.4 on Fedora 14.)

I haven't figured out how to make OSX's dtrace display the probes yet. But it compiles and links successfully. I'll r? when I can figure out how to use it there.

Comment 9

[Steve Fink [:sfink] [:s:]]

Comment on attachment 546671 [details] [diff] [review]
Fix --enable-dtrace build for both OSX and Linux

Ok, I got dtrace on OSX to admit that the probes exist. It's not very smart about it; it mixes in IS_ENABLED() tests with the actual probes, which makes it really hard to restrict to the right probe (and you need to, if you want to use the probe arguments).

Anyway, this does seem to be working on both Linux and OSX. I just pushed to try and requested results to be posted here. But my own debug builds on OSX and Linux64 both worked.

I'm absolutely kinda sorta sure that this doesn't depend on the two cleanup patches I have ahead of this in my queue.

Comment 10

[Mozilla RelEng Bot]

Try run for 90830cd2cf4a is complete.
Detailed breakdown of the results available here:
    http://tbpl.mozilla.org/?tree=Try&rev=90830cd2cf4a
Results:
    success: 11
Total buildrequests: 11

Comment 11

[Steve Fink [:sfink] [:s:]]

(In reply to comment #10)
> Try run for 90830cd2cf4a is complete.
> Detailed breakdown of the results available here:
>     http://tbpl.mozilla.org/?tree=Try&rev=90830cd2cf4a
> Results:
>     success: 11
> Total buildrequests: 11

...and please ignore that happy output. I neglected to hg add my mozconfig-extra, so dtrace wasn't enabled anyway. I just kicked off a run with that corrected, so 671f2368f50c should show up here eventually.

Comment 12

[Mozilla RelEng Bot]

Try run for 671f2368f50c is complete.
Detailed breakdown of the results available here:
    http://tbpl.mozilla.org/?tree=Try&rev=671f2368f50c
Results:
    success: 3
    warnings: 1
    failure: 7
Total buildrequests: 11

Comment 13

[Steve Fink [:sfink] [:s:]]

Comment on attachment 546671 [details] [diff] [review]
Fix --enable-dtrace build for both OSX and Linux

* sfink learns about the drawbacks of having try server update bugs

Comment 15

[Steve Fink [:sfink] [:s:]]

Attachment: Fix --enable-dtrace build for both OSX and Linux

This passes try. (In fact, none of the previous try failures were the fault of this patch. The last problem seems to have gone away on its own recently. Hopefully this won't bounce and make me a liar.)

Note that bug 673753 added the systemtap devel package to all Linux build machines, so this should be enable-able now.

Comment 16

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

http://hg.mozilla.org/mozilla-central/rev/fcb383e5e61e