813746 - WebRTC unit tests break the build with --enable-shared-js

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P1)

Description

[Boris Zbarsky [:bzbarsky]]

Building inbound from rev d8e020333f08 works fine.

Building inbound from rev 31d01eb0baf3 fails while building the webrtc unit tests on OS X 10.8 like so:

Undefined symbols for architecture x86_64:
  "_MOZ_Z_crc32", referenced from:
      _MOZ_PNG_reset_crc in png.o
      _MOZ_PNG_calc_crc in png.o
  "_MOZ_Z_deflate", referenced from:
      _MOZ_PNG_write_flush in pngwrite.o
      _MOZ_PNG_write_finish_row in pngwutil.o
      _MOZ_PNG_write_filtered_row in pngwutil.o
  "_MOZ_Z_deflateEnd", referenced from:
      _MOZ_PNG_write_destroy in pngwrite.o
      _MOZ_PNG_zlib_claim in pngwutil.o
  "_MOZ_Z_deflateInit2_", referenced from:
      _MOZ_PNG_zlib_claim in pngwutil.o
  "_MOZ_Z_deflateReset", referenced from:
      _MOZ_PNG_zlib_release in pngwutil.o
  "_MOZ_Z_inflate", referenced from:
      _MOZ_PNG_proc_IDAT_data in pngpread.o
      _MOZ_PNG_inflate in pngrutil.o
  "_MOZ_Z_inflateEnd", referenced from:
      _MOZ_PNG_read_dest in pngread.o
  "_MOZ_Z_inflateInit_", referenced from:
      _MOZ_PNG_cr_read_str in pngread.o
  "_MOZ_Z_inflateReset", referenced from:
      _MOZ_PNG_reset_zs in png.o
      _MOZ_PNG_inflate in pngrutil.o
      _MOZ_APNG_prog_read_reset in pngrutil.o
  "_MOZ_Z_uncompress", referenced from:
      (anonymous namespace)::ProcessGeneric(ots::OpenTypeFile*, unsigned int, ots::OTSStream*, unsigned char const*, unsigned long, std::vector<(anonymous namespace)::OpenTypeTable, std::allocator<(anonymous namespace)::OpenTypeTable> > const&, ots::Buffer&) in ots.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Undefined symbols for architecture x86_64:
  "_MOZ_Z_crc32", referenced from:
      _MOZ_PNG_reset_crc in png.o
      _MOZ_PNG_calc_crc in png.o

make[5]: *** [mediaconduit_unittests] Error 1

and various similar stuff for other things in that dir.

Clearly this works on tbpl...  No idea what the difference is, but this is being a complete development blocker for me.  :(

Comment 1

[Ethan Hugg [:ehugg]]

Haven't been able to repro this yet.  I have a couple macs here and jesup, ekr and crypt have built this as well.  Could you post your .mozconfig and the output of "clang --version"?  Let me try to replicate what you have.  Thanks.

Comment 2

[Boris Zbarsky [:bzbarsky]]

mozilla% clang --version
clang version 3.2 (trunk 163716)
Target: x86_64-apple-darwin12.2.0
Thread model: posix

(this is a build from clang svn, from that revision).

mozconfig is (after sorting through all the includes and removing the commented-out bits):

mk_add_options MOZ_CO_PROJECT=suite,browser,mail,xulrunner,calendar
mk_add_options MOZ_CO_MODULE="mozilla/other-licenses/libart_lgpl mozilla/tools"
mk_add_options MOZ_MAKE_FLAGS="-s -j6"
export MOZ_MAKE_FLAGS="-s -j6"
mk_add_options CC="clang"
export CC="clang"
mk_add_options CXX="clang++"
export CXX="clang++"
ac_add_options --disable-javaxpcom
ac_add_options --enable-tests
ac_add_options --enable-debugger-info-modules
ac_add_options --enable-debug-symbols
ac_add_options --enable-accessibility
ac_add_options --enable-codesighs
ac_add_options --disable-install-strip
ac_add_options --enable-shared-js
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-firefox
ac_add_options --enable-application=browser
ac_add_options --enable-debug=-gdwarf-2
ac_add_options --disable-optimize

Comment 3

[Boris Zbarsky [:bzbarsky]]

Note that this may be affected by what libs are in the link path too, in theory, and in particular on macports stuff.. or something.  I'm not sure what all these tests are linking to.

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

Hmm.  I suspect either a side-effect of some option, or maybe some different libs as you say - but more likely a buildsystem/option issue.  C++ unit tests are a mess to link currently, so a fair bit of trial-and-error was needed to get things linking correctly - in particular the gkmedia/xul break.

Adding Ted and some build peers while I investigate more

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

bz: can you get us the build line that fails?  (Better yet, the output of doing a build.)  Those are supposed to be the exported symbols from zlib, so I want to see what zlib the link line is trying to include, and how it was built.

Thanks

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

I suspect it's --enable-shared-js:

In rules.mk: LIBS += .... $(if $(JS_SHARED_LIBRARY),,$(MOZ_ZLIB_LIBS))

Can you try without that?  Though the build output would be the most useful.

Comment 7

[Ethan Hugg [:ehugg]]

I've confirmed it's something about the mozconfig.  I can dupe the problem with the mozconfig above on my Mac.  I will do some more builds and confirm which line it is that causes it.

Comment 8

[Boris Zbarsky [:bzbarsky]]

Attachment: zipped-up build output, without -s

Let me know if you want me to try without -j too...

Comment 9

[Boris Zbarsky [:bzbarsky]]

> I suspect it's --enable-shared-js:

Looks like you're right.  I turned that off for now, but that will significantly complicate profiling, so it would be nice to turn it back on...  Furthermore, I believe that's the default configuration on at least Windows (though it's possible WebRTC does not have the same linking issues there).

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

Verified: it's --enable-shared-js (sorry; I said this earlier and didn't notice I mid-air-collided with Ethan).

We should be able to add the zlib library to the link for the mediaconduit_unittest for the shared-js case.

Comment 11

[Boris Zbarsky [:bzbarsky]]

Attachment: This fixes the problem for me

Comment 12

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 683897 [details] [diff] [review]
This fixes the problem for me

Review of attachment 683897 [details] [diff] [review]:
-----------------------------------------------------------------

Fails if you *don't* have --enable-shared-js

Comment 13

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Link the webrtc unit tests to ZLIB as needed.

I think this will work; testing

Comment 14

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 683981 [details] [diff] [review]
Link the webrtc unit tests to ZLIB as needed.

Tested on Linux with and without --enable-shared-js - works
(it was failing for me there as well)

Comment 15

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 683981 [details] [diff] [review]
Link the webrtc unit tests to ZLIB as needed.

r=me, but how did the other fail if I might ask?  I wouldn't have expected it to...

Comment 16

[Randell Jesup [:jesup] (needinfo me)]

Multiple definitions (since ZLIB is in some other make var if it's not shared-js)

Comment 17

[Boris Zbarsky [:bzbarsky]]

Hmm.  So you can't use -Lfoo twice with the same "foo" safely?  That's unfortunate.  :(

Comment 18

[Randell Jesup [:jesup] (needinfo me)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/cf68f801ea57

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/cf68f801ea57