link failure with rust on macosx: too many personality routines for compact unwind

RESOLVED FIXED

Status

Firefox Build System
General
RESOLVED FIXED
3 years ago
5 months ago

People

(Reporter: rillian, Assigned: rillian)

Tracking

Trunk
Dependency tree / graph

Firefox Tracking Flags

(firefox42 affected)

Details

I've not been able to run './mach gtest' on my Mac for the past week or so. I first noticed it on a topic branch and thought it was something with the tests I'd added, but now master fails, so filing a bug. Unfortunately I don't get a coherent error, only:

 0:08.71 ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
 0:08.71 ld: too many personality routines for compact unwind to encode for architecture x86_64
 0:08.71 clang: error: linker command failed with exit code 1 (use -v to see invocation)
 0:08.71 make[3]: *** [XUL] Error 1
 0:08.71 make[2]: *** [toolkit/library/gtest/target] Error 2
 0:08.71 make[1]: *** [gtestxul] Error 2
 0:08.71 make: *** [gtest] Error 2
(Assignee)

Comment 1

3 years ago
I believe the compact-unwind warning is normal.

Stack overflow suggests this might be a linker bug, but isn't very helpful. https://stackoverflow.com/questions/21150223/ld-warning-too-many-personality-routines-for-compact-unwind-to-encode
(Assignee)

Comment 2

3 years ago
This is MacOS X 10.10.4 (14E46)
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
(Assignee)

Comment 3

3 years ago
Seems to be related to linking rust code. If I remove --enable-rust from my mozconfig the link command succeeds.
Blocks: 1135640
It may be possible to pass -Wl,-no_compact_unwind to the linker to solve this (not tested), but Rust does indeed use its own personality routine different than C/C++. You may be able to also solve this by disabling landing pads in Rust, but that'll be pretty difficult right now.
(Assignee)

Comment 5

3 years ago
With more recent rust (1.3.0) and xcode (clang-700.0.72) I still get the compact unwind warning, but I also get duplicate symbol failures on my local mac build. Works on Linux. This is with ac_add_options --enable-macos-target=10.7. Not sure why this doesn't happen with 1.0.0+notls on the integration builds.

duplicate symbol _je_tsd_init_finish in:
 0:10.01     ../../../dom/media/gtest/libhello.a(tsd.pic.o)
 0:10.01     ../../../media/libstagefright/libMP4Metadata.a(tsd.pic.o)
 0:10.01 ld: 152 duplicate symbols for architecture x86_64
Hm that is indeed a little odd, does libstagefright include its own copy of jemalloc? With Rust 1.4+ (I believe) a staticlib (e.g. `libhello.a`) shouldn't include jemalloc so that may help in situations like this.
(Assignee)

Comment 7

2 years ago
Just an update that I still get this on my local mac builds (MacOS X 10.10.5, XCode 7.2.1) with both normal and gtest builds.
Summary: link failure with ./mach gtest on macosx → link failure with rust on macosx: too many personality routines for compact unwind
(Assignee)

Comment 8

2 years ago
Based on irc discussions when bholley hit this bug today, I pushed a build to try with --enable-macos-target=10.7 to see if that reproduces with the rust 1.6.0+notls we use there.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=96725bdfc1ff
(Assignee)

Comment 9

2 years ago
Just targetting 10.7 didn't trigger the issue with the 1.6.0+notls builds we use on integration. Now trying with upstream 1.8.0 stable. https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5c27e9f2ab1

tooltool upload manifest:

[
{
"version": "rustc 1.8.0 stable repack for linux64 host, linux64+macosx64 targets",
"algorithm": "sha512",
"visibility": "public",
"filename": "rustc-x86_64-unknown-linux-gnu+mac-repack.tar.bz2",
"unpack": true,
"digest": "f23cefa07c6aeaa7d8d38e326f9b4fc302d8db6c91c3e01ac2589dcbd8fa10cc078945a0a57f3532f0e8e736d2ca03cd41b7876ee12e9ecef6851f1728c93536",
"size": 145818464
},
{
"version": "rustc 1.8.0 stable repack for macosx64 host, macosx32+64 targets",
"algorithm": "sha512",
"visibility": "public",
"filename": "rustc-x86_64-apple-darwin-repack.tar.bz2",
"unpack": true,
"digest": "816c324199ce9b5a7b5a2871fb423b4974a07a7e144bcc75d92549f8e513e0d86e4649c0b88cdd27b716c73f1df921a9d77fe2848e4759eee7d69340f9ca2fea",
"size": 138578729
}
]
Got an assertion failure from ld which might shed more light on this:

 13:16:46     INFO -  ld: warning: too many personality routines for compact unwind to encode
 13:16:46     INFO -  Assertion failed: (_mode == modeFinalAddress), function finalAddress, file /SourceCache/ld64/ld64-123.2.1/src/ld/ld.hpp, line 573.
 13:16:46     INFO -  0  0x10d08671c  __assert_rtn + 76
 13:16:46     INFO -  1  0x10d0ff01c  ld::tool::OutputFile::addressOf(ld::Internal const&, ld::Fixup const*, ld::Atom const**) + 172
 13:16:46     INFO -  2  0x10d101a25  ld::tool::OutputFile::applyFixUps(ld::Internal&, unsigned long long, ld::Atom const*, unsigned char*) + 3909
 13:16:46     INFO -  3  0x10d0fdf70  ld::tool::OutputFile::writeOutputFile(ld::Internal&) + 816
 13:16:46     INFO -  4  0x10d0f6ab9  ld::tool::OutputFile::write(ld::Internal&) + 153
 13:16:46     INFO -  5  0x10d086caa  main + 1178
 13:16:46     INFO -  6  0x10d0752b4  start + 52
Same assertion with rustc 1.9.
I tried some try pushes with rustc 1.10 and -C panic=abort, with and without -Z no-landing-pads. This fixed my local issue, but didn't make a difference with the 10.7 debug builds on try; I get the same assertion as above. :(
Out of curiosity, is there a minimally reproducible way to see this error with just some native code locally? Also, you may have more luck with either compiling the standard library with `-C panic=abort` or compiling with LTO (which will have the same effect). The extra personality routines are probably coming from there
FWIW :kinetik pointed out that 'LDFLAGS="-Wl,-no_compact_unwind" ./mach build' does work around the problem on his local system, and it also works for me. I don't know if rustc has changed, or if I hadn't tried that flag in the right place before.
(In reply to Ralph Giles (:rillian) needinfo me from comment #14)
> FWIW :kinetik pointed out that 'LDFLAGS="-Wl,-no_compact_unwind" ./mach
> build' does work around the problem on his local system, and it also works
> for me. I don't know if rustc has changed, or if I hadn't tried that flag in
> the right place before.

What sort of impact does that have on binary size?
Similar output with a repack of 1.10.0:

14:49:15     INFO -  ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
14:49:15     INFO -  ld: warning: too many personality routines for compact unwind to encode
14:49:15     INFO -  Assertion failed: (_mode == modeFinalAddress), function finalAddress, file /SourceCache/ld64/ld64-123.2.1/src/ld/ld.hpp, line 573.
14:49:15     INFO -  0  0x10fd5371c  __assert_rtn + 76
14:49:15     INFO -  1  0x10fdcc01c  ld::tool::OutputFile::addressOf(ld::Internal const&, ld::Fixup const*, ld::Atom const**) + 172
14:49:15     INFO -  2  0x10fdcea25  ld::tool::OutputFile::applyFixUps(ld::Internal&, unsigned long long, ld::Atom const*, unsigned char*) + 3909
14:49:15     INFO -  3  0x10fdcaf70  ld::tool::OutputFile::writeOutputFile(ld::Internal&) + 816
14:49:15     INFO -  4  0x10fdc3ab9  ld::tool::OutputFile::write(ld::Internal&) + 153
14:49:15     INFO -  5  0x10fd53caa  main + 1178
14:49:15     INFO -  6  0x10fd422b4  start + 52
14:49:15     INFO -  clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
14:49:15     INFO -  make[5]: *** [XUL] Error 1
(In reply to Nathan Froyd [:froydnj] [away 9 July through 20 July] from comment #15)

> > 'LDFLAGS="-Wl,-no_compact_unwind" ./mach build'
> 
> What sort of impact does that have on binary size?

Sorry, missed this before.

On my mac (MacOS X 10.10.5, XCode 7.2.1) building with `ac_add_options --enable-debug --disable-optimize` I get: 363251656 bytes for XUL with LDLAGS set, vs 363317200 bytes without. A difference of 64 kB, or 0.02%.

I didn't test an optimized build, but I don't get link error on try with optimized builds, so it's debug we currently need to worry about.
This *may* be fixed by https://github.com/rust-lang/rust/pull/34832 where Rust currently has two personality routines but once that lands on OSX at least we'll only have one.
That patch does fix the issue for my local build. Thanks for the heads up!
The patch also resolves the issue on our try builders. Huzzah!

Push at https://treeherder.mozilla.org/#/jobs?repo=try&revision=13a4160e8160 which also contains other unrelated patches.
(Assignee)

Updated

2 years ago
Blocks: 1249511
Confirmed that the new 1.12 official nightly build "rustc 1.12.0-nightly (9316ae515 2016-07-24)" resolves the issue for me locally.

Try push with a repack: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a19c7ba28041
Assignee: nobody → giles
Depends on: 1289583
Hmm, this version still isn't working. I won't pursue it further until 1.12 is in beta.
(Assignee)

Updated

2 years ago
Depends on: 1289847
On my Mac OSX, I also get this type of error. The workaround at comment 14 is work for me. After updating my rustc to rustc 1.13.0-nightly, I needn't this workaround anymore.
Thanks for confirming. Does updating to 1.12-beta or 1.11 stable also remove the need for the workaround?

If you use https://rustup.rs/ you can test this temporarily by:

rustup toolchain install beta
rustup run beta $PWD/mach build
(In reply to Ralph Giles (:rillian) needinfo me from comment #25)
> Thanks for confirming. Does updating to 1.12-beta or 1.11 stable also remove
> the need for the workaround?
> 
> If you use https://rustup.rs/ you can test this temporarily by:
> 
> rustup toolchain install beta
> rustup run beta $PWD/mach build

I originally use 1.12.0-nightly, probably 1.12-beta and 1.11 stable would have the same problem. But, it is worth to have a try.
Blocks: 1301001
Given that we require Rust 1.13 now, I assume we can mark this FIXED?
I haven't seen it in a while, so works for me.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED

Updated

5 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.