1288173 - stop exporting rust symbols from libxul

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Trevor Saunders (:tbsaunde)]

even with ltoing rust code we end up exporting a lot of rust symbols at this point mostly from the standard library.  We should fix that.  We could consider linker script tricks, but given the mangled C++ symbols we do want to export that isn't simple.

Related upstream bugs:
https://github.com/rust-lang/rust/issues/32887
https://github.com/rust-lang/rust/issues/33221

Comment 1

[Alex Crichton [:acrichto]]

Just to clarify, but with LTO enabled, could you past a list of rust symbols which are exported? (Just to know what we should be targeting)

Comment 2

[Nathan Froyd [:froydnj]]

(In reply to Alex Crichton [:acrichto] from comment #1)
> Just to clarify, but with LTO enabled, could you past a list of rust symbols
> which are exported? (Just to know what we should be targeting)

I tried this with my in-progress cargo patches and surprised to see only the following symbols that looked Rust-related pop out:

455705: 000000000302a4f0     6 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd11WEXITSTATUS20h661938ca194ff7cb0rbE
455706: 000000000302a5c0    41 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd5linux9CPU_EQUAL20ha7d252545865fa82cKbE
455707: 000000000302a540    58 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd5linux7CPU_SET20h1583ad6cc4610acaMIbE
455708: 000000000302a580    58 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd5linux9CPU_ISSET20h45a28aeb7cd80455tJbE
455709: 000000000302a510    35 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd5linux8CPU_ZERO20hc73111b1c496cbdatIbE
455710: 000000000302a4b0    40 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd7FD_ZERO20hd6078ec7f894c602srbE
455711: 000000000302a420    47 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd6FD_CLR20h3932b2d2c01891c73obE
455712: 000000000302a450    44 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd8FD_ISSET20h790945fb78c3071eRpbE
455713: 000000000302a480    45 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd6FD_SET20hd38e5c30a5f625b7GqbE
455714: 000000000302a500     6 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd8WTERMSIG20haeebfd03545b089fdsbE
455715: 000000000302a4e0     7 FUNC    GLOBAL DEFAULT   12 _ZN4unix6notbsd9WIFEXITED20h578e1093e2021c25NrbE
455716: 000000000302a390   122 FUNC    GLOBAL DEFAULT   12 __rust_reallocate
455717: 000000000302a350    52 FUNC    GLOBAL DEFAULT   12 __rust_allocate
455718: 000000000302a410     4 FUNC    GLOBAL DEFAULT   12 __rust_reallocate_inplace
455719: 00000000009cf678     5 FUNC    GLOBAL DEFAULT   12 __rust_deallocate
455720: 000000000302ed29   224 FUNC    GLOBAL DEFAULT   12 backtrace_create_state
455721: 00000000011f095a     4 FUNC    GLOBAL DEFAULT   12 __rust_usable_size
455722: 000000000302eb56   112 FUNC    GLOBAL DEFAULT   12 backtrace_pcinfo
455723: 0000000003023650     5 FUNC    GLOBAL DEFAULT   12 rust_eh_personality
455724: 000000000302ebc6   117 FUNC    GLOBAL DEFAULT   12 backtrace_syminfo
455725: 000000000302a110    17 FUNC    GLOBAL DEFAULT   12 rust_eh_personality_catch
455726: 000000000302a130    66 FUNC    GLOBAL DEFAULT   12 rust_begin_unwind
455727: 000000000302ecdf    74 FUNC    GLOBAL DEFAULT   12 backtrace_close
455728: 000000000302ec3b   164 FUNC    GLOBAL DEFAULT   12 backtrace_open
455729: 0000000003030445   481 FUNC    GLOBAL DEFAULT   12 backtrace_alloc
455730: 0000000003030147   352 FUNC    GLOBAL DEFAULT   12 backtrace_initialize
455731: 00000000030302a7   250 FUNC    GLOBAL DEFAULT   12 backtrace_get_view
455732: 0000000003030626   182 FUNC    GLOBAL DEFAULT   12 backtrace_free
455733: 00000000030303a1    92 FUNC    GLOBAL DEFAULT   12 backtrace_release_view
455734: 0000000003035cea   421 FUNC    GLOBAL DEFAULT   12 backtrace_qsort
455735: 0000000003030881    74 FUNC    GLOBAL DEFAULT   12 backtrace_vector_finish
455736: 0000000003035b8e   262 FUNC    GLOBAL DEFAULT   12 backtrace_dwarf_add
455737: 00000000030308cb   146 FUNC    GLOBAL DEFAULT   12 backtrace_vector_release
455738: 00000000030306dc   421 FUNC    GLOBAL DEFAULT   12 backtrace_vector_grow

This was on a release (--enable-optimize --disable-debug) build, and I'm pretty sure that we used to have thousands of Rust symbols exported.  Indeed, checking a nightly Linux build says that we have ~11k Rust-related symbols exported.

So perhaps moving to cargo compilation is going to address a lot of this?  Perhaps because we invoke cargo on our super-crate directly, so rustc sees all of our crates together, rather than the old model where we compiled a bunch of little static libraries and then linked those together?  Or we are somehow using the wrong flags on Nightly (which is a little worrisome...)?

Comment 3

[Nathan Froyd [:froydnj]]

Oh, I guess Nightly and related doesn't yet feature -C lto like the cargo builds do, so that would be a very notable difference.

Comment 4

[Alex Crichton [:acrichto]]

Awesome, thanks! Compiling with LTO will definitely reduce the number of exported symbols, so that's probably what's happening there. If necessary though we can try to figure out a solution where that's not necessary, just let us know if you need it to be so!

For the list you've given I think they're covered precisely by these three bugs:

* https://github.com/rust-lang/rust/issues/34984 - libbacktrace symbols
* https://github.com/rust-lang/rust/issues/34985 - weird ZN4 libc symbols
* https://github.com/rust-lang/rust/issues/34493#issuecomment-228837159 - other assorted rust symbols

None of those should be *too* onerous to implement, so if any of them turn into blockers we can be sure they're squared away quickly. Just let us know if they become pressing issues!

Comment 5

[Chris Peterson [:cpeterson]]

This bug is blocked by linking with rust -C lto (bug 1268547).

Comment 6

[Nicholas Nethercote [inactive]]

(In reply to Chris Peterson [:cpeterson] from comment #5)
> This bug is blocked by linking with rust -C lto (bug 1268547).

That bug was fixed a while ago. froydnj, what is the status of this bug now?

Comment 7

[Nathan Froyd [:froydnj]]

(In reply to Nicholas Nethercote [:njn] from comment #6)
> That bug was fixed a while ago. froydnj, what is the status of this bug now?

I thought that we had basically eliminated these, but I see a variety of Rust_Test-, encoding.rs-, Servo- and WebRender-related symbols exported on a recently nightly of mine.  I think these exist because they are |extern "C"| in Rust code, and Rust doesn't support making them hidden visibility, so we'll just have to live with them.

So in the absence of a good solution on the Rust side, I think we can close this bug.