Cross rustc for mac builds

RESOLVED FIXED in Firefox 47

Status

()

Core
Build Config
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: froydnj, Assigned: froydnj)

Tracking

Trunk
mozilla47
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox47 fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Splitting this off from bug 1220307 for the cross-mac case.  I think it's simpler to do this to an existing platform (mac), rather than trying to add support for a new platform (android) on top of changes for mac.
No longer depends on: 1220307
Packaging up our linux64 rustc with an osx stdlib didn't work; the configure test for a working cross rustc failed with an unknown file format error when linking.  I assume that's because the linux64 rustc only knows how to emit ELF files and not Mach-O files?

Next step is to compile our Linux64 rust with explicit osx target support and use that compiler instead.
All Rust compilers actually support emitting all targets, so in theory if the linux64 compiler you had was cross compiling to OSX it should have emitted a Mach-O file. Cross compilers, however, aren't very well configured by default with respect to the linker they should invoke, so you'd probably need to pass `-C linker=` to change away from the default (`ld`).

Could you gist the linker error? I may be able to recognize a thing or two in it.
(In reply to Alex Crichton [:acrichto] from comment #2)
> All Rust compilers actually support emitting all targets, so in theory if
> the linux64 compiler you had was cross compiling to OSX it should have
> emitted a Mach-O file. Cross compilers, however, aren't very well configured
> by default with respect to the linker they should invoke, so you'd probably
> need to pass `-C linker=` to change away from the default (`ld`).

Ah, good point!  Forgot about that bit.

> Could you gist the linker error? I may be able to recognize a thing or two
> in it.

 16:18:21     INFO -  error: linking with `cc` failed: exit code: 1
 16:18:21     INFO -  note: "cc" "-m64" "-L" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib" "conftest.0.o" "-o" "conftest" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/libstd-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/libcollections-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/librustc_unicode-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/librand-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/liballoc-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/liblibc-71b07a99.rlib" "/home/worker/workspace/build/src/rustc/lib/rustlib/x86_64-apple-darwin/lib/libcore-71b07a99.rlib" "-l" "System" "-l" "pthread" "-l" "c" "-l" "m" "-l" "compiler-rt"
 16:18:21     INFO -  note: conftest.0.o: file not recognized: File format not recognized
 16:18:21     INFO -  collect2: ld returned 1 exit status
 16:18:21     INFO -  error: aborting due to previous error
 16:18:21     INFO -  configure: error: cannot compile for x86_64-apple-darwin with /home/worker/workspace/build/src/rustc/bin/rustc
Ah yeah that'd be it, I doubt that "cc" is configured to compile for OSX! Presumably the builders have a gcc-like compiler which can compile to OSX, right? If you pass `-C linker=path/to/that/cc` it should in theory work ok!
(In reply to Alex Crichton [:acrichto] from comment #4)
> Ah yeah that'd be it, I doubt that "cc" is configured to compile for OSX!
> Presumably the builders have a gcc-like compiler which can compile to OSX,
> right? If you pass `-C linker=path/to/that/cc` it should in theory work ok!

Hm, our cc on osx-on-linux builds isn't a single command, but rather clang invoked with some extra arguments.  Is there a good way to pass this into rustc?  I looked around for where -C linker gets processed for hints, but I couldn't find it.  I see in librustc_back things like pre_link_args, but I'm not sure there's any way to get things to there from the command line, or whether those arguments would be passed in the correct place on the command line...

8:31:58     INFO -  configure:4606: /home/worker/workspace/build/src/rustc/bin/rustc -C linker=/usr/bin/ccache /home/worker/workspace/build/src/clang/bin/clang -target x86_64-apple-darwin10 -mlinker-version=136 -B /home/worker/workspace/build/src/cctools/bin -isysroot /home/worker/workspace/build/src/MacOSX10.7.sdk --target x86_64-apple-darwin conftest.rs
18:31:58     INFO -  error: could not exec the linker `/usr/bin/ccache /home/worker/workspace/build/src/clang/bin/clang -target x86_64-apple-darwin10 -mlinker-version=136 -B /home/worker/workspace/build/src/cctools/bin -isysroot /home/worker/workspace/build/src/MacOSX10.7.sdk`: No such file or directory (os error 2)
18:31:58     INFO -  configure: error: cannot compile for x86_64-apple-darwin with /home/worker/workspace/build/src/rustc/bin/rustc
Flags: needinfo?(acrichton)
Ah yes for us when you pass `-C linker=foo` (regardless of whether `foo` has spaces in it or not) the compiler will try to execute that as a literal executable (e.g. spawn a process named `foo`). This'd explain at least the error you're seeing.

Right now there's a few options:

* The easiest thing to do will likely be to create a script that looks like `my list of arguments "$@"` and just pass that as `-C linker`
* You can pass the executable as `-C linker` and more arguments as `-C link-args`, but it means you probably can't use ccache due to how the link arguments are ordered. Note that you're probably not benefiting from ccache anyway either way.

Does that make sense?
Flags: needinfo?(acrichton)
We aren't building dynamic libraries, right? Why does rustc need a linker at all?
rustc doesn't need a linker to build an rlib, yeah, but judging from that error message it was requested that a binary was generated (hence why rustc attempted to call a linker)
The relevant test is:

    dnl Test C linkage against rust code to see if the rust
    dnl toolchain output is compatible.
    cat > conftest.rs <<EOF
      [#[no_mangle]]
      pub extern fn rusty_answer() -> u8 { 42 }
EOF
    ac_try="$RUSTC --crate-type staticlib -o conftest.a conftest.rs >/dev/null"
    AC_TRY_EVAL(ac_try)
    save_LDFLAGS=$LDFLAGS
    LDFLAGS="$LDFLAGS conftest.a -lpthread -lm"
    AC_TRY_LINK_FUNC([rusty_answer], [
      AC_MSG_RESULT([$MACOSX_DEPLOYMENT_TARGET is ok with this rustc])
    ], [
      AC_MSG_RESULT([cannot link for $MACOSX_DEPLOYMENT_TARGET])
      MOZ_RUST=
    ])
    LDFLAGS=$save_LDFLAGS
    rm -rf conftest*

The linker is *not* called by rustc.
There's that test; I was adding a test to make sure standard libraries for the requested --target exist, and using linking a binary to do it.  If I can get by with just compiling a staticlib instead (and that will glom in the relevant standard libraries where necessary), that would make life much easier.  Is that how things work?
Flags: needinfo?(acrichton)
Yeah that should do the trick! When building a staticlib rustc will try to find the standard library, so if it's not there it'll fail.
Flags: needinfo?(acrichton)
Created attachment 8714959 [details] [diff] [review]
part 1 - tooltool cross-rustc packages for OSX-on-Linux builds

These packages have been generated by careful editing of some packages already
in tooltool.  They can be generated automatically from Ralph's rust build
scripts, but I haven't made the necessary changes to said build scripts yet.
Attachment #8714959 - Flags: review?(mshal)
Created attachment 8714961 [details] [diff] [review]
part 2 - use common mozconfig.rust on OS X

Now that we have cross-compilation tooltool packages for OS X, we can
use the common mozconfig to enable Rust on all OS X builds, regardless
of host OS.

This patch depends on the patch from bug 1177599 to work properly, I think.
Attachment #8714961 - Flags: review?(mshal)
Created attachment 8714963 [details] [diff] [review]
part 2 - use common mozconfig.rust on OS X

Now with the actual removal of the OS X-specific mozconfig.rust.  Still depends
on bug 1177599.
Attachment #8714963 - Flags: review?(mshal)
Attachment #8714961 - Attachment is obsolete: true
Attachment #8714961 - Flags: review?(mshal)

Updated

2 years ago
Attachment #8714959 - Flags: review?(mshal) → review+
Comment on attachment 8714963 [details] [diff] [review]
part 2 - use common mozconfig.rust on OS X

woot!
Attachment #8714963 - Flags: review?(mshal) → review+

Comment 16

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/9f24d3935ab9
https://hg.mozilla.org/integration/mozilla-inbound/rev/18868fec4be0

Comment 17

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/9f24d3935ab9
https://hg.mozilla.org/mozilla-central/rev/18868fec4be0
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox47: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
You need to log in before you can comment on or make changes to this bug.