Enable rust for macosx64 nightly/aurora builds

RESOLVED FIXED in Firefox 42

Status

()

Core
Build Config
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: rillian, Assigned: rillian)

Tracking

Trunk
mozilla42
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox42 fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Like 1175359 for linux, we want to have rust enabled on integration builds so we get some testing of code landing in the tree.
(Assignee)

Comment 1

2 years ago
While linux builds are blocked on updating the valgrind test script, I've been trying to work on this. Currently stuck on the rust 1.1.0 toolchain I built for mac with --disable-elf-tls segfaulting on the builders. I don't know why.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e0915c60fa15
Assignee: nobody → giles
(Assignee)

Updated

2 years ago
Depends on: 1183864
I'm not sure I quite know how to read the output of those builds, I couldn't find a segfault in the logs, although I'm probably looking in the wrong place! Are there some logs or dumps or stack traces I could take a peek at? (or reproduce locally as well)
(Assignee)

Comment 3

2 years ago
I have a loaner machine now. Official builds work, but my --disable-elf-tls build fails with SIGILL.

$ DYLD_LIBRARY_PATH=$PWD/lib bin/rustc --version
Illegal instruction: 4

$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz
$ uname -a
Darwin bld-lion-r5-004.build.releng.scl3.mozilla.com 11.2.0 Darwin Kernel Version 11.2.0: Tue Aug  9 20:54:00 PDT 2011; root:xnu-1699.24.8~1/RELEASE_X86_64 x86_64

Darwin 11.2 is MacOS X 10.7 (Mac OS X Server 10.7.2 (11C74) in this case.)

gdb fails before it gets to main with "unable to read unknown load command 0x80000028"

Both builds work on my 10.10 MacBook Pro. Suggestions for further debugging?
Flags: needinfo?(acrichton)
(Assignee)

Comment 4

2 years ago
Build I'm trying is also available at https://people.mozilla.org/~rgiles/2015/rustc-1.1.0-notls-x86_64-apple-darwin.tar.bz2
Ok I can confirm that build segfaults on a 10.7 OSX machine (some of our bots are running 10.7). The issue I see is:

$ lldb ./bin/rustc
Current executable set to './bin/rustc' (x86_64).
(lldb) r
Process 713 launched: './bin/rustc' (x86_64)
Process 713 stopped
* thread #1: tid = 0x1f03, 0x00007fff5fc01028 dyld`_dyld_start, stop reason = EXC_BAD_ACCESS (code=1, address=0xfffffffffffffff8)
    frame #0: 0x00007fff5fc01028 dyld`_dyld_start
dyld`_dyld_start:
-> 0x7fff5fc01028:  pushq  $0
   0x7fff5fc0102a:  movq   %rsp, %rbp
   0x7fff5fc0102d:  andq   $-16, %rsp
   0x7fff5fc01031:  movq   8(%rbp), %rdi
(lldb) print/x $rsp
(unsigned long) $0 = 0x0000000000000000

That's unfortunately not super informative so I'm not 100% sure what's going on here. My best guess is that the compiler was built on a newer version of OSX and may be using some weird option? Did you build the compiler itself on 10.7? 10.10? We build all our nightlies on 10.7 and that may be why we haven't run into this before perhaps.
Flags: needinfo?(acrichton)
Note we're really close to switching mac builds to be cross-compiled from linux... So however you fix this is going to be obsoleted quite quickly since the osx rust compiler is not going to work in that context. Ted will have a better ETA to give.
Flags: needinfo?(ted)
(Assignee)

Comment 7

2 years ago
Yes, I built on 10.10. I can try to do a build on 10.7 or 10.8. Which toolchain do you use to build llvm? The stock clang on the 10.7 loaner is too old.
I think we updated xcode somewhat manually at some point, but the clang that we're running on 10.7 is:

$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
(Assignee)

Comment 9

2 years ago
Hmm. The loaner machine came with Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn) which is definitely older.

Our tooltool releng.manifest has clang version 3.3 (branches/release_33 183744). It isn't necessarily older(?) but doesn't work either. Configure accepts it, but it can't compile some of the llvm source; const expression issues.

However, setting MACOSX_DEPLOYMENT_TARGET=10.7 when I build rust on my 10.10 machine produces working binaries, so I can just do that. Thanks for the help, Alex!
Oh wow, that's... incredibly useful to know! I wonder if we could start moving to 10.10 for our builders now... Glad it worked out!

Comment 11

2 years ago
(In reply to Mike Hommey [:glandium] from comment #6)
> Note we're really close to switching mac builds to be cross-compiled from
> linux... So however you fix this is going to be obsoleted quite quickly
> since the osx rust compiler is not going to work in that context. Ted will
> have a better ETA to give.

Mike, that is so exciting. Is there any chance the Rust team can get access to your Linux->Mac cross toolchain so we can start working on support for it in the Rust toolchain? We are very eager to create a Rust development environment where one can cross from Linux to *everything*.
(In reply to Mike Hommey [:glandium] from comment #6)
> Note we're really close to switching mac builds to be cross-compiled from
> linux... So however you fix this is going to be obsoleted quite quickly
> since the osx rust compiler is not going to work in that context. Ted will
> have a better ETA to give.

I'm planning on having cross-compile builds stood up this quarter, but I don't know whether they'll replace the existing builds or not.

To make this work there we'll need to sort out a cross-rustc that can run on Linux and produce Mac-compatible code. (Hopefully not that big a deal as it doesn't have to link anything in this situation.)
> To make this work there we'll need to sort out a cross-rustc that can run on Linux and produce Mac-compatible code. (Hopefully not that big a deal as it doesn't have to link anything in this situation.)

This should be as simple as downloading a few things and passing an extra argument or two, but having never been done before I'm sure there's a bug or two lurking! If you guys have a toolchain we can play around with though I'd love to iron out any bugs!
(In reply to Ralph Giles (:rillian) from comment #9)

> However, setting MACOSX_DEPLOYMENT_TARGET=10.7 when I build rust on my 10.10
> machine produces working binaries, so I can just do that.

Confirming this build of rust master (1.3.0-dev) works on try.

10:34:28     INFO - TEST-START | rust.MP4MetadataEmpty
10:34:28     INFO - TEST-PASS | rust.MP4MetadataEmpty | test completed (time: 0ms)
10:34:28     INFO - TEST-START | rust.MP4Metadata
10:34:28     INFO - 'ftyp' 24 bytes 'MP42' v0
10:34:28     INFO - TEST-PASS | rust.MP4Metadata | test completed (time: 19ms)
10:34:28     INFO - TEST-START | rust.CallFromCpp
10:34:28     INFO - TEST-PASS | rust.CallFromCpp | test completed (time: 0ms)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=59d0632b0298
Created attachment 8635539 [details] [diff] [review]
Part 1 - Add rust to macosx64 tooltool manifest
Attachment #8635539 - Flags: review?(mshal)
Created attachment 8635542 [details] [diff] [review]
Part 2 - Enable rust for macosx64 Nightly/Dev Edition

Turn on rust for macosx64 official builds. I'm skeptical this will work without figuring something out for 32-bit mac as well, but it's green on try, so inbound is the next thing to try. :-)

Glandium, are you ok with just Nightly/Aurora until we resolve the memory issues you raised? That will get us testing without untracked allocations making it to release.
Attachment #8635542 - Flags: review?(mshal)
Attachment #8635542 - Flags: feedback?(mh+mozilla)

Updated

2 years ago
Attachment #8635539 - Flags: review?(mshal) → review+
Comment on attachment 8635542 [details] [diff] [review]
Part 2 - Enable rust for macosx64 Nightly/Dev Edition

>Subject: Bug 1183850 - Enable rust for macosx64 Nightly and Dev. r=ted

Did you intend for ted to review this instead?

>diff --git a/build/macosx/mozconfig.rust.macosx b/build/macosx/mozconfig.rust.macosx

nit: You can probably just call this file 'mozconfig.rust' since you're already in the macosx directory.
Attachment #8635542 - Flags: review?(mshal) → review+
(In reply to Michael Shal [:mshal] from comment #17)

> >Subject: Bug 1183850 - Enable rust for macosx64 Nightly and Dev. r=ted
> 
> Did you intend for ted to review this instead?

Ted's been reviewing the earlier work, but I didn't notice he was only vacation until after I'd uploaded the patch. This is similar to what he r+'d for linux, so I think he'll be ok with it.

> nit: You can probably just call this file 'mozconfig.rust' since you're
> already in the macosx directory.

Ok. I'll rename.
Created attachment 8636182 [details] [diff] [review]
Part 2 - Enable rust for macosx64 Nightly/Dev Edition v2

Update for nits. Carrying forward r=mshal
Attachment #8635542 - Attachment is obsolete: true
Attachment #8635542 - Flags: feedback?(mh+mozilla)
Attachment #8636182 - Flags: review+

Comment 20

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/3c73a85e2304
https://hg.mozilla.org/integration/mozilla-inbound/rev/6b1f724e69f2
https://hg.mozilla.org/mozilla-central/rev/3c73a85e2304
https://hg.mozilla.org/mozilla-central/rev/6b1f724e69f2
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox42: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla42
(In reply to Ralph Giles (:rillian) from comment #16)
> Glandium, are you ok with just Nightly/Aurora until we resolve the memory
> issues you raised? That will get us testing without untracked allocations
> making it to release.

Didn't even get to respond to this after the week-end and it's already landed, and the f? request gone so that didn't pop in my dashboard...

So the answer is: it would be better if it stayed out of aurora.
(In reply to Mike Hommey [:glandium] from comment #6)
> Note we're really close to switching mac builds to be cross-compiled from
> linux... So however you fix this is going to be obsoleted quite quickly
> since the osx rust compiler is not going to work in that context. Ted will
> have a better ETA to give.

I apparently didn't clear the needinfo back when I made this comment, but I've filed bug 1197256 on making rust work in cross-mac builds.
Flags: needinfo?(ted)
You need to log in before you can comment on or make changes to this bug.