1391023 - Update Mac builds to the OS X 10.12 SDK (or newer) when possible

Status: RESOLVED DUPLICATE of bug 1475652

(Firefox Build System :: General, defect)

Description

[(not currently active) Ted Mielczarek]

In bug 1324892 we're going to update the OS X SDK to the 10.11 SDK. We can't currently use the 10.12 SDK because Apple hasn't open sourced the version of their linker that can read the .tbd files in the 10.12 SDK, and we need the open source version so we can build a Linux binary to use in our cross-compile environment.

Once that becomes available we should try updating our cctools and then updating the OS X SDK to 10.12 (or whatever is available at that time).

Comment 1

[(not currently active) Ted Mielczarek]

jwatt asked if I could comment here with information on how we will know when the appropriate version of Apple's linker is open-sourced. I don't have a great answer for you, unfortunately. We're currently using ld64 274.2 from https://github.com/tpoechtrager/cctools-port .

Apple publishes their open source packages here:
https://opensource.apple.com/

Clicking on the newest link under "Developer Tools" brings you to:
https://opensource.apple.com/release/developer-tools-821.html

Which links to the version we're using:
https://opensource.apple.com/source/ld64/ld64-274.2/

The version of ld64 on my Mac running XCode 9.0 is:

mbp:~ luser$ ld -v
@(#)PROGRAM:ld  PROJECT:ld64-302.3
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
LTO support using: LLVM version 9.0.0, (clang-900.0.37) (static support for 21, runtime is 21)
TAPI support using: Apple TAPI version 900.0.14 (tapi-900.0.14)

Comment 2

[(not currently active) Ted Mielczarek]

Although I just noticed this commit, which claims to have added support for these newer-format tbd stubs in cctools-port:
https://github.com/tpoechtrager/cctools-port/commit/22ebe727a5cdc21059d45313cf52b4882157f6f0

We'd have to tweak our build script to build Apple's libtapi first:
https://github.com/tpoechtrager/cctools-port#installation

We should try that, it might unblock this.

Comment 3

[(not currently active) Ted Mielczarek]

If someone does get around to trying the suggested fixes in comment 2, I had previously uploaded a 10.12 SDK to tooltool, you could update the in-tree tooltool manifests to use it to validate that the new toolchain works:
https://bug1324892.bmoattachments.org/attachment.cgi?id=8820705

Comment 4

[Hiroyuki Ikezoe (:hiro)]

I did try it and got an error;
 ld: unexpected token: !tapi-tbd-v2 file '/builds/worker/workspace/build/src/MacOSX10.12.sdk/usr/lib/libSystem.tbd' for architecture x86_64

https://treeherder.mozilla.org/#/jobs?repo=try&revision=96dad4f8ecdfd5ca3d8997ac703284816f0cdd5f&selectedJob=161066151

If I understand correctly what we need to do, we also need to update cctools-port to this revision?  But I don't know how to do that.
 https://github.com/tpoechtrager/cctools-port/commit/22ebe727a5cdc21059d45313cf52b4882157f6f0

Comment 5

[Nathan Froyd [:froydnj]]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #4)
> If I understand correctly what we need to do, we also need to update
> cctools-port to this revision?  But I don't know how to do that.
>  https://github.com/tpoechtrager/cctools-port/commit/
> 22ebe727a5cdc21059d45313cf52b4882157f6f0

You can update the git commit hash here:

http://dxr.mozilla.org/mozilla-central/source/taskcluster/scripts/misc/build-cctools-port-macosx.sh#9

and push to try, and a rebuild of cctools will happen automatically, and builds will then use that new version of cctools.

Comment 6

[Nathan Froyd [:froydnj]]

Possibly also here as well:

http://dxr.mozilla.org/mozilla-central/source/taskcluster/scripts/misc/build-cctools-port.sh#9

Comment 7

[Hiroyuki Ikezoe (:hiro)]

Thank you Nathan.  That's a fancy way. :)

I had tried to update cctools as well, but hasn't succeed to make the apple-libtapi build on Linux yet.

Comment 8

[Gregory Szorc [:gps]]

Note that we have bug 1475652 tracking moving to the 10.14 SDK. That bug tree is a bit more active than this one. Perhaps we should dupe this?

Comment 9

[Hiroyuki Ikezoe (:hiro)]

Oh nice.  Let's dupe then.