1475652 - (mojave-sdk) [meta] Build Firefox with macOS 10.14 SDK (CI is currently using the 10.11 SDK)

Status: RESOLVED DUPLICATE of bug 1696504

(Core :: Widget: Cocoa, enhancement, P2)

Description

[Tim Nguyen :ntim]

Apparently it gives some benefits in regards to dark mode integration, so it's probably worth doing.

Comment 1

[Stephen A Pohl [:spohl]]

(In reply to Tim Nguyen :ntim from comment #0)
> Apparently it gives some benefits in regards to dark mode integration, so
> it's probably worth doing.

What specific benefits are you referring to? Note that we're currently doing a lot of "non-standard" things, so building with the 10.14 SDK may not give us any benefits at all with respect to dark mode.

There are other bugs (such as bug 1470597) that may accelerate the timeline to build with the 10.14 SDK, but dark mode may not be it.

Comment 2

[Tim Nguyen :ntim]

(In reply to Stephen A Pohl [:spohl] from comment #1)
> (In reply to Tim Nguyen :ntim from comment #0)
> > Apparently it gives some benefits in regards to dark mode integration, so
> > it's probably worth doing.
> 
> What specific benefits are you referring to? Note that we're currently doing
> a lot of "non-standard" things, so building with the 10.14 SDK may not give
> us any benefits at all with respect to dark mode.
> 
> There are other bugs (such as bug 1470597) that may accelerate the timeline
> to build with the 10.14 SDK, but dark mode may not be it.

Dialogs and different -moz-appearance values get the dark mode integration automatically.

It looks like the profile selector/crash reporter already benefit from dark mode integration (with the right native textures) in my local build. There may be more, but unfortunately my build crashes so I don't know.

Comment 3

[Tim Nguyen :ntim]

To build with MacOS 10.14 SDK:
- Download Xcode beta
- Add this to mozconfig:

ac_add_options --with-macos-sdk=/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

Comment 4

[Tim Nguyen :ntim]

Note that you'll need to work around bug 1475654 by commenting out: https://searchfox.org/mozilla-central/rev/46292b1212d2d61d7b5a7df184406774727085b8/widget/cocoa/nsChildView.mm#1564

In order for the build to actually run.

It has a couple of bugs with the dark mode, but it also fixes the appearance of the Crash reporter, the "Save As" dialog and the "Print" dialog.

Comment 6

[Hiroyuki Ikezoe (:hiro)]

As far as I can tell the hardest part of this bug is to build on Linux.  We need some works in cctools.

Comment 7

[(not currently active) Ted Mielczarek]

I haven't looked into this lately, so bug 1391023 comment 2 is the last I knew of what we'd need to do that.

Comment 8

[Hiroyuki Ikezoe (:hiro)]

FWIW, building apple-libtapi with '/usr/bin/cc' and '/usr/bin/c++' causes some errors[1] like this;

 /builds/worker/workspace/apple-libtapi/src/apple-llvm/src/projects/libtapi/include/tapi/Core/Symbol.h:114:21: error: 'class std::map<tapi::internal::Arch, tapi::internal::AvailabilityInfo>' has no member named 'emplace'

I guess the compiler is older than expected one. 

Also, building the apple-libapi with clang (which is used for building cctools-port) causes other errors [2] like this;
 CMake Error at /usr/share/cmake-3.7/Modules/CMakeDetermineCCompiler.cmake:48 (message):
 Could not find compiler set in environment variable CC:

I have no idea what this error means.:/

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=f5f5b2af1046823d01423661345e3beb7499638c&selectedJob=196340566
[2] https://treeherder.mozilla.org/logviewer.html#?job_id=196303372&repo=try&lineNumber=183

Comment 9

[Mike Kaply [:mkaply]]

Is this still being prioritized?

Comment 11

[Hiroyuki Ikezoe (:hiro)]

This issue is no longer blocking my tasks. So to me it's not high priority but I am not sure for other people.

Comment 12

[Stephen A Pohl [:spohl]]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #11)

This issue is no longer blocking my tasks. So to me it's not high priority but I am not sure for other people.

This doesn't quite answer my question in comment 10. Also, not sure why that comment was marked private. I would need to know if all the tools necessary to cross-compile with the 10.15 SDK are available to us. In the past, we had to wait for Apple to open source the necessary tools. Do you happen to know if all the tools to cross-compile with the 10.15 SDK are available to us? If not, do you happen to know who might know? Thank you

Comment 13

[Hiroyuki Ikezoe (:hiro)]

I am afraid I don't know. I am not an expert about Mac OS at all. The reason I got involved this is that I thought we had to build our source code which is using newer APIs introduced 10.14 SDK on our CIs with cross-compling. But it's no longer issue for me.

I think you know about Mac better than me. :)

Comment 14

[Stephen A Pohl [:spohl]]

The question here is about cross-compilation on Linux. So if anyone on the CC list can chime in on the state of the required tools to cross-compile on Linux, it would be appreciated (Ted's status says that he's not currently active, unfortunately).

Comment 15

[Hiroyuki Ikezoe (:hiro)]

Bouncing to glandium.

Comment 16

[Mike Hommey [:glandium]]

Our cctools-port copy builds with tapi now, so presumably it might be possible? You'll only know if you try...
If someone with a real SDK 10.14 and/or 10.15 can tell whether the files in https://github.com/phracker/MacOSX-SDKs/ are pristine, that would be useful.

Comment 17

[Stephen A Pohl [:spohl]]

(In reply to Mike Hommey [:glandium] from comment #16)

Our cctools-port copy builds with tapi now, so presumably it might be possible? You'll only know if you try...

Is there a document that would tell me how to "try" this..? Or can you walk me through the steps? This is rather unfamiliar territory for me.

If someone with a real SDK 10.14 and/or 10.15 can tell whether the files in https://github.com/phracker/MacOSX-SDKs/ are pristine, that would be useful.

I have a real 10.15 SDK, but I seem to have a more recent version than the one on GitHub and there are tons of differences. I'm not sure how we will be able to tell that the copy is pristine, unless we know exactly what version of the 10.15 SDK to compare it to.

Comment 18

[Stephen A Pohl [:spohl]]

:hiro, I see that you have tried this back in bug 1391023 comment 4 and bug 1391023 comment 7. Could you walk me through the steps to verify that we have the necessary tools to crosscompile with the 10.14 or 10.15 SDK? And assuming that this is successful, do you know what I need to do to push a new SDK to the build machines? Thank you!

Comment 19

[Hiroyuki Ikezoe (:hiro)]

I actually couldn't recall what I did before, so I tried it again.

1. Clone https://github.com/tpoechtrager/apple-libtapi
2. pushd apple-libtapi && INSTALLPREFIX=$HOME/cctools ./build.sh && ./install.sh && popd
3. Clone https://github.com/tpoechtrager/cctools-port (we might need to use a specific revision)
4. pushd cctools-port/cctools && ./autogen.sh && ./configure --prefix=$HOME/cctools --with-libtapi=$HOME/cctools --target=x86_64-apple-darwin --enable-tapi-support --with-libtapi=$HOME/cctools --enable-lto-support && make install
5. ln -sf $HOME/cctools/bin/x86_64-apple-darwin-ld $HOME/cctools/bin/ld64 (this might be unnecessary)
6. Clone https://github.com/phracker/MacOSX-SDKs
7. rustup target add x86_64-apple-darwin
8. build firefox with below .mozconfig

.mozconfig

ac_add_options --target=x86_64-apple-darwin
ac_add_options --with-macos-sdk=$HOME/MacOSX-SDKs/MacOSX10.14.sdk
mk_add_options "export PATH=$PATH:$HOME/cctools/bin"

Now building seems to start working on my local Linux box (but it still takes a lot of times on toolkit/crashreporter/google-breakpad/src/tools/mac/dump_syms/dump_syms now, I mean the build hasn't finished yet).

Anyways, as glandium said in comment 16, cctools on CIs has been updated so it seems to me that all we have to do is to upload the new MacOSX tar ball to an appropriate place and use it, I don't know where the place is though, the way to generate the tar ball is well documented.

(In reply to Mike Hommey [:glandium] from comment #16)

If someone with a real SDK 10.14 and/or 10.15 can tell whether the files in https://github.com/phracker/MacOSX-SDKs/ are pristine, that would be useful.

A fun fact I noticed is that the SDK 10.14 in the repo was uploaded by kats, so it is a resonable one I believe.

Comment 20

[Hiroyuki Ikezoe (:hiro)]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #19)

Now building seems to start working on my local Linux box (but it still takes a lot of times on toolkit/crashreporter/google-breakpad/src/tools/mac/dump_syms/dump_syms now, I mean the build hasn't finished yet).

Caused an error

33:07.75 ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
33:08.60 Undefined symbols for architecture x86_64:
33:08.60   "___isOSVersionAtLeast", referenced from

No idea how we can fix this error and no idea this would happen on our CIs.

Comment 21

[Kartikaya Gupta (email:[e-mail])]

(In reply to Mike Hommey [:glandium] from comment #16)

If someone with a real SDK 10.14 and/or 10.15 can tell whether the files in https://github.com/phracker/MacOSX-SDKs/ are pristine, that would be useful.

FWIW the 10.14 SDK in the github repo was pristine at the time I submitted it (https://github.com/phracker/MacOSX-SDKs/pull/20) and it doesn't look like it's been touched since.

Comment 22

[Kartikaya Gupta (email:[e-mail])]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #20)

Caused an error

33:07.75 ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
33:08.60 Undefined symbols for architecture x86_64:
33:08.60   "___isOSVersionAtLeast", referenced from

No idea how we can fix this error and no idea this would happen on our CIs.

This sounds quite familiar. I feel like I saw this when building with the 10.14 SDK locally back when I was working on bug 1549052 and I must have found some way to fix it. I don't recall what it was but if you provide a bit more of the error message I might be able to help.

Comment 23

[Nathan Froyd [:froydnj]]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #20)

(In reply to Hiroyuki Ikezoe (:hiro) from comment #19)

Now building seems to start working on my local Linux box (but it still takes a lot of times on toolkit/crashreporter/google-breakpad/src/tools/mac/dump_syms/dump_syms now, I mean the build hasn't finished yet).

Caused an error

33:07.75 ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
33:08.60 Undefined symbols for architecture x86_64:
33:08.60   "___isOSVersionAtLeast", referenced from

No idea how we can fix this error and no idea this would happen on our CIs.

We'd need more context on the error (like what file it happened in) and if possible, the preprocessed source for the file to figure out what's going on (make file.i in the directory containing file.mm or similar). That symbol should be provided by the runtime libraries that we're linking in.

Comment 24

[Stephen A Pohl [:spohl]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #21)

(In reply to Mike Hommey [:glandium] from comment #16)

If someone with a real SDK 10.14 and/or 10.15 can tell whether the files in https://github.com/phracker/MacOSX-SDKs/ are pristine, that would be useful.

FWIW the 10.14 SDK in the github repo was pristine at the time I submitted it (https://github.com/phracker/MacOSX-SDKs/pull/20) and it doesn't look like it's been touched since.

We may want to check if building with the 10.15 SDK is possible. If so, I'd be happy to upload that and for us to switch straight to the 10.15 SDK on our build system.

Comment 25

[Kartikaya Gupta (email:[e-mail])]

I can try doing that build locally on macOS, I have a 10.15 SDK in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/. Don't know if that will automatically make things work when cross-compiling from Linux though.

Comment 26

[Hiroyuki Ikezoe (:hiro)]

Attachment: The build error log

This is the build error log including references.

Comment 27

[Kartikaya Gupta (email:[e-mail])]

My build with the 10.15 SDK failed with this:

29:07.12 /Users/kats/zspace/gecko-mac/memory/replace/logalloc/replay/Replay.cpp:332:18: error: 'aligned_alloc' is only available on macOS 10.15 or newer [-Werror,-Wunguarded-availability-new]
29:07.12     aSlot.mPtr = ::aligned_alloc_impl(alignment, size);
29:07.12                  ^~~~~~~~~~~~~~~~~~~~
29:07.13 /Users/kats/zspace/gecko-mac/obj-host-debugopt/dist/include/malloc_decls.h:59:1: note: 'aligned_alloc' has been marked as being introduced in macOS 10.15 here, but the deployment target is macOS 10.9.0
29:07.13 NOTHROW_MALLOC_DECL(aligned_alloc, void*, size_t, size_t)
29:07.13 ^
29:07.13 /Users/kats/zspace/gecko-mac/obj-host-debugopt/dist/include/malloc_decls.h:44:33: note: expanded from macro 'NOTHROW_MALLOC_DECL'
29:07.13 #    define NOTHROW_MALLOC_DECL MALLOC_DECL
29:07.13                                 ^
29:07.13 /Users/kats/zspace/gecko-mac/memory/replace/logalloc/replay/Replay.cpp:256:15: note: expanded from macro 'MALLOC_DECL'
29:07.13   return_type name##_impl(__VA_ARGS__);
29:07.13               ^
29:07.13 <scratch space>:18:1: note: expanded from here
29:07.13 aligned_alloc_impl
29:07.13 ^
29:07.13 /Users/kats/zspace/gecko-mac/obj-host-debugopt/dist/include/mozmemory_wrap.h:140:47: note: expanded from macro 'aligned_alloc_impl'
29:07.13 #define aligned_alloc_impl mozmem_malloc_impl(aligned_alloc)
29:07.13                                               ^
29:07.13 /Users/kats/zspace/gecko-mac/memory/replace/logalloc/replay/Replay.cpp:332:18: note: enclose 'aligned_alloc' in a __builtin_available check to silence this warning
29:07.13     aSlot.mPtr = ::aligned_alloc_impl(alignment, size);
29:07.13                  ^~~~~~~~~~~~~~~~~~~~
29:07.13 1 error generated.

Comment 28

[Kartikaya Gupta (email:[e-mail])]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #26)

This is the build error log including references.

Ah, so your linker errors are coming from the @available checks in the code, e.g. this one and the blame for that led me to bug 1564216 which has this relevant comment but I haven't read through the rest of the bug in detail. Presumably people who were involved that bug (glandium, spohl, etc.) know better what's going on there.

Comment 29

[Stephen A Pohl [:spohl]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #27)

My build with the 10.15 SDK failed with this:

Are you saying that a regular build on macOS with the 10.15 SDK is failing for you? Those builds should succeed as of bug 1583854, and they do work on my end.

Is there a way to get the 10.15 SDK into tooltool so that we can use try to test crosscompilation?

Comment 30

[Stephen A Pohl [:spohl]]

I had a conversation with :froydnj the other day and here is what was said as being necessary to start building with the 10.15 SDK on our build system:

you need to upload the new SDK into tooltool (https://mozilla-releng.net/tooltool/) and then modify the cross-*.manifest files in https://searchfox.org/mozilla-central/source/browser/config/tooltool-manifests/macosx64 to point at the new SDK

The one thing we haven't been able to figure out was how to upload the new SDK into tooltool. Tom, would you know?

Comment 31

[Kartikaya Gupta (email:[e-mail])]

(In reply to Stephen A Pohl [:spohl] from comment #29)

Are you saying that a regular build on macOS with the 10.15 SDK is failing for you? Those builds should succeed as of bug 1583854, and they do work on my end.

Yeah, it's a regular build on macOS that's failing. I can file a bug for that if we care enough to fix it. It might be something in my mozconfig. Unless we're building with that in automation I'm not expecting it to stay working in the long term though.

Comment 32

[Stephen A Pohl [:spohl]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #31)

(In reply to Stephen A Pohl [:spohl] from comment #29)

Are you saying that a regular build on macOS with the 10.15 SDK is failing for you? Those builds should succeed as of bug 1583854, and they do work on my end.

Yeah, it's a regular build on macOS that's failing. I can file a bug for that if we care enough to fix it. It might be something in my mozconfig. Unless we're building with that in automation I'm not expecting it to stay working in the long term though.

The goal is to move to the most recent version of the SDK that we can build with, i.e. hopefully 10.15. In the past we weren't always able to switch to the most recent version of the SDK due to some tools not having been open-sourced yet, which is what we're trying to figure out here. If I had to make a guess I'd say it is most likely something in your mozconfig that is causing the build to fail at the moment.

Comment 33

[Kartikaya Gupta (email:[e-mail])]

Ok, I filed bug 1624895 for the failure I was seeing.

Comment 34

[Stephen A Pohl [:spohl]]

:catlee or :jlund, would you be able to walk me through the steps to get the macOS 10.15 SDK uploaded to tooltool?

Comment 35

[Chris AtLee [:catlee]]

(In reply to Stephen A Pohl [:spohl] from comment #34)

:catlee or :jlund, would you be able to walk me through the steps to get the macOS 10.15 SDK uploaded to tooltool?

yes, but please hold off for now.

Comment 36

[Sean Voisen (:svoisen)]

FWIW getting the build system updated would also help the accessibility team in efforts to support VoiceOver.

Comment 37

[Stephen A Pohl [:spohl]]

(In reply to Sean Voisen (:svoisen) from comment #36)

FWIW getting the build system updated would also help the accessibility team in efforts to support VoiceOver.

We have typically been able to work around SDK limitations like these. Could you point me to a bug about this?

Comment 38

[Sean Voisen (:svoisen)]

(In reply to Stephen A Pohl [:spohl] from comment #37)

We have typically been able to work around SDK limitations like these. Could you point me to a bug about this?

No bug filed; the team can work around these limitations at the moment, but there are API differences between 10.11 and 10.14. :eeejay would be better able to provide details.

Comment 39

[Eitan Isaacson [:eeejay]]

I don't think we are currently blocked by the SDK version. We need to define some constants locally, it's not ideal, but not terrible.
I'll let you know if that changes.
Thanks!

Comment 40

[(Away)]

This would render bug 1616694 unnecessary, but until one of the bugs moves forward, I'm going to mark it as "See also" rather than depends or duplicate.

Comment 41

[cniff]

Time is ticking for this issue to become a real blocker for the entire macOS platform. Due to Apple's switch to Arm on Mac, it is foreseeable that macOS binaries need to be compiled with macOS 10.16 SDK presumably in 2022. Right now Rosetta 2 makes it possible to run old x86 binaries on Arm Macs. However Apple already announced a 2 year transition. So it can be expected that Rosetta 2 (like Rosetta 1) will be removed at the end of that process.

I am not sure if anyone already tried cross compilation with the macOS 10.16 SDK at all and how bad it is to fix the compiler and linker errors with a Linux cross compiler. If it is really a lot of work to get it working, then another option would be running either Catalina or Big Sur e.g. in QEMU and using native Xcode to bake the Firefox binary. That would also make it more easy to keep track with latest macOS SDKs in future.

Comment 42

[lolrepeatlol]

(In reply to cniff from comment #41)

Time is ticking for this issue to become a real blocker for the entire macOS platform. Due to Apple's switch to Arm on Mac, it is foreseeable that macOS binaries need to be compiled with macOS 10.16 SDK presumably in 2022. Right now Rosetta 2 makes it possible to run old x86 binaries on Arm Macs. However Apple already announced a 2 year transition. So it can be expected that Rosetta 2 (like Rosetta 1) will be removed at the end of that process.

I am not sure if anyone already tried cross compilation with the macOS 10.16 SDK at all and how bad it is to fix the compiler and linker errors with a Linux cross compiler. If it is really a lot of work to get it working, then another option would be running either Catalina or Big Sur e.g. in QEMU and using native Xcode to bake the Firefox binary. That would also make it more easy to keep track with latest macOS SDKs in future.

Firefox 84 is already native on Apple Silicon and is built with the 11.0 SDK. There is no benefit to upgrading the SDK for Intel, so the SDK is still on 10.11/12 there.

Comment 43

[Kai]

(In reply to Alexei Solonari from comment #42)

(In reply to cniff from comment #41)

Time is ticking for this issue to become a real blocker for the entire macOS platform. Due to Apple's switch to Arm on Mac, it is foreseeable that macOS binaries need to be compiled with macOS 10.16 SDK presumably in 2022. Right now Rosetta 2 makes it possible to run old x86 binaries on Arm Macs. However Apple already announced a 2 year transition. So it can be expected that Rosetta 2 (like Rosetta 1) will be removed at the end of that process.

I am not sure if anyone already tried cross compilation with the macOS 10.16 SDK at all and how bad it is to fix the compiler and linker errors with a Linux cross compiler. If it is really a lot of work to get it working, then another option would be running either Catalina or Big Sur e.g. in QEMU and using native Xcode to bake the Firefox binary. That would also make it more easy to keep track with latest macOS SDKs in future.

Firefox 84 is already native on Apple Silicon and is built with the 11.0 SDK. There is no benefit to upgrading the SDK for Intel, so the SDK is still on 10.11/12 there.

In light of Big Sur's UI changes especially, having an actually native dark mode and not just a chrome dark theme is one benefit. macOS elements such as right click menus, unstyles drop-downs or input boxes, buttons, etc., all remain light in the browser despite the OS being set to dark mode and the browser being set to dark mode. It's easy to mistake it as a bug (that's how I've ended up here).

Comment 44

[Markus Stange [:mstange]]

That part is tracked by bug 1623686.

Comment 45

[Kai]

(In reply to Markus Stange [:mstange] from comment #44)

That part is tracked by bug 1623686.

Thanks, apologies. I overlooked this ticket.

Comment 47

[Stephen A Pohl [:spohl]]

We're now building with the macOS 11 SDK (see bug 1696504). I'm going to duplicate this bug to bug 1696504 to keep track of it.