411588 - Support for 64bit universal binaries for Mac OS X

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Simon Howes]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en; rv:1.9b3pre) Gecko/2008010900 Camino/2.0a1pre (like Firefox/3.0b3pre)
Build Identifier: 

To be able to compile Firefox and other Mozilla apps as 32 and 64 bit universal binaries for Mac OS X.

We should at least be able to build as a 64 bit application without errors.

Unsure if there would be a performance gain, however could be good marketing for Mozilla.

Reproducible: Always

Comment 1

[(not currently active) Ted Mielczarek]

This would be a lot easier with bug 417044 fixed. bsmedberg said he had that mostly working...

Can you build on Mac targetting x86_64? That would be the first step, I'd think.

Comment 2

[Simon Howes]

Does not compile at all.

After 5 seconds of compiling:

nsprpub/config/now.c
gcc-4.2  now.o   -o now
ld: warning in now.o, file is not of required architecture
Undefined symbols:
  "_main", referenced from:
      __start in crt1.o
ld: symbol(s) not found
collect2: ld returned 1 exit status

Comment 3

[(not currently active) Ted Mielczarek]

File that separately (if it's not already), and mark it blocking this bug?

Comment 4

[Josh Aas]

This bug should be about a universal configuration that includes 64-bit Mac OS X support. 64-bit Mac OS X support itself does not necessarily have to be part of a universal binary. 64-bit Mac OS X support is bug 468509.

Comment 5

[Nomis101]

On Mac OS X a 32/64-bit Universal application has a checkbox in the Information-View (if you rightclick the application) to choose if the Application should start in 32bit- or 64bit-mode. But I don't know if this must be specified in the application or if this checkbox is a standard if the application contains a 64bit binary.
Some 32/64-bit Universal applications (like XCode 3.0 or Adobe Lightroom 2) have also an <key>LSArchitecturePriority</key> key in its Info.plist to define the priority of the binary architectures. This is not standard, because there are also 32/64-bit Universal applications without this key.
Is it planed to include this in mozilla applications?

Comment 6

[Josh Aas]

We don't use Xcode UI and we have not made any decisions about how we're going to package a universal binary. That is a long ways off.

Comment 7

[Nomis101]

Since we can build in x86_64 on Mac OS X now, is it now possible in any manner to build a i386/x86_64 binarie? Or are there first some modifications of the build system needed?
And does something like "ac_add_app_options x86_64" exist?

Comment 8

[(not currently active) Ted Mielczarek]

I don't see why it wouldn't be. There's nothing magic about the current universal configuration, it just does ppc/i386 builds in two separate passes, and stitches the results together afterwards. The mozconfig it uses is here:
http://mxr.mozilla.org/mozilla-central/source/build/macosx/universal/mozconfig

You'll notice that that specifies a MOZ_POSTFLIGHT_ALL makefile:
http://mxr.mozilla.org/mozilla-central/source/build/macosx/universal/flight.mk
which is included by client.mk and has its postflight_all target run after both builds complete. That target uses the "unify" script to combine the two builds into a universal build.

Comment 9

[Nomis101]

Thanks, that was a bit helpfull. I supplemented my mozconfig with /universal/mozconfig and changed all "ppc" to "x86_64". Than I've also changed all "ppc" in /universal/flight.mk to "x86_64". Than the x86_64 part builds just fine, but the i386 part failed, because CROSS_COMPILE=1 was not properly set. So I've set CROSS_COMPILE=1 for all archs. Than it builds just fine in x86_64 and after that in i386. But the flight.mk failed. I get
make[1]: *** [postflight_all] Error 1
make: *** [build] Error 2
But I don't know why, I don't see any other errors. So now I have obj-x86_64-apple-darwin10.2.0/i386/mozilla with the i386 binarie and obj-x86_64-apple-darwin10.2.0/x86_64/mozilla with my x86_64 binarie.

Comment 10

[(not currently active) Ted Mielczarek]

Can you find me on IRC and pastebin me the end of your build log?

Comment 11

[Nomis101]

Hm, I've never used IRC till now and I had no previous experience with this. But I can email you the end of my build log, if thats OK also...

Comment 12

[(not currently active) Ted Mielczarek]

That's fine, yes.

Comment 13

[Nomis101]

Attachment: Open in 32-bit mode checkbox

Now I can answer my own question from Comment 5. If you build an i386/x86_64 UB, than you will see the "Open in 32-bit mode" checkbox per default.
(I've added the LSArchitecturePriority key to my Info.plist to set i386 as the default)

Comment 14

[(not currently active) Ted Mielczarek]

I talked with Josh, my plan is to provide another mozconfig in build/macosx/universal that configures things for an i386/x86_64 universal build. I'm going to make it so that it shares most of its core with the existing mozconfig.

Comment 15

[(not currently active) Ted Mielczarek]

Attachment: add a mozconfig for i386/x86_64 universal builds

This adds a "mozconfig-64" alongside the mozconfig in build/macosx/universal that you can source to do an i386/x86_64 universal build. I made it so both mozconfigs source a common file since it's mostly shared.

Comment 16

[(not currently active) Ted Mielczarek]

I've only done a traditional ppc/i386 UB with this, but it built fine.

Comment 17

[Josh Aas]

i386/x86_64 build still fails but that is because crashreporter hasn't been ported to x86_64 yet

Comment 18

[(not currently active) Ted Mielczarek]

That might get fixed by jimb's work in bug 517832. bug 548025 covers making everything work natively on 64-bit OSX.

Comment 19

[(not currently active) Ted Mielczarek]

Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/af2a60e9e64e

Comment 20

[(not currently active) Ted Mielczarek]

Crud, I missed a spot:
http://mxr.mozilla.org/mozilla-central/source/build/macosx/universal/flight.mk#42

hardcodes ppc/i386.

Comment 21

[Nomis101]

Aren't there also hardcoded ppc/i386 lines in unify?
E.g.: http://hg.mozilla.org/mozilla-central/file/ebfe0dc025da/build/macosx/universal/unify#l629

Comment 22

[(not currently active) Ted Mielczarek]

No, those are just variable names.

Comment 23

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

ted: do you have any ETA on this or scope on how tricky this would be? Its blocking a Q1 goal.

Comment 24

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #23)
> ted: do you have any ETA on this or scope on how tricky this would be? Its
> blocking a Q1 goal.


per irc with ted and bear just now, we're going to do 64bit-only for now, and will revisit universal builds later after these makefiles are fixed up.

Comment 25

[Nomis101]

Question. I've wondered why I can't open my 32/64bit Firefox in 32bit. Now I noticed the following warning while building:
> warning: mac68k alignment pragma is deprecated for 64-bit Darwin
Further reading gives me:
>"When compiling 64-bit code, the compiler ignores the pragmas and your code works correctly. If, >however, you currently use the mac68k alignment pragma in your own data structures that will be >shared between 32-bit and 64-bit versions of your application (or if you use the mac68k pragma for >a data structure that corresponds with the register layout of a physical device), you must rewrite the >data structure to use a packed alignment and pad the structure appropriately."
http://developer.apple.com/mac/library/documentation/Darwin/Conceptual/64bitPorting/MakingCode64-BitClean/MakingCode64-BitClean.html

So, could this be the reason?
http://mxr.mozilla.org/mozilla-central/search?string=mac68k

Comment 26

[Josh Aas]

Attachment: flight.mk changes, v1.0

Comment 27

[Josh Aas]

Attachment: error with flight.mk changes v1.0

I get this error at the end of the build with flight.mk changes v1.0.

Comment 28

[(not currently active) Ted Mielczarek]

Comment on attachment 450703 [details] [diff] [review]
flight.mk changes, v1.0

>diff --git a/build/macosx/universal/flight.mk b/build/macosx/universal/flight.mk
>--- a/build/macosx/universal/flight.mk
>+++ b/build/macosx/universal/flight.mk
>@@ -34,22 +34,22 @@
> #
> # ***** END LICENSE BLOCK *****
> 
> # BE CAREFUL!  This makefile handles a postflight_all rule for a
> # multi-project build, so DON'T rely on anything that might differ between
> # the two OBJDIRs.
> 
> ifndef OBJDIR
>-OBJDIR_PPC = $(MOZ_OBJDIR)/ppc
>-OBJDIR_X86 = $(MOZ_OBJDIR)/i386
>-DIST_PPC = $(OBJDIR_PPC)/dist
>-DIST_X86 = $(OBJDIR_X86)/dist
>-DIST_UNI = $(DIST_PPC)/universal
>-OBJDIR = $(OBJDIR_PPC)
>+OBJDIR_ARCH_1 = $(MOZ_OBJDIR)/$(firstword $(MOZ_BUILD_PROJECTS))
>+OBJDIR_ARCH_2 = $(MOZ_OBJDIR)/$(word 2,$(MOZ_BUILD_PROJECTS))
>+DIST_ARCH_1 = $(OBJDIR_ARCH_1)/dist
>+DIST_ARCH_2 = $(OBJDIR_ARCH_2)/dist
>+DIST_UNI = $(DIST_ARCH_1)/universal
>+OBJDIR = $(DIST_ARCH_1)

This line is the source of your build error. It was originally $(OBJDIR) = $(OBJDIR_PPC), so it should now be $(OBJDIR) = $(OBJDIR_ARCH_1), but you have DIST_ARCH_1.

Looks pretty good otherwise.

Comment 29

[Josh Aas]

Attachment: flight.mk changes, v1.1

Ugh, fixed. Thanks.

Comment 30

[Josh Aas]

pushed to mozilla-central

http://hg.mozilla.org/mozilla-central/rev/f9b239a5a9ef