bootstrap support for OS X

RESOLVED FIXED in mozilla18

Status

Firefox Build System
General
RESOLVED FIXED
6 years ago
3 months ago

People

(Reporter: gps, Assigned: gps)

Tracking

(Depends on: 1 bug, {dev-doc-complete})

Trunk
mozilla18
dev-doc-complete
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 3 obsolete attachments)

(Assignee)

Description

6 years ago
Mach should have support for bootstrapping the development environment on OS X.

1) Start with a fresh install of OS X.
2) Obtain source code for mozilla-central (either a source bundle or repository clone)
3) Run ./mach and all build prerequisites are downloaded, installed, and configured properly.

The goal is to make it super simple for people to build mozilla-central. Your terminal-literate grandmother should be able to do it.

This bug essentially involves automating https://developer.mozilla.org/En/Developer_Guide/Build_Instructions/Mac_OS_X_Prerequisites. Basically,

1) Install Xcode
2) Install Homebrew or MacPorts
3) Install required packages from Homebrew or MacPorts

Ideally, bootstrap support would not require a full copy of mozilla-central. We could provide a one-liner for someone to `curl | python` and magic would ensue. This could, of course, just talk to hg.mozilla.org over HTTPS (not using Mercurial) and pull down the latest .py files needed to start performing more magic. This minimal bootstrap is beyond the scope of this bug, but it should be considered during implementation.
(Assignee)

Comment 1

6 years ago
Since we now build with Clang on OS X and since we have the SVN revision of LLVM/Clang we build with checked into the repository, it would be splendidly awesome if bootstrap also ensured the appropriate version of Clang were installed.

Bug 768879 tracks exposing the Clang builds Mozilla uses on its official infrastructure to the public internet. Bootstrap could theoretically fetch binaries from there.
Depends on: 768879
(Assignee)

Comment 2

6 years ago
I just discovered B2G has a bootstrap script for OS X. We should consider stealing parts. https://github.com/mozilla-b2g/B2G/blob/master/scripts/bootstrap-mac.sh
(Assignee)

Comment 3

6 years ago
Created attachment 660328 [details] [diff] [review]
Bootstrap backend for OS X, v1

It bootstraps a vanilla OS X install to the point where it can build m-c.

I've only tested this on OS X 10.8. I could probably check this in without review b/c it isn't part of the build. But, I'd like just a cursory glance over the code.

Things like user prompting and support for additional OS X versions will come in follow-up bugs. Something is better than nothing.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Attachment #660328 - Flags: review?(rnewman)
(Assignee)

Comment 4

6 years ago
This isn't part of mach.
No longer blocks: 774108
Summary: mach: bootstrap support for OS X → bootstrap support for OS X
Whiteboard: [mach]
Doesn't apply to current m-c…

unable to find 'python/mozboot/mozboot/base.py' for patching
1 out of 1 hunks FAILED -- saving rejects to file python/mozboot/mozboot/base.py.rej
unable to find 'python/mozboot/mozboot/bootstrap.py' for patching
1 out of 1 hunks FAILED -- saving rejects to file python/mozboot/mozboot/bootstrap.py.rej
unable to find 'python/mozboot/mozboot/osx.py' for patching
1 out of 1 hunks FAILED -- saving rejects to file python/mozboot/mozboot/osx.py.rej

Missing dependency?
(Assignee)

Comment 6

6 years ago
(In reply to Richard Newman [:rnewman] from comment #5)
> Doesn't apply to current m-c…
> 
> unable to find 'python/mozboot/mozboot/base.py' for patching
> 1 out of 1 hunks FAILED -- saving rejects to file
> python/mozboot/mozboot/base.py.rej
> unable to find 'python/mozboot/mozboot/bootstrap.py' for patching
> 1 out of 1 hunks FAILED -- saving rejects to file
> python/mozboot/mozboot/bootstrap.py.rej
> unable to find 'python/mozboot/mozboot/osx.py' for patching
> 1 out of 1 hunks FAILED -- saving rejects to file
> python/mozboot/mozboot/osx.py.rej
> 
> Missing dependency?

Those files are in inbound and haven't been merged to central yet.
(Assignee)

Comment 7

6 years ago
Created attachment 660536 [details] [diff] [review]
Bootstrap backend for OS X, v2

Performed some general code cleanup by refactoring the printed content to not be inline.

More importantly, I added a check for the OS X command line tools version. Previously, we allowed older, non-functional compilers in /usr/bin to exist. Now, we prompt to upgrade the Xcode command line tools version.

I've still only tested this on 10.8. But, I'll soon have my hands on a 10.5, 10.6, and 10.7 machine. If the instructions need refined on older OS versions, I'm fine with these being follow-up bugs.
Attachment #660328 - Attachment is obsolete: true
Attachment #660328 - Flags: review?(rnewman)
Attachment #660536 - Flags: review?(rnewman)
(Assignee)

Comment 8

6 years ago
Created attachment 660655 [details] [diff] [review]
Bootstrap backend for OS X, v3

I /think/ this will work with OS X 10.6. Although, I haven't actually tested. Apply on top of m-c (merge finally happened, yay) and let me know if it works.
Attachment #660536 - Attachment is obsolete: true
Attachment #660536 - Flags: review?(rnewman)
Attachment #660655 - Flags: review?(rnewman)
(In reply to Gregory Szorc [:gps] from comment #8)

> I /think/ this will work with OS X 10.6. Although, I haven't actually
> tested. Apply on top of m-c (merge finally happened, yay) and let me know if
> it works.

Running "python python/mozboot/mozboot/bootstrap.py" on 10.6.8 with current m-c:

---
Traceback (most recent call last):
  File "python/mozboot/mozboot/bootstrap.py", line 10, in <module>
    from mozboot.centos import CentOSBootstrapper
ImportError: No module named mozboot.centos
(Assignee)

Comment 10

6 years ago
Run bin/bootstrap.py instead.
(In reply to Gregory Szorc [:gps] from comment #10)
> Run bin/bootstrap.py instead.

Traceback (most recent call last):
  File "python/mozboot/bin/bootstrap.py", line 134, in <module>
    sys.exit(main(sys.argv))
  File "python/mozboot/bin/bootstrap.py", line 125, in main
    dasboot.bootstrap()
  File "python/mozboot/mozboot/bootstrap.py", line 54, in bootstrap
    instance.install_system_packages()
  File "python/mozboot/mozboot/osx.py", line 15, in install_system_packages
    raise NotImplementedError('OS X bootstrap not yet implemented.')
NotImplementedError: OS X bootstrap not yet implemented.
Wait, I qpopped. I actually get:

Traceback (most recent call last):
  File "python/mozboot/bin/bootstrap.py", line 134, in <module>
    sys.exit(main(sys.argv))
  File "python/mozboot/bin/bootstrap.py", line 125, in main
    dasboot.bootstrap()
  File "python/mozboot/mozboot/bootstrap.py", line 68, in bootstrap
    instance.install_system_packages()
  File "python/mozboot/mozboot/osx.py", line 98, in install_system_packages
    self.ensure_xcode()
  File "python/mozboot/mozboot/osx.py", line 105, in ensure_xcode
    if not os.path.exsts('/Developer/Applications/Xcode.app'):
AttributeError: 'module' object has no attribute 'exsts'

Let me fix that typo…
With typo fixed I get:

Homebrew needs XQuartz installed in order to build some dependencies. Please
download and install XQuartz from the following URL:

    http://xquartz.macosforge.org/downloads/SL/XQuartz-2.7.3.dmg
When that has finished, please relaunch bootstrap.

This seems inaccurate, because I have a mozilla-central build running with a HomeBrew-installed clang *right now*. Correct?
(Assignee)

Comment 14

6 years ago
Created attachment 660703 [details] [diff] [review]
Bootstrap backend for OS X, v4

Conditionally require XQuartz depending on whether Python is already installed.

This may not be perfect. I'm trying to sort out what Homebrew is doing...
Attachment #660655 - Attachment is obsolete: true
Attachment #660655 - Flags: review?(rnewman)
Attachment #660703 - Flags: review?(rnewman)
Comment on attachment 660703 [details] [diff] [review]
Bootstrap backend for OS X, v4

This works. It ignored the hg 1.8.2 that I had on my path, installing 2.3 from HomeBrew, and installed autoconf-2.13.
Attachment #660703 - Flags: review?(rnewman) → review+
(Assignee)

Comment 16

6 years ago
https://hg.mozilla.org/mozilla-central/rev/e80ec1d9b1b1
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Keywords: dev-doc-needed
Resolution: --- → FIXED
Target Milestone: --- → mozilla18
Depends on: 790883

Updated

5 years ago
Blocks: 838384

Comment 18

5 years ago
I was looking at the code you guys wrote for bootstrapping the development environment on OS X while Googling around to see if anyone had scripted a bootstrapper for Homebrew itself. I'm in the middle of writing a script that bootstraps a development stack on my OS X machine myself.

I love the Python code; I've been doing things in bash so far, but thought about using Python, and like your approach better. I think the script could be improved if you pulled the command line tools binaries off of Apple's web site, dumped them into SourceForge or something like that, and then downloaded and installed them as part of your bootstrapping script. Doing so might raise Apple's hackles, but should be legal due to the open source nature of the software, and would fully automate your script. At the same time, such an install should not interfere with an existing Xcode installation; it is the command line equivalent downloading the latest version of Xcode and installing the command line tools from within the GUI. (There could be issues if the command line tools downloaded did not match the version of an existing Xcode install, I suppose, but that scenario could be tested.)

If you guys think it's a reasonable idea, I'd be happy to contribute patches sometime over the next couple weeks, since I'm in the process of doing the same thing in my spare time anyway.
(Assignee)

Comment 19

5 years ago
(In reply to goxberry from comment #18)
> I love the Python code; I've been doing things in bash so far, but thought
> about using Python, and like your approach better. I think the script could
> be improved if you pulled the command line tools binaries off of Apple's web
> site, dumped them into SourceForge or something like that, and then
> downloaded and installed them as part of your bootstrapping script. Doing so
> might raise Apple's hackles, but should be legal due to the open source
> nature of the software, and would fully automate your script. At the same
> time, such an install should not interfere with an existing Xcode
> installation; it is the command line equivalent downloading the latest
> version of Xcode and installing the command line tools from within the GUI.
> (There could be issues if the command line tools downloaded did not match
> the version of an existing Xcode install, I suppose, but that scenario could
> be tested.)

Fully automating things would be desirable. Avoiding the Xcode install would be nice.

That being said, I do worry about legal implications. We shouldn't have the bootstrapper bypass ToS acceptance, etc.

> If you guys think it's a reasonable idea, I'd be happy to contribute patches
> sometime over the next couple weeks, since I'm in the process of doing the
> same thing in my spare time anyway.

If there is a way to do things that doesn't upset lawyers, we'd likely be interested.

Also, commenting on old resolved bugs is generally frowned upon. If you have additional comments or patches, please file a new bug.
The command line tools don't include the OS X SDKs, so I don't think you can build m-c just from that. Might also be worth mentioning to the user in the bootstrap process that Mozilla's OS X builds use the 10.6 SDK, so if you just install latest Xcode, you won't be building the same code as the test and release builds do.
Depends on: 952395

Updated

3 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.