The default bug view has changed. See this FAQ.

mach: bootstrap support for Linux

RESOLVED FIXED in mozilla18

Status

()

Core
Build Config
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: gps, Assigned: kmm)

Tracking

(Blocks: 1 bug, {dev-doc-complete})

Trunk
mozilla18
dev-doc-complete
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(4 attachments, 5 obsolete attachments)

(Reporter)

Description

5 years ago
+++ This bug was initially created as a clone of Bug #774111 +++

Mach should have support for bootstrapping the development environment on systems running the yum package manager (RedHat Linux, CentOS, Fedora, others).

1) Start with a fresh/minimal install of the OS.
2) Obtain source code for mozilla-central (either a source bundle or repository clone)
3) Run ./mach and all build prerequisites are installed automatically.

The goal is to make it super simple for people to build mozilla-central. Your terminal-literate grandmother should be able to do it.

This bug essentially involves automating https://developer.mozilla.org/En/Developer_Guide/Build_Instructions/Linux_Prerequisites. Basically, query yum and offer to install missing packages.

Ideally, bootstrap support would not require a full copy of mozilla-central. We could provide a one-liner for someone to `curl | python` and magic would ensue. This could just talk to hg.mozilla.org over HTTPS (not using Mercurial) and pull down the latest .py files needed to start performing more magic. This minimal bootstrap is beyond the scope of this bug, but it should be considered during implementation.
(Reporter)

Updated

5 years ago
Assignee: nobody → kmm537
Whiteboard: [mach] → [mach][mentor=gps][lang=python]
(Assignee)

Comment 1

5 years ago
Created attachment 656958 [details]
very very rough draft of a working script for CentOS/Fedora 16

Tested on a Fedora 16 VM successfully, CentOS Live CD or base installs may give trouble with dependencies even after the yum installs; working on that now
Attachment #656958 - Flags: review+
(Reporter)

Comment 2

5 years ago
Comment on attachment 656958 [details]
very very rough draft of a working script for CentOS/Fedora 16

When requesting review, use the '?'. '+' is granted review.

Since this is an early draft, I'm switching it to the feedback field. Review is typically reserved for things that are capable of landing in the tree.
Attachment #656958 - Flags: review+ → feedback?(gps)
(Assignee)

Comment 3

5 years ago
Created attachment 657440 [details]
unified dependency installer including both apt and yum support

unified dependency installer with apt and yum support; tested on CentOS (not working yet), Fedora (should work on 16 at least), Ubuntu/Mint (worked test beds)
Attachment #656958 - Attachment is obsolete: true
Attachment #656958 - Flags: feedback?(gps)
Attachment #657440 - Flags: review?(gps)
(Assignee)

Updated

5 years ago
Attachment #657440 - Flags: review?(gps) → feedback?(gps)
(Reporter)

Comment 4

5 years ago
Created attachment 657444 [details] [diff] [review]
mozboot core

I coded up a basic skeleton for the Python package. This patch should be applied inside mozilla-central. Or, you can patch -p2 it to strip the leading python/ directory.

It doesn't do much. It should be pretty compatible with what you coded up.
(Assignee)

Comment 5

5 years ago
Created attachment 658163 [details]
bootstrapper with working CentOS 6 dependency installer

tested on 32 and 64 bit CentOS 6.3 Live CD; doesn't install libstdc++-static, but appears to be able to build; have not completed a full build with it but it passes all the checks at the start of the build process; leaving other files intact
Attachment #658163 - Flags: review?(gps)
(Assignee)

Updated

5 years ago
Attachment #658163 - Flags: review?(gps) → feedback?(gps)
(Reporter)

Updated

5 years ago
Duplicate of this bug: 774111
(Reporter)

Updated

5 years ago
Summary: mach: bootstrap support for yum (RHEL/CentOS/Fedora) → mach: bootstrap support for Linux
(Reporter)

Comment 7

5 years ago
Comment on attachment 658163 [details]
bootstrapper with working CentOS 6 dependency installer

First, this is looking really good. This bug excites me.

Things to change:

* Nuke the |make -f client.mk| bit. Bootstrap doesn't need to actually build the tree, just ensure the system is in the proper state to build the tree.

* Along the same vein, Mercurial cloning should at least be factored out into a common routine. Theoretically, Mercurial cloning should be identical across all systems. So, this code can be shared. I would also be fine if the initial landing of this bug did not contain support for cloning the repository. This is because IMO setting up Mercurial should be more than just cloning the repo (we also want to write out a config file that uses Mozilla's preferred settings).

* If Mercurial cloning makes it in, we need more flexibility here. Users should be able to choose the destination directory (currently it goes in $CWD/mozilla-central). It should also be optional.

* I have mixed feelings about using '-y'. On one hand, I think it is proper to prompt users before doing something. On the other, concerned users can look at the source code before they run something.

* You don't need a "pass" after each indented block. pass is only needed if the block is empty.

* That trick of updating self.__class__ is nifty. I didn't even know you could switch the class of an object after its creation! I'm pretty sure this is considered bad Python, however. This will probably go away when you merge things on top of the mozboot core. Just keep it in mind when writing Python in the future.
Attachment #658163 - Flags: feedback?(gps) → feedback+
(Reporter)

Comment 8

5 years ago
Created attachment 658637 [details] [diff] [review]
mozboot core, v2

I fixed the code. To run this, apply the patch inside a cloned mozilla-central repository. Then:

  cd python/mozboot
  PYTHONPATH=. python bin/bootstrap.py
Attachment #657444 - Attachment is obsolete: true
(Reporter)

Comment 9

5 years ago
Created attachment 658652 [details] [diff] [review]
mozboot core, v3

Added OS X skeleton. Found a few more bugs as part of this. This should actually work and should be ready for you to merge things in.
Attachment #658637 - Attachment is obsolete: true
(Reporter)

Comment 10

5 years ago
Created attachment 658663 [details] [diff] [review]
mozboot core, v4

I should add files before creating patches. Looks like I picked the wrong week to quit sniffing glue.
Attachment #658652 - Attachment is obsolete: true
(Assignee)

Comment 11

5 years ago
Created attachment 659336 [details]
haven't got setup.py working yet

run "python ./bootstrap1.py" with all files unzipped to same directory
will modify sys.path to allow for imports as intended; just wanted to get things up and running quickly without adding too much code; CentOS 6.3 only has Python 2.6.6 by default, so commented out argparse
Attachment #659336 - Flags: feedback?(gps)
(Reporter)

Comment 12

5 years ago
Comment on attachment 657440 [details]
unified dependency installer including both apt and yum support

The latest zip file makes this irrelevant.
Attachment #657440 - Flags: feedback?(gps)
(Reporter)

Comment 13

5 years ago
Created attachment 660237 [details] [diff] [review]
Part 1: bootstrap core, v1

Core classes to perform bootstrapping. This isn't the best code in the world. It doesn't need to be. Something (that works) is better than nothing.

Also, this is just the base classes. kmm will be providing actual code that builds on top of this.
Attachment #658663 - Attachment is obsolete: true
Attachment #660237 - Flags: review?(jhammel)
(Reporter)

Comment 14

5 years ago
kmm: I don't mind cleaning up your patch for you. Before I check things in, I'll need to know what name and email address to affiliate with the checkin. If you want to use "kmm" for you name, we can do that.

Comment 15

5 years ago
Comment on attachment 660237 [details] [diff] [review]
Part 1: bootstrap core, v1

+#!python

#!/usr/bin/env python plz

+        raise Exception('Not sure how to handle repo type: %s' % repo_type)

I'd use NotImplementedError if you anticipate adding more

+        include = os.path.normpath(os.path.join(os.path.dirname(__file__),
+            os.path.pardir))

indentation

+            files = fetch_files(repo_url, repo_type)

I'd only call this if these aren't None, None

+    options, leftover = parser.parse_args(args)

I'd be inclined to err out/print help if leftover 

+    def __init__(self):
+        pass

Please don't do this

+    def install_system_packages(self):
+        raise NotImplemented('%s must implement install_system_packages()' %
+            __name__)

indentation

wfm, with nits addressed
Attachment #660237 - Flags: review?(jhammel) → review+
(Reporter)

Comment 16

5 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/544c405fb456
https://hg.mozilla.org/integration/mozilla-inbound/rev/ac75030f29a7

I cleaned up kmm's patch and checked it in \o/

If we any follow-up work, we can do that on a new bug.

We'll need to update the docs on MDN to reflect the new world for bootstrapping:

  curl https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py | python -

Thank you again, kmm, for contributing the code and taking the time to verify bootstrap on all the different OSs works. Scores of Firefox developers will have you to thank.
Status: NEW → ASSIGNED
Keywords: dev-doc-needed
Whiteboard: [mach][mentor=gps][lang=python]
Target Milestone: --- → mozilla18
(Reporter)

Comment 17

5 years ago
Fixed an oops I missed in the review.

https://hg.mozilla.org/integration/mozilla-inbound/rev/58338ee54795
(Reporter)

Comment 18

5 years ago
Updated the following to reference the new one-line bootstrap method:

https://developer.mozilla.org/en-US/docs/Simple_Firefox_build
https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Linux_Prerequisites
Keywords: dev-doc-needed → dev-doc-complete
https://hg.mozilla.org/mozilla-central/rev/544c405fb456
https://hg.mozilla.org/mozilla-central/rev/ac75030f29a7
https://hg.mozilla.org/mozilla-central/rev/58338ee54795
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Depends on: 790995
(Reporter)

Comment 20

4 years ago
Comment on attachment 659336 [details]
haven't got setup.py working yet

Patch is stale.
Attachment #659336 - Flags: feedback?(gps)
You need to log in before you can comment on or make changes to this bug.