Closed Bug 913965 Opened 6 years ago Closed 6 years ago
Ubuntu bootstrapper should not use build-dep for finding dependencies / install modern Mercurial
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0 (Beta/Release) Build ID: 20130803215302 Steps to reproduce: OS : Ubuntu 12.04.2 LTS Followed instructions from https://developer.mozilla.org/en-US/docs/Simple_Firefox_build/Linux_and_MacOS_build_preparation Run the following command : wget --no-check-certificate https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py Actual results: --2013-09-09 05:58:10-- https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py Resolving hg.mozilla.org (hg.mozilla.org)... 220.127.116.11 Connecting to hg.mozilla.org (hg.mozilla.org)|18.104.22.168|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 4449 (4.3K) [text/x-python] Saving to: `bootstrap.py' 100%[======================================================================================================>] 4,449 --.-K/s in 0.006s 2013-09-09 05:58:12 (772 KB/s) - `bootstrap.py' saved [4449/4449] Executing as root: sudo apt-get build-dep firefox Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to find a source package for firefox Traceback (most recent call last): File "bootstrap.py", line 143, in <module> sys.exit(main(sys.argv)) File "bootstrap.py", line 134, in main dasboot.bootstrap() File "/tmp/tmpg0MQa2/mozboot/bootstrap.py", line 85, in bootstrap # The next fallback is to download the files from the source File "/tmp/tmpg0MQa2/mozboot/ubuntu.py", line 17, in install_system_packages File "/tmp/tmpg0MQa2/mozboot/base.py", line 106, in run_as_root File "/usr/lib/python2.7/subprocess.py", line 511, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '[u'sudo', 'apt-get', 'build-dep', 'firefox']' returned non-zero exit status 100
I'm pretty sure this has been reported before. The underling issue is we're relying on the user's machine to have the sources package repository available so the "build-dep" command works. As has been noted many times, we should just maintain our own explicit list of dependency packages instead of relying on the build dependencies for the distro's firefox package, which likely lags behind m-c by a few releases.
Summary: Bootstrap didn't work → Ubuntu bootstrapper should not use build-dep for finding dependencies
This seems to do the trick (at least with the base Ubuntu 13.04 VM I was using). A build-dep firefox still wants the following packages: cdbs dh-translations diffstat gconf-service gconf-service-backend gconf2 gconf2-common gir1.2-gconf-2.0 gir1.2-gnomekeyring-1.0 hardening-wrapper intltool libart-2.0-2 libart-2.0-dev libavahi-client-dev libavahi-common-dev libavahi-glib-dev libavahi-glib1 libbonobo2-0 libbonobo2-common libbonobo2-dev libbonoboui2-0 libbonoboui2-common libbonoboui2-dev libcanberra-dev libencode-locale-perl libfile-listing-perl libgail-common libgail-dev libgail18 libgconf-2-4 libgconf2-dev libglade2-0 libgnome-keyring-common libgnome-keyring-dev libgnome-keyring0 libgnome2-0 libgnome2-bin libgnome2-common libgnome2-dev libgnomecanvas2-0 libgnomecanvas2-common libgnomecanvas2-dev libgnomeui-0 libgnomeui-common libgnomeui-dev libgnomevfs2-0 libgnomevfs2-common libgnomevfs2-dev libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libidl-common libidl-dev libidl0 libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libnet-http-perl libnet-ssleay-perl liborbit2 liborbit2-dev libpopt-dev libselinux1-dev libsepol1-dev libstartup-notification0 libstartup-notification0-dev liburi-perl libwww-perl libwww-robotrules-perl libx11-xcb1 libxcb-util0 libxml-parser-perl python-scour quilt xvfb Should I throw any of these in here or should I just defer to configure?
I looked at the package metadata and added some missing -dev packages. This appears to satisfy all but the Debian package development dependency set with the following build-dep packages outstanding: cdbs dh-translations diffstat intltool libencode-locale-perl libfile-listing-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libnet-http-perl libnet-ssleay-perl liburi-perl libwww-perl libwww-robotrules-perl libxml-parser-perl python-scour quilt
Attachment #801890 - Flags: review?(mh+mozilla)
Verified the bootstrapper works with 12.04 as well. I assume 12.10 behaves the same. I can individually verify other releases upon request.
Comment on attachment 801890 [details] [diff] [review] Don't use build-dep for installing Ubuntu dependencies Review of attachment 801890 [details] [diff] [review]: ----------------------------------------------------------------- Please also change debian.py. Or, in fact, since the only real difference between debian.py and ubuntu.py, besides the fact that debian.py has been bitrotted, is apt-get build-dep iceweasel instead of apt-get build-dep firefox, please merge both. ::: python/mozboot/mozboot/ubuntu.py @@ +17,3 @@ > self.apt_install( > 'autoconf2.13', > + 'binutils-gold', This makes /usr/bin/ld point to gold, which may not be wanted for other things than firefox. The package is not necessary for the availability of gold itself: it is in binutils. And the build system knows how to use gold from there. So please remove binutils-gold. (and binutils is in build-essential, so no need to replace) @@ +18,5 @@ > 'autoconf2.13', > + 'binutils-gold', > + 'build-essential', > + 'ccache', > + 'clang', Not sure this is helpful. Until 13.04 (which i wouldn't advise anyone to use yet), the clang package is clang 3.0, which can't be used to build the tree. @@ +19,5 @@ > + 'binutils-gold', > + 'build-essential', > + 'ccache', > + 'clang', > + 'dbus-x11', we're not using dbus-launch, which is about the only thing in that package. @@ +25,5 @@ > 'libcurl4-openssl-dev', > + 'libdbus-1-dev', > + 'libdbus-glib-1-dev', > + 'libgconf2-dev', > + 'libgnomeui-dev', libgnomeui is optional and not enabled by default. We're using gio now. @@ +33,3 @@ > 'libiw-dev', > 'libnotify-dev', > + 'libstartup-notification0-dev', startup notification is disabled by default. @@ +36,2 @@ > 'libxt-dev', > 'mercurial', Relatedly, i'm not sure mercurial has its place here. Because if you have that code, you obviously have got the tree. Which you may well have done with git, and don't care about mercurial.
Attachment #801890 - Flags: review?(mh+mozilla) → review-
wrt mercurial, it's possible to run bootstrap.py standalone, without a copy of the tree.
(In reply to Gregory Szorc [:gps] from comment #6) > wrt mercurial, it's possible to run bootstrap.py standalone, without a copy > of the tree. Where does bootstrap.py get ubuntu.py from?
Oh wow python/mozboot/bin/bootstrap.py.
Whiteboard: kanbanzilla[Needs Investigation]
Status: UNCONFIRMED → NEW
Ever confirmed: true
Whiteboard: kanbanzilla[Needs Investigation]
Removed "bad" packages and consolidated debian and ubuntu package lists.
Attachment #802683 - Flags: review?(mh+mozilla)
Attachment #801890 - Attachment is obsolete: true
Comment on attachment 802683 [details] [diff] [review] Don't use build-dep for installing Ubuntu dependencies Review of attachment 802683 [details] [diff] [review]: ----------------------------------------------------------------- I'd prefer ubuntu.py to go away, the _update_package_manager and upgrade_mercurial functions to move to debian.py, and the ubuntu and elementary distros to use DebianBootstrapper.
Attachment #802683 - Flags: review?(mh+mozilla) → review-
For upgrade_mercurial(), since Ubuntu is currently shipping an ancient Mercurial, I was thinking that Ubuntu's implementation would hook up a PPA. This would necessitate a different class from Debian. Ubuntu could definitely inherit from the Debian class, however.
Ubuntu now inherits from Debian. While I was here, I hooked up a Mercurial PPA to Ubuntu because I don't feel like dealing with the overhead of another bug and patch.
Attachment #803188 - Flags: review?(mh+mozilla)
Attachment #802683 - Attachment is obsolete: true
Comment on attachment 803188 [details] [diff] [review] Don't use build-dep for installing Ubuntu dependencies Review of attachment 803188 [details] [diff] [review]: ----------------------------------------------------------------- ::: python/mozboot/mozboot/ubuntu.py @@ +14,5 @@ > > +# Ubuntu shares much logic with Debian, so it inherits from it. > +class UbuntuBootstrapper(DebianBootstrapper): > + DISTRO_PACKAGES = [ > + 'software-properties-common', Please add a comment saying this contains add-apt-repository. @@ +33,5 @@ > + for source in os.listdir('/etc/apt/sources.list.d'): > + if source.startswith(list_file) and source.endswith('.list'): > + return > + > + print(MERCURIAL_PPA) debugging leftover?
Attachment #803188 - Flags: review?(mh+mozilla) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/6d30f146c472 With both feedback points addressed.
Status: NEW → ASSIGNED
Summary: Ubuntu bootstrapper should not use build-dep for finding dependencies → Ubuntu bootstrapper should not use build-dep for finding dependencies / install modern Mercurial
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
You need to log in before you can comment on or make changes to this bug.