Closed Bug 1232486 Opened 8 years ago Closed 8 years ago

Mach Bootstrap fails on Fedora for fennec

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox46 fixed)

RESOLVED FIXED
mozilla46
Tracking Status
firefox46 --- fixed

People

(Reporter: bmurray7jhu, Assigned: bmurray7jhu)

References

Details

Attachments

(2 files, 2 obsolete files)

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36

Steps to reproduce:

Mach Bootstrap fails on Fedora 23.  The method install_mobile_android_packages() is not implemented.

To reproduce:
1. Clone mercurial tree onto a Fedora 23 x64 system
2. Run the command `./mach bootstrap`



Actual results:

[brian@brian-fedor22-lan-clover-com firefox]$ ./mach bootstrap

Please choose the version of Firefox you want to build:
1. Firefox for Desktop
2. Firefox for Android
Your choice:
2
Executing as root: sudo yum groupinstall "Development Tools" "Development Libraries"
[sudo] password for brian: 
Yum command has been deprecated, redirecting to '/usr/bin/dnf groupinstall Development Tools Development Libraries'.                           
See 'man dnf' and 'man yum2dnf' for more information.                                                                                          
To transfer transaction metadata from yum to DNF, run:                                                                                         
'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate'                                                                               
                                                                                                                                               
Last metadata expiration check performed 0:23:15 ago on Mon Dec 14 14:48:41 2015.                                                              
Group 'Development Tools' is already installed, skipping.                                                                                      
Group 'Development Libraries' is already installed, skipping.                                                                                  
Dependencies resolved.                                                                                                                         
Nothing to do.                                                                                                                                 
Complete!                                                                                                                                      
Executing as root: sudo yum install autoconf213 mercurial                                                                                      
Yum command has been deprecated, redirecting to '/usr/bin/dnf install autoconf213 mercurial'.                                                  
See 'man dnf' and 'man yum2dnf' for more information.                                                                                          
To transfer transaction metadata from yum to DNF, run:                                                                                         
'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate'                                                                               
                                                                                                                                               
Last metadata expiration check performed 0:23:19 ago on Mon Dec 14 14:48:41 2015.                                                              
Package autoconf213-2.13-34.fc22.noarch is already installed, skipping.                                                                        
Package mercurial-3.5.1-1.fc22.x86_64 is already installed, skipping.                                                                          
Dependencies resolved.                                                                                                                         
Nothing to do.                                                                                                                                 
Complete!                                                                                                                                      
Error running mach:                                                                                                                            
                                                                                                                                               
    ['bootstrap']                                                                                                                              
                                                                                                                                               
The error occurred in code that was called by the mach command. This is either                                                                 
a bug in the called code itself or in the way that mach is calling it.                                                                         
                                                                                                                                               
You should consider filing a bug for this issue.                                                                                               
                                                                                                                                               
If filing a bug, please include the full output of mach, including this error                                                                  
message.                                                                                                                                       
                                                                                                                                               
The details of the failure are as follows:                                                                                                     
                                                                                                                                               
NotImplementedError: Cannot bootstrap Firefox for Android: mozboot.base does not yet implement install_mobile_android_packages()               
                                                                                                                                               
  File "/home/brian/github/mozilla/firefox42/firefox-42.0/python/mozboot/mozboot/mach_commands.py", line 23, in bootstrap                      
    bootstrapper.bootstrap()                                                                                                                   
  File "/home/brian/github/mozilla/firefox42/firefox-42.0/python/mozboot/mozboot/bootstrap.py", line 134, in bootstrap                         
    getattr(self.instance, 'install_%s_packages' % application)()                                                                              
  File "/home/brian/github/mozilla/firefox42/firefox-42.0/python/mozboot/mozboot/base.py", line 125, in install_mobile_android_packages        
    % __name__) 


Expected results:

Mach should successfully bootstrap the build environment.
Attachment #8698247 - Attachment description: Mach Bootstrap fails on Fedora for fennec → Fix Mach Bootstrap on Fedora for fennec
Attachment #8698247 - Flags: feedback?(nalexander)
Comment on attachment 8698247 [details] [diff] [review]
Fix Mach Bootstrap  on Fedora for fennec

Review of attachment 8698247 [details] [diff] [review]:
-----------------------------------------------------------------

Hi Brian -- thanks for helping with this!

I have a few small nits and one real question for you before we land this.  Could you answer the Q, and push an updated patch addressing the minor comments?

I can't really test this, so it's more-or-less "if it works for you, it works for me".

Thanks again!

::: python/mozboot/mozboot/fedora.py
@@ +1,5 @@
>  # This Source Code Form is subject to the terms of the Mozilla Public
>  # License, v. 2.0. If a copy of the MPL was not distributed with this file,
>  # You can obtain one at http://mozilla.org/MPL/2.0/.
> +import os
> +import sys

Do you need `sys`?  It looks like no.

@@ +14,5 @@
>          self.version = version
>          self.dist_id = dist_id
>  
>          self.group_packages = [
> +            'C Development Tools and Libraries',

I don't know much about Fedora, but where are you getting this change from?  https://fedoraproject.org/wiki/RPMGroups suggests something closer to what we had before.

@@ +60,5 @@
>  
> +    def install_mobile_android_packages(self):
> +        import android
> +
> +        #install android mobile packages

nit: space after #, full sentence:
```
# Install Android-specific packages.
```

@@ +61,5 @@
> +    def install_mobile_android_packages(self):
> +        import android
> +
> +        #install android mobile packages
> +        self.dnf_install(*self.mobile_android_packages)

I think you want to
```
self.dnf_groupinstall(*self.group_packages)
```
first, since you want things like Make and autoconf to be available.

@@ +63,5 @@
> +
> +        #install android mobile packages
> +        self.dnf_install(*self.mobile_android_packages)
> +
> +        #Get NDK and SDK

nit: space after #, full sentence:
# Fetch Android SDK and NDK.
Attachment #8698247 - Flags: feedback?(nalexander) → feedback+
Assignee: nobody → bmurray7jhu
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
>Do you need `sys`?  It looks like no.
Removed.

>I don't know much about Fedora, but where are you getting this change from?  >https://fedoraproject.org/wiki/RPMGroups suggests something closer to what we had before.
I'm running `dnf grouplist`.

>I think you want to
>```
>self.dnf_groupinstall(*self.group_packages)
>```
>first, since you want things like Make and autoconf to be available.

In bootstrap.py, `install_system_packages()` is called before `install_mobile_android_packages()`, so there is no need to redundantly install the group packages.
Comment on attachment 8698287 [details] [diff] [review]
Mach Bootstrap fails on Fedora for fennec

Bad patch, please ignore
Comment on attachment 8698287 [details] [diff] [review]
Mach Bootstrap fails on Fedora for fennec

bad patch, please ignore
Attachment #8698287 - Attachment is obsolete: true
Comment on attachment 8698288 [details] [diff] [review]
Mach Bootstrap fails on Fedora for fennec

># HG changeset patch
># User Brian Murray <bmurray7jhu@gmail.com>
># Date 1450142238 28800
>#      Mon Dec 14 17:17:18 2015 -0800
># Node ID 4d2d4e985072faa2132c4c3d600ff7d62b97fd42
># Parent  bbadc89e49910f344caebdbb242c74f5276291af
>Bug 1232486 - Mach Bootstrap fails on Fedora for fennec
>
>diff --git a/python/mozboot/mozboot/fedora.py b/python/mozboot/mozboot/fedora.py
>--- a/python/mozboot/mozboot/fedora.py
>+++ b/python/mozboot/mozboot/fedora.py
>@@ -1,13 +1,12 @@
> # This Source Code Form is subject to the terms of the Mozilla Public
> # License, v. 2.0. If a copy of the MPL was not distributed with this file,
> # You can obtain one at http://mozilla.org/MPL/2.0/.
> import os
>-import sys
> 
> from mozboot.base import BaseBootstrapper
> 
> 
> class FedoraBootstrapper(BaseBootstrapper):
>     def __init__(self, version, dist_id, **kwargs):
>         BaseBootstrapper.__init__(self, **kwargs)
> 
>@@ -56,20 +55,20 @@ class FedoraBootstrapper(BaseBootstrappe
> 
>     def install_browser_packages(self):
>         self.dnf_groupinstall(*self.browser_group_packages)
>         self.dnf_install(*self.browser_packages)
> 
>     def install_mobile_android_packages(self):
>         import android
> 
>-        #install android mobile packages
>+        # Install Android specific packages.
>         self.dnf_install(*self.mobile_android_packages)
> 
>-        #Get NDK and SDK
>+        # Fetch Android SDK and NDK.
>         mozbuild_path = os.environ.get('MOZBUILD_STATE_PATH', os.path.expanduser(os.path.join('~', '.mozbuild')))
>         self.sdk_path = os.environ.get('ANDROID_SDK_HOME', os.path.join(mozbuild_path, 'android-sdk-linux'))
>         self.ndk_path = os.environ.get('ANDROID_NDK_HOME', os.path.join(mozbuild_path, 'android-ndk-r10e'))
>         self.sdk_url = 'https://dl.google.com/android/android-sdk_r24.0.1-linux.tgz'
>         self.ndk_url = android.android_ndk_url('linux')
> 
>         android.ensure_android_sdk_and_ndk(path=mozbuild_path,
>                                            sdk_path=self.sdk_path, sdk_url=self.sdk_url,
Attachment #8698288 - Attachment is obsolete: true
Attachment #8698289 - Flags: feedback?(nalexander)
Attachment #8698289 - Attachment description: Mach Bootstrap fails on Fedora for fennec → Fix Mach Bootstrap on Fedora for fennec
Attachment #8698289 - Attachment description: Fix Mach Bootstrap on Fedora for fennec → Fix Mach Bootstrap on Fedora for fennec patch 4
Comment on attachment 8698289 [details] [diff] [review]
Fix Mach Bootstrap on Fedora for fennec patch 4

Review of attachment 8698289 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm.  Thanks for this, Brian!

I'll land it right now.

I'm assuming you fixed this since you want to build Fennec -- feel free to needinfo me on whatever tickets you're interested in, or contact me to find interesting projects.  #mobile is a good place to start, too.  Thanks!

::: python/mozboot/mozboot/fedora.py
@@ +13,5 @@
>          self.version = version
>          self.dist_id = dist_id
>  
>          self.group_packages = [
> +            'C Development Tools and Libraries',

OK, I see lots of references to this on the interwebs, so let's try it out.
Attachment #8698289 - Flags: feedback?(nalexander) → review+
See Also: → 1158443
https://hg.mozilla.org/mozilla-central/rev/4f55bdddc174
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla46
Yesterday when I was building Fennec on Fedora 24, `wget -q https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py` completed "sucessfully" (at least it told me my system should be ready to build Firefox for Android) but it never installed a JDK and the `~/.mozbuild/android-sdk-linux/platforms` directory was left empty and I had to manually install most of the Android toolchain by `~/.mozbuild/android-sdk-linux/tools/android`. It did manage to install various other dependencies through `dnf` however ("C Development Tools and Libraries" group, autoconf-2.13 etc.)
I also manually installed a JDK but Gradle didn't seem to like my configuration. It wasn't finding `tools.jar` because it was searching the JRE that came with the system instead of the JDK I had installed. I repaired it by symlinking `/usr/bin/java` to `/usr/java/jdk1.8.0_102/bin/java`

Maybe bootstrap is broken on Fedora 24?
This is the output of running `./mach bootstrap` after moving ~/.mozbuild/android-sdk-linux` to a temp location.

Please choose the version of Firefox you want to build:
1. Firefox for Desktop
2. Firefox for Android Artifact Mode
3. Firefox for Android

Note: (For Firefox for Android)

The Firefox for Android front-end is built using Java, the Android
Platform SDK, JavaScript, HTML, and CSS. If you want to work on the
look-and-feel of Firefox for Android, you want "Firefox for Android
Artifact Mode".

Firefox for Android is built on top of the Gecko technology
platform. Gecko is Mozilla's web rendering engine, similar to Edge,
Blink, and WebKit. Gecko is implemented in C++ and JavaScript. If you
want to work on web rendering, you want "Firefox for Android".

If you don't know what you want, start with just "Firefox for Android
Artifact Mode". Your builds will be much shorter than if you build
Gecko as well. But don't worry! You can always switch configurations
later.

You can learn more about Artifact mode builds at
https://developer.mozilla.org/en-US/docs/Artifact_builds.

Your choice:
3
Executing as root: sudo dnf groupinstall "C Development Tools and Libraries"
Last metadata expiration check: 2:09:24 ago on Thu Jul 28 21:55:05 2016.
Group 'C Development Tools and Libraries' is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!
Executing as root: sudo dnf install autoconf213 mercurial python2-devel
Last metadata expiration check: 2:09:25 ago on Thu Jul 28 21:55:05 2016.
Package autoconf213-2.13-36.fc24.noarch is already installed, skipping.
Package mercurial-3.7.3-1.fc24.x86_64 is already installed, skipping.
Package python-devel-2.7.12-1.fc24.x86_64 is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!
Executing as root: sudo dnf install ncurses-devel.i686 libstdc++.i686 zlib-devel.i686
Last metadata expiration check: 2:09:26 ago on Thu Jul 28 21:55:05 2016.
Package ncurses-devel-6.0-5.20160116.fc24.i686 is already installed, skipping.
Package libstdc++-6.1.1-3.fc24.i686 is already installed, skipping.
Package zlib-devel-1.2.8-10.fc24.i686 is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

Looks like you have the Android NDK installed at:
/home/benjamin/.mozbuild/android-ndk-r11b

--2016-07-29 00:04:32--  https://dl.google.com/android/android-sdk_r24.0.1-linux.tgz
Resolving dl.google.com (dl.google.com)... 172.217.22.174, 2a00:1450:400f:804::200e
Connecting to dl.google.com (dl.google.com)|172.217.22.174|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

Unpacking /home/benjamin/.mozbuild/mozboot/android-sdk_r24.0.1-linux.tgz...
Unpacking /home/benjamin/.mozbuild/mozboot/android-sdk_r24.0.1-linux.tgz... DONE
Your version of Mercurial (3.7.3) is sufficiently modern.
Your version of Python (2.7.12) is new enough.
Mozilla recommends a number of changes to Mercurial to enhance your
experience with it.

Would you like to run a configuration wizard to ensure Mercurial is
optimally configured?

  1. Yes
  2. No

Please enter your reply: 2

Your system should be ready to build Firefox for Android!


Paste the lines between the chevrons (>>> and <<<) into your mozconfig file:

<<<
# Build Firefox for Android:
ac_add_options --enable-application=mobile/android
ac_add_options --target=arm-linux-androideabi

# With the following Android SDK and NDK:
ac_add_options --with-android-sdk="/home/benjamin/.mozbuild/android-sdk-linux"
ac_add_options --with-android-ndk="/home/benjamin/.mozbuild/android-ndk-r11b"
>>>

Once again, `~/.mozbuild/android-sdk-linux/platforms` directory was left empty.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.