1570352 - ensure_java does not handle JAVA_HOME correctly due to *args passing

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

When ensure_java handles JAVA_HOME it does so using the += operator then passes that result to which. which in turn has an *args/varargs style argument. However, with the way ensure_java currently calls which, the args passed are not the unpacked members of the extra_search_dirs, but is instead the tuple itself. This results in the bootstrap failing.

STR:

• Don't have a 1.8 java on your path.
• Set JAVA_HOME to Java 1.8 dir.
• ./mach bootstrap.
• Select one of the android bootstrap options.
• Command fails.

I've tested this on a couple of machines, and I'm not sure if my linux box where this is failing has anything unique to it that's not captured in the STR above. Please let me know if any more information or me reproing is useful.

I think I should be able to create a patch for this is relatively short order.

Example output for reference:

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ MOZCONFIG=.mozconfig-android-i686-opt ./mach bootstrap

Note on Artifact Mode:

Artifact builds download prebuilt C++ components rather than building
them locally. Artifact builds are faster!

Artifact builds are recommended for people working on Firefox or
Firefox for Android frontends, or the GeckoView Java API. They are unsuitable
for those working on C++ code. For more information see:
https://developer.mozilla.org/en-US/docs/Artifact_builds.

Please choose the version of Firefox you want to build:
  1. Firefox for Desktop Artifact Mode
  2. Firefox for Desktop
  3. GeckoView/Firefox for Android Artifact Mode
  4. GeckoView/Firefox for Android
Your choice: 4
Executing as root: sudo apt-get install autoconf2.13 build-essential nodejs python-dev python-pip python-setuptools unzip uuid zip
[sudo] password for b:       
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.4ubuntu1).
python-dev is already the newest version (2.7.15~rc1-1).
python-setuptools is already the newest version (39.0.1-2).
unzip is already the newest version (6.0-21ubuntu1).
zip is already the newest version (3.0-11build1).
autoconf2.13 is already the newest version (2.13-68).
uuid is already the newest version (1.6.2-1.5build4).
nodejs is already the newest version (8.10.0~dfsg-2ubuntu0.4).
python-pip is already the newest version (9.0.1-2.3~ubuntu1.18.04.1).
The following packages were automatically installed and are no longer required:
  libclang-common-7-dev libclang1-7 libllvm7
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Executing as root: sudo apt-get install openjdk-8-jdk-headless wget
Reading package lists... Done
Building dependency tree       
Reading state information... Done
wget is already the newest version (1.19.4-1ubuntu2.2).
The following packages were automatically installed and are no longer required:
  libclang-common-7-dev libclang1-7 libllvm7
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  openjdk-8-jdk openjdk-8-jre openjdk-8-jre-headless
Suggested packages:
  openjdk-8-demo openjdk-8-source visualvm icedtea-8-plugin fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-zenhei
The following packages will be upgraded:
  openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless
4 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Need to get 37.5 MB of archives.
After this operation, 459 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jdk amd64 8u222-b10-1ubuntu1~18.04.1 [1,756 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jdk-headless amd64 8u222-b10-1ubuntu1~18.04.1 [8,267 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jre amd64 8u222-b10-1ubuntu1~18.04.1 [69.3 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jre-headless amd64 8u222-b10-1ubuntu1~18.04.1 [27.4 MB]
Fetched 37.5 MB in 5s (7,011 kB/s)                 
(Reading database ... 338065 files and directories currently installed.)
Preparing to unpack .../openjdk-8-jdk_8u222-b10-1ubuntu1~18.04.1_amd64.deb ...
Unpacking openjdk-8-jdk:amd64 (8u222-b10-1ubuntu1~18.04.1) over (8u212-b03-0ubuntu1.18.04.1) ...
Preparing to unpack .../openjdk-8-jdk-headless_8u222-b10-1ubuntu1~18.04.1_amd64.deb ...
Unpacking openjdk-8-jdk-headless:amd64 (8u222-b10-1ubuntu1~18.04.1) over (8u212-b03-0ubuntu1.18.04.1) ...
Preparing to unpack .../openjdk-8-jre_8u222-b10-1ubuntu1~18.04.1_amd64.deb ...
Unpacking openjdk-8-jre:amd64 (8u222-b10-1ubuntu1~18.04.1) over (8u212-b03-0ubuntu1.18.04.1) ...
Preparing to unpack .../openjdk-8-jre-headless_8u222-b10-1ubuntu1~18.04.1_amd64.deb ...
Unpacking openjdk-8-jre-headless:amd64 (8u222-b10-1ubuntu1~18.04.1) over (8u212-b03-0ubuntu1.18.04.1) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for desktop-file-utils (0.23+linuxmint5) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Setting up openjdk-8-jre-headless:amd64 (8u222-b10-1ubuntu1~18.04.1) ...
Setting up openjdk-8-jdk-headless:amd64 (8u222-b10-1ubuntu1~18.04.1) ...
Setting up openjdk-8-jre:amd64 (8u222-b10-1ubuntu1~18.04.1) ...
Setting up openjdk-8-jdk:amd64 (8u222-b10-1ubuntu1~18.04.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AttributeError: 'tuple' object has no attribute 'endswith'

  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/mach_commands.py", line 44, in bootstrap
    bootstrapper.bootstrap()
  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/bootstrap.py", line 430, in bootstrap
    getattr(self.instance, 'install_%s_packages' % application)()
  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/debian.py", line 128, in install_mobile_android_packages
    self.ensure_mobile_android_packages()
  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/debian.py", line 147, in ensure_mobile_android_packages
    self.ensure_java()
  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/base.py", line 804, in ensure_java
    java = self.which('java', extra_search_dirs)
  File "/home/b/projects/mozilla/mozilla-unified/python/mozboot/mozboot/base.py", line 330, in which
    test = os.path.join(path, name)
  File "/usr/lib/python2.7/posixpath.py", line 70, in join
    elif path == '' or path.endswith('/'):

Comment 1

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1570352 - Make sure ensure_java correctly passes extra search dirs to which. r?nalexander!

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/584fc57556aa
Make sure ensure_java correctly passes extra search dirs to which. r=nalexander

Comment 3

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/584fc57556aa