1477487 - "./mach bootstrap" with Firefox for Android fails with java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Andreas Tolfsen ❲:ato❳]

Running "./mach bootstrap" and selecting the fourth option, "Firefox
for Android", fails with a java.lang.NoClassDefFoundError:
javax/xml/bind/annotation/XmlSchema exception:

> % ./mach bootstrap
> 
> Note on Artifact Mode:
> 
> Artifact builds download prebuilt C++ components rather than building
> them locally.
> 
> Artifact builds are recommended for people working on Firefox or
> Firefox for Android frontends. They are unsuitable for those working
> on C++ code. For more information see:
> https://developer.mozilla.org/en-US/docs/Artifact_builds.
> 
> Please choose the version of Firefox you want to build:
> 1. Firefox for Desktop Artifact Mode
> 2. Firefox for Desktop
> 3. Firefox for Android Artifact Mode
> 4. Firefox for Android
> Your choice: 4
> Executing as root: sudo apt-get install autoconf2.13 build-essential nodejs python-dev python-pip python-setuptools unzip uuid zip
> 
> Reading package lists... 100%
> Reading package lists... Done
> Building dependency tree... 0%
> Building dependency tree... 50%
> Building dependency tree       
> Reading state information... Done
> autoconf2.13 is already the newest version (2.13-68).
> build-essential is already the newest version (12.5).
> uuid is already the newest version (1.6.2-1.5+b6).
> python-dev is already the newest version (2.7.15-3).
> python-pip is already the newest version (9.0.1-2.3).
> python-setuptools is already the newest version (39.2.0-1).
> unzip is already the newest version (6.0-21).
> zip is already the newest version (3.0-11+b1).
> nodejs is already the newest version (8.11.3-1nodesource1).
> 0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
> Executing as root: sudo dpkg --add-architecture i386
> Executing as root: sudo apt-get update
> 
> Hit:1 http://security.debian.org/debian-security buster/updates InRelease
>  buster/updates InRelease
> 0% [1 InRelease gpgv 38.3 kB] [Connecting to ftp.de.debian.org (141.76.2.4)]
> Hit:2 http://ftp.de.debian.org/debian buster InRelease
> 
> 0% [1 InRelease gpgv 38.3 kB]
> Hit:3 https://deb.nodesource.com/node_8.x buster InRelease
> 
> 0% [1 InRelease gpgv 38.3 kB]
> 0% [Working]
> 0% [2 InRelease gpgv 150 kB]
> 0% [Working]
> 0% [3 InRelease gpgv 4,643 B]
>              
> Reading package lists... 0%
> Reading package lists... 0%
> Reading package lists... 0%
> Reading package lists... 37%
> Reading package lists... 74%
> Reading package lists... 83%
> Reading package lists... 98%
> Reading package lists... 98%
> Reading package lists... 98%
> Reading package lists... Done
> Executing as root: sudo apt-get install default-jdk wget libncurses5:i386 libstdc++6:i386
> 
> Reading package lists... 100%
> Reading package lists... Done
> Building dependency tree... 0%
> Building dependency tree... 50%
> Building dependency tree       
> Reading state information... Done
> default-jdk is already the newest version (2:1.10-67).
> wget is already the newest version (1.19.5-1).
> libstdc++6:i386 is already the newest version (8.1.0-10).
> libncurses5:i386 is already the newest version (6.1+20180210-4).
> 0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
> 
> Looks like you have the Android NDK installed at:
> /home/ato/.mozbuild/android-ndk-r15c
> 
> 
> Looks like you have the Android SDK installed at:
> /home/ato/.mozbuild/android-sdk-linux
> We will install all required Android packages.
> 
> 
> We are now installing the following Android packages:
> platform-tools
> build-tools;26.0.2
> platforms;android-26
> extras;android;m2repository
> extras;google;m2repository
> emulator
> 
> You may be prompted to agree to the Android license. You may see some of
> output as packages are downloaded and installed.
> 
> Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
> 	at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
> 	at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
> 	at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
> 	at com.android.sdklib.tool.SdkManagerCli.main(SdkManagerCli.java:117)
> 	at com.android.sdklib.tool.SdkManagerCli.main(SdkManagerCli.java:93)
> Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
> 	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
> 	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
> 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
> 	... 5 more
> Error running mach:
> 
>     ['bootstrap']
> 
> The error occurred in code that was called by the mach command. This is either
> a bug in the called code itself or in the way that mach is calling it.
> 
> You should consider filing a bug for this issue.
> 
> If filing a bug, please include the full output of mach, including this error
> message.
> 
> The details of the failure are as follows:
> 
> CalledProcessError: Command '['/home/ato/.mozbuild/android-sdk-linux/tools/bin/sdkmanager', '--package_file=/home/ato/src/gecko/python/mozboot/mozboot/android-packages.txt']' returned non-zero exit status 1
> 
>   File "/home/ato/src/gecko/python/mozboot/mozboot/mach_commands.py", line 32, in bootstrap
>     bootstrapper.bootstrap()
>   File "/home/ato/src/gecko/python/mozboot/mozboot/bootstrap.py", line 281, in bootstrap
>     getattr(self.instance, 'install_%s_packages' % application)()
>   File "/home/ato/src/gecko/python/mozboot/mozboot/debian.py", line 127, in install_mobile_android_packages
>     self.ensure_mobile_android_packages()
>   File "/home/ato/src/gecko/python/mozboot/mozboot/debian.py", line 154, in ensure_mobile_android_packages
>     no_interactive=self.no_interactive)
>   File "/home/ato/src/gecko/python/mozboot/mozboot/android.py", line 193, in ensure_android
>     ensure_android_packages(sdkmanager_tool=sdkmanager_tool, no_interactive=no_interactive)
>   File "/home/ato/src/gecko/python/mozboot/mozboot/android.py", line 249, in ensure_android_packages
>     subprocess.check_call(args)
>   File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
>     raise CalledProcessError(retcode, cmd)

Comment 1

[Andreas Tolfsen ❲:ato❳]

Not sure what to check for here.  Do you have any clues?

I forgot to mention in the bug that the target system is Debian testing.

Comment 2

[Axel Hecht [:Pike]]

I'm hitting the same problem in macos now.

Comment 3

[Nick Alexander :nalexander [he/him]]

(In reply to Andreas Tolfsen 「:ato」 from comment #1)
> Not sure what to check for here.  Do you have any clues?
> 
> I forgot to mention in the bug that the target system is Debian testing.

My guess is that we're not getting the right version of the JDK, although it would be odd to get it on both Linux and macOS.

ato, Pike: can you tell me what java version is being invoked?  (java -version, javac -version)  I'm pretty sure we require 1.8, and we might need to be more scrupulous about _checking_ the version at bootstrap time, since OS vendors seem to be moving it forward on us frequently.

Comment 4

[Axel Hecht [:Pike]]

Fuchsia:src axelhecht$ which java 
/usr/bin/java
Fuchsia:src axelhecht$ java --version
java 9.0.4
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
Fuchsia:src axelhecht$ javac --version
javac 9.0.4


and then there's 

Fuchsia:src axelhecht$ ls -l /Library/Java/
total 0
drwxr-xr-x  2 root  wheel   64 Jul 15  2017 Extensions
lrwxr-xr-x  1 root  wheel   64 Jul 23 11:12 Home -> /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
drwxr-xr-x  5 root  wheel  160 Jul 23 11:12 JavaVirtualMachines
lrwxr-xr-x  1 root  wheel   62 Feb 12 20:47 MacOS -> /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/MacOS


Fuchsia:src axelhecht$ /Library/Java/Home/bin/java --version
Unrecognized option: --version
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

but


Fuchsia:src axelhecht$ /Library/Java/Home/bin/java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Comment 5

[Andreas Tolfsen ❲:ato❳]

> % java --version
> openjdk 10.0.2 2018-07-17
> OpenJDK Runtime Environment (build 10.0.2+13-Debian-1)
> OpenJDK 64-Bit Server VM (build 10.0.2+13-Debian-1, mixed mode)

> % javac --version
> javac 10.0.2

# update-java-alternatives -l
> java-1.10.0-openjdk-amd64      1101       /usr/lib/jvm/java-1.10.0-openjdk-amd64

I thought this initially had to do with multuple Java installations
on the system, but apparently not.

Comment 6

[James Graham [:jgraham]]

sudo update-java-alternatives --jre-headless --jre --set java-1.8.0-openjdk-amd64

helped me get past that point. We really ought to have some check for whether the correct jre is installed/active because it took me a long time to figure out what was going on here.

Comment 8

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Pre: Don't require outdated packages. r?agi

These are downloaded from maven.google.com by Gradle now.

Comment 9

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Pre: Bump Android SDK version.

Just hygiene, no reason to lag, no reason to expect changes.

Depends on D16134

Comment 10

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Part 0: Make GeckoView more prominent in |mach bootstrap|. r?agi

This is part of the larger shift toward GeckoView and away from
Firefox for Android.  We need GV to "show up" in these kind of
onboarding tools; here's a small first step.

Depends on D16135

Comment 11

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Part 1: Look for java and pin to Java 1.8 in |mach bootstrap|; r?agi

What is happening is that distribution JRE and JDK packages roll
forward, installing a different version of Java than what is expected.
We don't check the version installed, so sadness ensues.

Right now, we require Java 1.8 to build, but in the near future, after
Android-Gradle plugin 3.2.1+, we'll be free to use later Java
versions.

However, Android's `sdkmanager` itself requires exactly Java 1.8.  We
only require `sdkmanager` to install `emulator`, really -- everything
else will be fetched by Gradle -- but I don't want to unravel that
right now.

So let's just provide decent error messages and try to prevent the
worst of the footguns.

Depends on D16136

Comment 12

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Part 2: Make |mach bootstrap| for Android recommend toolchain clang and lld. r?agi,#firefox-build-system-reviewers

We're seeing all sorts of problems with the NDK toolchain compilers
and the various versions of the NDK.  In automation we build with r17b
and our own toolchain clang; let's try to standardize on that.

This patch is ugly because of the evolution of mozboot.  Long ago, we
passed arguments around and interpolated strings into function names
rather than setting members with the application name and whether
we're in artifact mode.  The places I needed to modify didn't have the
right data at the right time so I added it to the bootstrap instance.
I don't have the time or energy to use the instance variables
through-out: that'll have to be future follow-up.

Depends on D16137

Comment 13

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1477487 - Part 3: Make |mach bootstrap| for Android install Rust toolchains. r?agi

Depends on D16138

Comment 14

[Nick Alexander :nalexander [he/him]]

paul: on your macOS device, could you:

• apply the patch stack from phab
• invoke ./mach bootstrap --no-interactive --application-choice mobile_android
• update your mozconfig as suggested, choosing a --target you care about
• invoke ./mach build && ./mach package

and report back to me? I expect that these changes, which make using a Mozilla-produced clang that "supported configuration", will address your segfaults -- but I'd definitely like confirmation. Thank you!

Comment 15

[Paul Rouget [:paul]]

It works! Thank you.

Comment 16

[Petru-Mugurel Lingurar [:petru]]

It worked for me also but ony when targeting arm-linux-androideabi

Otherwise when targeting
x86_64-linux-android I get

0:09.40 checking whether the C compiler (/Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi -fno-short-enums -fno-exceptions -L/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib -Wl,-rpath-link=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib --sysroot=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64 -Wl,--allow-shlib-undefined -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -fuse-ld=lld) works... no
0:09.40 configure: error: installation or configuration problem: C compiler cannot create executables.
0:09.40 DEBUG: <truncated - see config.log for full output>
0:09.40 DEBUG: configure:1360: /Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi -o conftest -fno-short-enums -fno-exceptions -isystem /Users/petru.lingurar/.mozbuild/android-ndk-r17b/sysroot/usr/include/x86_64-linux-android -isystem /Users/petru.lingurar/.mozbuild/android-ndk-r17b/sysroot/usr/include -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -D__ANDROID_API__=21 -L/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib -Wl,-rpath-link=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib --sysroot=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64 -Wl,--allow-shlib-undefined -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -fuse-ld=lld conftest.c 1>&5
0:09.40 DEBUG: configure:1357:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
0:09.40 DEBUG: main(){return(0);}
0:09.40 DEBUG: ^
0:09.40 DEBUG: 1 warning generated.
0:09.40 DEBUG: ld.lld: error: cannot open crtbegin_dynamic.o: No such file or directory
0:09.40 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.40 DEBUG: ld.lld: error: unable to find library -ldl
0:09.40 DEBUG: ld.lld: error: unable to find library -lc
0:09.40 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.40 DEBUG: ld.lld: error: unable to find library -ldl
0:09.40 DEBUG: ld.lld: error: cannot open crtend_android.o: No such file or directory
0:09.40 DEBUG: clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
0:09.40 DEBUG: configure: failed program was:
0:09.40 DEBUG:
0:09.40 DEBUG: #line 1355 "configure"
0:09.40 DEBUG: #include "confdefs.h"
0:09.40 DEBUG:
0:09.40 DEBUG: main(){return(0);}
0:09.41 DEBUG: configure: error: installation or configuration problem: C compiler cannot create executables.
0:09.41 ERROR: old-configure failed
0:09.44 *** Fix above errors and then restart with "./mach build"
0:09.44 make: *** [configure] Error 1

i686-linux-android or
aarch64-linux-android
gives mainly the same error but from ld.lld just

0:09.91 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.91 DEBUG: ld.lld: error: unable to find library -lgcc

Also when trying a robocop test I see

▶ ./mach robocop --serve
Error running mach:

['robocop', '--serve']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ADBTimeoutError: args: /Users/petru.lingurar/.mozbuild/android-sdk-macosx/platform-tools/adb -s 2259f4e8480d7ece wait-for-device shell su 0 id; echo adb_returncode=$?, exitcode: None, stdout:

File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mochitest/mach_commands.py", line 572, in run_robocop
device_serial=device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 237, in verify_android_device
device = _get_device(build_obj.substs, device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 101, in _get_device
device = ADBDevice(adb=adb_path, verbose=verbose_logging, device=device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozdevice/mozdevice/adb.py", line 642, in init
self.shell_output("su 0 id", timeout=timeout).find(uid) != -1):
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozdevice/mozdevice/adb.py", line 1443, in shell_output
raise ADBTimeoutError("%s" % adb_process)

Don't really know if it's because of this patch or not. Tested on a real device. com.android.shell was granted su.
For the ADBTimeout I see there's already a ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=1510059

Comment 17

[Nick Alexander :nalexander [he/him]]

Petru: the ADB issue is almost certainly unrelated.

The logs you post look very confused -- you say you're targeting x86_64-linux-android but the invocations look like /Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi ... (i.e., regular ARM).

Can you ensure you have exactly one --target line in your mozconfig, and then mach clobber before trying again?

Thanks!

Comment 18

[Petru-Mugurel Lingurar [:petru]]

Thanks Nick.

Indeed i686-linux-android worked now and the robocop tests also (on the emulator).
I deleted other options I had in my mozconfig, and kept only the output from bootstrap so maybe that helped.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/b7b1e008be45
Pre: Don't require outdated packages. r=agi
https://hg.mozilla.org/mozilla-central/rev/9c33fda84724
Pre: Bump Android SDK version. r=agi
https://hg.mozilla.org/mozilla-central/rev/8b70ac3ef8d0
Part 0: Make GeckoView more prominent in |mach bootstrap|. r=agi
https://hg.mozilla.org/mozilla-central/rev/b1f74fc90f37
Part 1: Look for java and pin to Java 1.8 in |mach bootstrap|; r=agi
https://hg.mozilla.org/mozilla-central/rev/bbfe0cbb90b4
Part 2: Make |mach bootstrap| for Android recommend toolchain clang and lld. r=agi,froydnj,#firefox-build-system-reviewers
https://hg.mozilla.org/mozilla-central/rev/dc41aecedaac
Part 3: Make |mach bootstrap| for Android install Rust toolchains. r=agi

Comment 20

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/b7b1e008be45
https://hg.mozilla.org/mozilla-central/rev/9c33fda84724
https://hg.mozilla.org/mozilla-central/rev/8b70ac3ef8d0
https://hg.mozilla.org/mozilla-central/rev/b1f74fc90f37
https://hg.mozilla.org/mozilla-central/rev/bbfe0cbb90b4
https://hg.mozilla.org/mozilla-central/rev/dc41aecedaac