Closed Bug 1519489 Opened 10 months ago Closed 9 months ago

Android emulator stuck in reboot loop (libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x97377001 in tid 5790 (system_server)) (Android arm 7.0 avd busted)

Categories

(Firefox for Android :: Testing, defect, P2)

defect

Tracking

()

RESOLVED FIXED
Firefox 67
Tracking Status
firefox67 --- fixed

People

(Reporter: botond, Assigned: aerickson)

References

Details

Attachments

(2 files)

Attached file Logcat

I'm not able to run Android tests on an emulator because my emulator is stuck in what appears to be a reboot loop.

Messages similar to the following keep appearing in the logcat:

12-31 16:13:08.267 5790 5790 F libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x97377001 in tid 5790 (system_server)

Attached is a logcat with several iterations of this.

I see the same problem when I start the default arm (7.0) emulator, even with --force-update. I suppose the avd is incompatible with recent versions of the android sdk.

Assignee: nobody → gbrown
Priority: -- → P2

./mach android-emulator --version 7.0 --force-update --verbose
DEBUG: Running on Linux-4.15.0-43-generic-x86_64-with-Ubuntu-18.04-bionic
DEBUG: Emulator created with type 7.0
DEBUG: emulator found at /home/gbrown/.mozbuild/android-sdk-linux/emulator/emulator
DEBUG: ...with creation time Thu Dec 13 13:53:22 2018
DEBUG: ...with SDK version in /home/gbrown/.mozbuild/android-sdk-linux/emulator/source.properties: Pkg.Revision=28.0.20
0:01.44 Fetching and installing AVD. This may take a few minutes...
DEBUG: Copied tooltool manifest /home/gbrown/src/testing/config/tooltool-manifests/androidarm_7_0/mach-emulator.manifest to /home/gbrown/.mozbuild/android-device/releng.manifest
DEBUG: Download https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py to /home/gbrown/.mozbuild/android-device/tooltool.py...
DEBUG: Downloaded https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py to /home/gbrown/.mozbuild/android-device/tooltool.py
DEBUG: INFO - untarring "AVDs-arm64v8a-android-7.0-build-2018-06-07.tar.gz"
0:20.55 Starting Android emulator running Android 7.0...
DEBUG: deleted /home/gbrown/.emulator_console_auth_token
DEBUG: Starting the emulator with this command: /home/gbrown/.mozbuild/android-sdk-linux/emulator/emulator -avd mozemulator-7.0 -gpu swiftshader_indirect -skip-adb-auth -verbose -ranchu -selinux permissive -memory 3072 -cores 4
DEBUG: Emulator output will be written to '/home/gbrown/.mozbuild/android-device/emulator.log'
DEBUG: Emulator started with pid 15673
DEBUG: Waiting for device status...
DEBUG: adb found at /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/adb
DEBUG: ...with creation time Thu Sep 13 13:52:18 2018
DEBUG: ...with SDK version in /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/source.properties: Pkg.Revision=28.0.1
DEBUG: Device status verified.
DEBUG: Checking that Android has booted...
DEBUG: adb found at /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/adb
DEBUG: ...with creation time Thu Sep 13 13:52:18 2018
DEBUG: ...with SDK version in /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/source.properties: Pkg.Revision=28.0.1

(In reply to Geoff Brown [:gbrown] from comment #1)

I suppose the avd is incompatible with recent versions of the android sdk.

(Ah, that's what changed on my end. I updated the NDK from r15c to r18b. Makes more sense now.)

Updating the emulator to 28.0.22 did not help.

DEBUG: ...with SDK version in /home/gbrown/.mozbuild/android-sdk-linux/emulator/source.properties: Pkg.Revision=28.0.22

The older arm avd works for me:

./mach android-emulator --version 4.3

Summary: Android emulator stuck in reboot loop (libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x97377001 in tid 5790 (system_server)) → Android emulator stuck in reboot loop (libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x97377001 in tid 5790 (system_server)) (Android arm 7.0 avd busted)

(In reply to Geoff Brown [:gbrown] from comment #5)

The older arm avd works for me:

./mach android-emulator --version 4.3

(To clarify: is this a suggested workaround? I've tried this and get other errors, should I pursue them?)

(In reply to Botond Ballo [:botond] from comment #6)

(In reply to Geoff Brown [:gbrown] from comment #5)

The older arm avd works for me:

./mach android-emulator --version 4.3

(To clarify: is this a suggested workaround?

I'm going to guess not, as Firefox won't even run on Android 4 any more.

Do you think I might have better luck with the x86 emulator?

Firefox's min-sdk is api 16 == Android 4.1. It definitely runs on Android 4.3.

The x86 emulator is much faster and has lots of fans, myself included. I use it on Ubuntu 18.04.1. You will need an x86 or x86_64 build of Firefox for Android, and you'll need kvm installed.

(In reply to Geoff Brown [:gbrown] from comment #8)

Firefox's min-sdk is api 16 == Android 4.1. It definitely runs on Android 4.3.

My bad; I was thinking of Android 4.0.

Here's what I get when I try to run the 4.3 emulator:

$ ./mach android-emulator --version 4.3 --verbose
DEBUG: Running on Linux-4.9.0-6-amd64-x86_64-with-debian-9.6
DEBUG: Emulator created with type 4.3
DEBUG: emulator found at /home/botond/.mozbuild/android-sdk-linux/emulator/emulator
DEBUG:   ...with creation time Fri Jan  4 16:56:49 2019
DEBUG:   ...with SDK version in /home/botond/.mozbuild/android-sdk-linux/emulator/source.properties: Pkg.Revision=28.0.22
DEBUG: AVD found at /home/botond/.mozbuild/android-device/avd/mozemulator-4.3.avd
 0:00.33 Starting Android emulator running Android 4.3...
DEBUG: deleted /home/botond/.emulator_console_auth_token
DEBUG: Starting the emulator with this command: /home/botond/.mozbuild/android-sdk-linux/emulator/emulator -avd mozemulator-4.3 -gpu swiftshader_indirect -skip-adb-auth -verbose -show-kernel
DEBUG: Emulator output will be written to '/home/botond/.mozbuild/android-device/emulator.log'
DEBUG: Emulator started with pid 40371
DEBUG: Waiting for device status...
DEBUG: adb found at /home/botond/.mozbuild/android-sdk-linux/platform-tools/adb
DEBUG:   ...with creation time Fri Jan  4 16:56:44 2019
DEBUG:   ...with SDK version in /home/botond/.mozbuild/android-sdk-linux/platform-tools/source.properties: Pkg.Revision=28.0.1
WARNING: Emulator has already completed!
WARNING: See log at /home/botond/.mozbuild/android-device/emulator.log and/or use --verbose for more information.
 0:10.46 Unable to verify that emulator is running.
 0:10.46 Use 'mach install' to install or update Firefox on your emulator.

$ cat /home/botond/.mozbuild/android-device/emulator.log
<PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value [/home/botond/.mozbuild/android-sdk-linux]!>
<emulator: Android emulator version 28.0.22.0 (build_id 5194135) (CL:0514023f0a9dfa2f6382c333f2fb30e3a17fd074)>
<emulator: Found AVD name 'mozemulator-4.3'>
<emulator: Found AVD target architecture: arm>
<emulator: argv[0]: '/home/botond/.mozbuild/android-sdk-linux/emulator/emulator'; program directory: '/home/botond/.mozbuild/android-sdk-linux/emulator'>
<emulator:  Not a directory: /home/botond/.mozbuild/android-sdk-linux/system-images/android-18/default/armeabi-v7a/>
<>
<emulator: Cannot find system image path. Please define ANDROID_SDK_ROOT>
<emulator: Auto-config: -engine classic (based on configuration)>
<emulator:  Not a directory: /home/botond/.mozbuild/android-sdk-linux/system-images/android-18/default/armeabi-v7a/>
<>

(In reply to Botond Ballo [:botond] from comment #9)

Here's what I get when I try to run the 4.3 emulator:

I could not reproduce this. Maybe something to do with environment variables??

But whatever is happening with your 4.3 emulator, it is distinct from the problem with the 7.0 avd, so, if that persists, please file a separate bug.

:aerickson -- Could you take this bug? I don't know how this happened, but from what I can tell, the current Android arm 7.0 avd just doesn't work. This avd doesn't affect continuous integration, but is used from 'mach android-emulator --version 7.0'. We should probably just throw away the existing avd and generate a new one. In terms of sdkmanager labels, the desired avd should be based on system-images;android-24;default;arm64-v8a.

Flags: needinfo?(aerickson)

Yeah, definitely.

OK, I'll make a new image and package it up.

Assignee: gbrown → aerickson
Flags: needinfo?(aerickson)

I was able to boot the current image... it just took a really long time. 30-40 minutes on a brand new 2018 MacBook Pro. After booting and quitting, boots are very fast (5 seconds), so it seems it's not broken just setting up and caching lots during the first boot.

Botond, will you please retry and give it like an hour to boot?

Assignee: aerickson → botond
Status: NEW → ASSIGNED

This is what I get after about 10 minutes:

$ ./mach android-emulator
 0:01.22 Starting Android emulator running Android 7.0...
Error running mach:

    ['android-emulator']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ADBTimeoutError: ADBDevice: /sdcard not found.

  File "/home/botond/dev/mozilla/android/mobile/android/mach_commands.py", line 712, in emulator
    if emulator.wait_for_start():
  File "/home/botond/dev/mozilla/android/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 608, in wait_for_start
    device = _get_device(self.substs, self.device_serial)
  File "/home/botond/dev/mozilla/android/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 102, in _get_device
    device = ADBDevice(adb=adb_path, verbose=verbose_logging, device=device_serial)
  File "/home/botond/dev/mozilla/android/testing/mozbase/mozdevice/mozdevice/adb.py", line 685, in __init__
    raise ADBTimeoutError("ADBDevice: /sdcard not found.")

Note, though, that I've largely stopped caring. I've switched to using the x86 emulator and have not looked back.

Assignee: botond → aerickson

This issue was discussed at this week's Mobile Testing meeting also and the general feeling was that android arm 7.0 emulator would be nice to have but not worth a great deal of effort. There are other options:

  • mach's 7.0 x86 emulator is fast and convenient
  • mach's 4.3 arm emulator is okay and provides arm emulation
  • some folks have rooted physical devices
  • make your own avd from Android Studio

wontfix?

(In reply to Botond Ballo [:botond] from comment #14)

This is what I get after about 10 minutes:

Mach will error out like this (as it only looks for X minutes), but the emulator will continue booting and will eventually make it.

(In reply to Geoff Brown [:gbrown] from comment #15)

wontfix?

Actually, I guess we should do something, but maybe that's just removing the 7.0 arm avd and option from mach.

(In reply to Andrew Erickson [:aerickson] from comment #16)

(In reply to Botond Ballo [:botond] from comment #14)

This is what I get after about 10 minutes:

Mach will error out like this (as it only looks for X minutes), but the emulator will continue booting and will eventually make it.

You're right. That said, I gave it about 80 minutes and it did not complete booting. Logcat shows similar errors to the ones from comment 0.

(In reply to Geoff Brown [:gbrown] from comment #17)

Actually, I guess we should do something, but maybe that's just removing the 7.0 arm avd and option from mach.

Yeah, that may be the best option.

:gbrown, should I prepare a diff that does that?

(In reply to Andrew Erickson [:aerickson] from comment #19)

Yes, please.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6f44a6959199
remove the android 7.0 arm emulator r=gbrown
Status: ASSIGNED → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67

I think this broke mach install on non-emulator devices.

I see

$ ./mach install

Error running mach:

['install']

...
KeyError: '7.0'

File "/home/bclary/mozilla/builds/inbound/mozilla/python/mozbuild/mozbuild/mach_commands.py", line 814, in install
verify_android_device(self, verbose=verbose)
File "/home/bclary/mozilla/builds/inbound/mozilla/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 188, in verify_android_device
emulator = AndroidEmulator('*', substs=build_obj.substs, verbose=verbose)
File "/home/bclary/mozilla/builds/inbound/mozilla/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 457, in init
self.avd_info = AVD_DICT[self.avd_type]

So, I guess it is assuming a running emulator in verify_android_device.

Depends on: 1527708
You need to log in before you can comment on or make changes to this bug.