Closed Bug 691071 Opened 13 years ago Closed 13 years ago

debug build of android crash on tegra during startup

Categories

(Firefox for Android Graveyard :: General, defect)

Product:
Firefox for Android Graveyard
Component:
General
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: jmaher, Assigned: glandium)

References

Details

(Keywords: intermittent-failure, Whiteboard: [mobile_unittests][android_tier_1])

Attachments

(1 file)

dltest
3.29 KB, text/plain
Details
Taking the latest debug build of android produced from mozilla-central, I run into a crash while starting up the test: I/ActivityManager( 1021): Starting activity: Intent { act=android.intent.action.VIEW dat=http://192.168.1.109:8888/jsreftest/tests/jstests.list flg=0x10000000 pkg=org.mozilla.fennec cmp=org.mozilla.fennec/.App (has extras) } I/ActivityManager( 1021): Start proc org.mozilla.fennec for activity org.mozilla.fennec/.App: pid=4399 uid=10034 gids={3003, 1015, 1007, 1006} D/dalvikvm( 1448): GC_EXPLICIT freed 1398 objects / 162776 bytes in 95ms I/wpa_supplicant( 1335): CTRL-EVENT-STATE-CHANGE id=-1 state=1 V/WifiMonitor( 1021): Event [CTRL-EVENT-STATE-CHANGE id=-1 state=1] V/WifiStateTracker( 1021): Changing supplicant state: SCANNING ==> INACTIVE D/NetworkStateTracker( 1021): setDetailed state, old =SCANNING and new state=IDLE D/ConnectivityService( 1021): Dropping ConnectivityChange for WIFI: DISCONNECTED/IDLE D/dalvikvm( 1448): GC_EXPLICIT freed 344 objects / 53872 bytes in 75ms D/dalvikvm( 4399): GC_FOR_MALLOC freed 2023 objects / 151456 bytes in 36ms I/GeckoApp( 4399): create I/GeckoApp( 4399): Checking for an update I/GeckoApp( 4399): start I/dalvikvm( 4399): Could not find method android.hardware.Camera.getNumberOfCameras, referenced from method org.mozilla.gecko.GeckoAppShell.initCamera W/dalvikvm( 4399): VFY: unable to resolve static method 129: Landroid/hardware/Camera;.getNumberOfCameras ()I D/dalvikvm( 4399): VFY: replacing opcode 0x71 at 0x004b I/dalvikvm( 4399): Could not find method android.hardware.Camera.open, referenced from method org.mozilla.gecko.GeckoAppShell.initCamera W/dalvikvm( 4399): VFY: unable to resolve static method 131: Landroid/hardware/Camera;.open (I)Landroid/hardware/Camera; D/dalvikvm( 4399): VFY: replacing opcode 0x71 at 0x0053 D/dalvikvm( 4399): VFY: dead code 0x004e-0051 in Lorg/mozilla/gecko/GeckoAppShell;.initCamera (Ljava/lang/String;III)[I D/dalvikvm( 4399): VFY: dead code 0x0056-0176 in Lorg/mozilla/gecko/GeckoAppShell;.initCamera (Ljava/lang/String;III)[I D/dalvikvm( 4399): VFY: dead code 0x0190-0192 in Lorg/mozilla/gecko/GeckoAppShell;.initCamera (Ljava/lang/String;III)[I D/dalvikvm( 4399): Trying to load lib /data/data/org.mozilla.fennec/lib/libmozutils.so 0x4430a418 D/dalvikvm( 4399): Added shared lib /data/data/org.mozilla.fennec/lib/libmozutils.so 0x4430a418 D/dalvikvm( 4399): No JNI_OnLoad found in /data/data/org.mozilla.fennec/lib/libmozutils.so 0x4430a418, skipping init I/GeckoAppShell( 4399): env0: ANDROID_SOCKET_zygote=10 I/GeckoAppShell( 4399): env1: ANDROID_BOOTLOGO=1 I/GeckoAppShell( 4399): env2: EXTERNAL_STORAGE=/mnt/sdcard I/GeckoAppShell( 4399): env3: ANDROID_ASSETS=/system/app I/GeckoAppShell( 4399): env4: ASEC_MOUNTPOINT=/mnt/asec I/GeckoAppShell( 4399): env5: PATH=/sbin:/system/sbin:/system/bin:/system/xbin I/GeckoAppShell( 4399): env6: ANDROID_DATA=/data I/GeckoAppShell( 4399): env7: BOOTCLASSPATH=/system/framework/core.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/com.nvidia.graphics.jar I/GeckoAppShell( 4399): env8: ANDROID_PROPERTY_WORKSPACE=9,32768 I/GeckoAppShell( 4399): env9: ANDROID_ROOT=/system I/GeckoAppShell( 4399): env10: LD_LIBRARY_PATH=/system/lib I/GeckoAppShell( 4399): env11: MOZ_CRASHREPORTER_NO_REPORT=1 I/GeckoAppShell( 4399): env12: MOZ_CRASHREPORTER=1 I/GeckoAppShell( 4399): env13: XPCOM_MEM_BLOAT_LOG=/tmp/tmpvNNtBm/runreftest_leaks.log I/GeckoAppShell( 4399): env14: XPCOM_DEBUG_BREAK=stack I/GeckoAppShell( 4399): env15: MOZ_PROCESS_LOG=/tmp/tmpuRQVJPpidlog I/GeckoAppShell( 4399): env16: NO_EM_RESTART=1 I/GeckoAppShell( 4399): env17: null I/GeckoApp( 4399): resume W/InputManagerService( 1021): Starting input on non-focused client com.android.internal.view.IInputMethodClient$Stub$Proxy@4450a838 (uid=10031 pid=1448) I/GeckoSurfaceView( 4399): surface created I/GeckoSurfaceView( 4399): surfaceChanged: fmt: -1 dim: 1600 1175 I/ActivityManager( 1021): Displayed activity org.mozilla.fennec/.App: 450 ms (total 450 ms) E/GeckoLibLoad( 4399): resolved libmozalloc.so to /data/data/org.mozilla.fennec/cache/libmozalloc.so E/GeckoLibLoad( 4399): libmozalloc.so: spent 16930 E/GeckoLibLoad( 4399): resolved libnspr4.so to /data/data/org.mozilla.fennec/cache/libnspr4.so E/GeckoLibLoad( 4399): libnspr4.so: spent 38936 E/GeckoLibLoad( 4399): resolved libplc4.so to /data/data/org.mozilla.fennec/cache/libplc4.so E/GeckoLibLoad( 4399): libplc4.so: spent 27119 E/GeckoLibLoad( 4399): resolved libplds4.so to /data/data/org.mozilla.fennec/cache/libplds4.so E/GeckoLibLoad( 4399): libplds4.so: spent 22732 E/GeckoLibLoad( 4399): resolved libmozsqlite3.so to /data/data/org.mozilla.fennec/cache/libmozsqlite3.so E/GeckoLibLoad( 4399): libmozsqlite3.so: spent 247655 E/GeckoLibLoad( 4399): resolved libnssutil3.so to /data/data/org.mozilla.fennec/cache/libnssutil3.so E/GeckoLibLoad( 4399): libnssutil3.so: spent 55698 E/GeckoLibLoad( 4399): resolved libnss3.so to /data/data/org.mozilla.fennec/cache/libnss3.so E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libnss3.so because Cannot load library: link_image[2046]: failed to link libnss3.so E/GeckoLibLoad( 4399): libnss3.so: spent 276113 E/GeckoLibLoad( 4399): resolved libssl3.so to /data/data/org.mozilla.fennec/cache/libssl3.so E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libssl3.so because Cannot load library: link_image[2046]: failed to link libssl3.so E/GeckoLibLoad( 4399): libssl3.so: spent 93572 E/GeckoLibLoad( 4399): resolved libsmime3.so to /data/data/org.mozilla.fennec/cache/libsmime3.so E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libsmime3.so because Cannot load library: link_image[2046]: failed to link libsmime3.so E/GeckoLibLoad( 4399): libsmime3.so: spent 58890 E/GeckoLibLoad( 4399): resolved libxul.so to /data/data/org.mozilla.fennec/cache/libxul.so E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libxul.so because Cannot load library: link_image[2046]: failed to link libxul.so E/GeckoLibLoad( 4399): libxul.so: spent 4786889 E/GeckoLibLoad( 4399): resolved libxpcom.so to /data/data/org.mozilla.fennec/cache/libxpcom.so E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libxpcom.so because Cannot load library: link_image[2046]: failed to link libxpcom.so E/GeckoLibLoad( 4399): libxpcom.so: spent 18018 E/GeckoLibLoad( 4399): resolved libnssckbi.so to /data/data/org.mozilla.fennec/cache/libnssckbi.so E/GeckoLibLoad( 4399): libnssckbi.so: spent 176333 E/GeckoLibLoad( 4399): resolved libfreebl3.so to /data/data/org.mozilla.fennec/cache/libfreebl3.so E/GeckoLibLoad( 4399): libfreebl3.so: spent 56058 E/GeckoLibLoad( 4399): resolved libsoftokn3.so to /data/data/org.mozilla.fennec/cache/libsoftokn3.so E/GeckoLibLoad( 4399): libsoftokn3.so: spent 81144 E/GeckoLibLoad( 4399): Couldn't get a handle to libxul! E/GeckoLibLoad( 4399): Loaded libs in 5959ms total, 700ms user, 510ms system, 22 faults I/DEBUG ( 937): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** I/DEBUG ( 937): Build fingerprint: 'nvidia/harmony/harmony/harmony:2.2/FRF91/20101105.183900:eng/test-keys' I/DEBUG ( 937): pid: 4399, tid: 4409 >>> org.mozilla.fennec <<< I/DEBUG ( 937): signal 11 (SIGSEGV), fault addr 00000000 I/DEBUG ( 937): r0 00205048 r1 44384520 r2 42944fc4 r3 00000000 I/DEBUG ( 937): r4 47c97b70 r5 00000000 r6 44384520 r7 47c97b40 I/DEBUG ( 937): r8 47c97b50 r9 42944f84 10 42944f70 fp 00221918 I/DEBUG ( 937): ip 80c19361 sp 47c97b40 lr 80c19377 pc 00000000 cpsr a0000010 I/DEBUG ( 937): d0 0000005042a00000 d1 0000000042a00000 I/DEBUG ( 937): d2 3f8000003f800000 d3 c16da00040718000 I/DEBUG ( 937): d4 4113200000000000 d5 000000004bb50000 I/DEBUG ( 937): d6 c48ee00000093200 d7 3eaaaaab3f800000 I/DEBUG ( 937): d8 0000000000000000 d9 0000000000000000 I/DEBUG ( 937): d10 0000000000000000 d11 0000000000000000 I/DEBUG ( 937): d12 0000000000000000 d13 0000000000000000 I/DEBUG ( 937): d14 0000000000000000 d15 0000000000000000 I/DEBUG ( 937): scr 80000012 I/DEBUG ( 937): I/DEBUG ( 937): #00 pc 00000000 I/DEBUG ( 937): #01 lr 80c19377 /data/data/org.mozilla.fennec/lib/libmozutils.so I/DEBUG ( 937): I/DEBUG ( 937): code around pc: I/DEBUG ( 937): I/DEBUG ( 937): code around lr: I/DEBUG ( 937): 80c19354 46180300 0718f107 bd8046bd b082b580 I/DEBUG ( 937): 80c19364 6078af00 4b056039 681b447b 68396878 I/DEBUG ( 937): 80c19374 f1074798 46bd0708 bf00bd80 00010e5c I/DEBUG ( 937): 80c19384 b084b580 60f8af00 607a60b9 447b4b05 I/DEBUG ( 937): 80c19394 68f8681b 687a68b9 f1074798 46bd0710 I/DEBUG ( 937): I/DEBUG ( 937): stack: I/DEBUG ( 937): 47c97b00 00000000 I/DEBUG ( 937): 47c97b04 afd10500 /system/lib/libc.so I/DEBUG ( 937): 47c97b08 afd43728 /system/lib/libc.so I/DEBUG ( 937): 47c97b0c 0021daa0 [heap] I/DEBUG ( 937): 47c97b10 0024d958 [heap] I/DEBUG ( 937): 47c97b14 0000a000 [heap] I/DEBUG ( 937): 47c97b18 80816fc0 /system/lib/libdvm.so I/DEBUG ( 937): 47c97b1c 00221918 [heap] I/DEBUG ( 937): 47c97b20 42944f70 I/DEBUG ( 937): 47c97b24 afd0be23 /system/lib/libc.so I/DEBUG ( 937): 47c97b28 0021da70 [heap] I/DEBUG ( 937): 47c97b2c 0022c3f8 [heap] I/DEBUG ( 937): 47c97b30 00001400 I/DEBUG ( 937): 47c97b34 00221918 [heap] I/DEBUG ( 937): 47c97b38 df002777 I/DEBUG ( 937): 47c97b3c e3a070ad I/DEBUG ( 937): #00 47c97b40 44384520 /dev/ashmem/mspace/dalvik-heap/2 (deleted) I/DEBUG ( 937): 47c97b44 00205048 [heap] I/DEBUG ( 937): 47c97b48 42944f8c I/DEBUG ( 937): 47c97b4c 80816df8 /system/lib/libdvm.so I/DEBUG ( 937): 47c97b50 00221918 [heap] I/DEBUG ( 937): 47c97b54 4125edb8 /dev/ashmem/dalvik-LinearAlloc (deleted) I/DEBUG ( 937): 47c97b58 44384520 /dev/ashmem/mspace/dalvik-heap/2 (deleted) I/DEBUG ( 937): 47c97b5c 42944f84 I/DEBUG ( 937): 47c97b60 80816fc0 /system/lib/libdvm.so I/DEBUG ( 937): 47c97b64 00221918 [heap] I/DEBUG ( 937): 47c97b68 47c97b70 I/DEBUG ( 937): 47c97b6c 80844977 /system/lib/libdvm.so I/DEBUG ( 937): 47c97b70 42944f84 I/DEBUG ( 937): 47c97b74 42936348 /data/dalvik-cache/data@app@org.mozilla.fennec-1.apk@classes.dex I/DEBUG ( 937): 47c97b78 80c19361 /data/data/org.mozilla.fennec/lib/libmozutils.so I/DEBUG ( 937): 47c97b7c 47c97c28 I/DEBUG ( 937): 47c97b80 47c97c28 I/DEBUG ( 937): 47c97b84 00000001 I/WindowManager( 1021): WIN DEATH: Window{444a0a48 org.mozilla.fennec/org.mozilla.fennec.App paused=false} I/ActivityManager( 1021): Process org.mozilla.fennec (pid 4399) has died. I/WindowManager( 1021): WIN DEATH: Window{44468b18 SurfaceView paused=false}
There was a fix for an Android debug startup crash checked in yesterday (bug 690856), but it looks like builds are still crashing.
Depends on: 690856
:blassey had mentioned a fix for debug builds and I waited until that fix produced a debug android build from m-c and tested it.
Which build was that exactly?
# cat /data/data/org.mozilla.fennec/application.ini [App] Vendor=Mozilla Name=Fennec Version=10.0a1 BuildID=20110930120933 SourceRepository=http://hg.mozilla.org/mozilla-central SourceStamp=1463dc6308a8 ID={a23983c0-fd0e-11dc-95ff-0800200c9a66} [Gecko] MinVersion=1.9.2b5pre MaxVersion=10.0a1 [XRE] EnableExtensionManager=1 [Crash Reporter] Enabled=1 ServerURL=https://crash-reports.mozilla.com/submit #
This is a call on a null pointer, which happens in the Java_org_mozilla_gecko_GeckoAppShell_nativeRun wrapper function in libmozutils.so, because it couldn't get the corresponding function from libxul.so. It can't do so because libxul.so wasn't loaded, because a bunch of libs weren't loaded. All in all, figure out why you get this: E/GeckoLibLoad( 4399): Couldn't load /data/data/org.mozilla.fennec/cache/libnss3.so because Cannot load library: link_image[2046]: failed to link libnss3.so and you'll have it figured. I'll take a look on monday.
Interestingly, on my Asus Transformer, the same build doesn't crash in the main process, but the content process crashes in a similar way, but with a slightly different root cause: 10-02 00:04:26.940 E/GeckoLibLoad( 3019): Couldn't load /data/data/org.mozilla.fennec/cache/libnss3.so because Cannot load library: reloc_library[1315]: 3019 cannot locate '__aeabi_llsl'...
I can reliably reproduce with that build, but I can't reproduce on latest tinderbox debug build nor on a custom debug build (neither current trunc nor revision 1463dc6308a8) :(
ok, I tried a build from this morning and it seems to be working fine. I assume we can close this bug?
No :( I just tried a m-i build (704f37801611) and it crashes too.
Attached file dltest —
On my Nexus S, depending on the build used, putting the attached dltest file in /data/local, the following command works or fails: $ LD_LIBRARY_PATH=/data/data/org.mozilla.fennec/lib:/data/data/org.mozilla.fennec/cache /data/local/dltest /data/data/org.mozilla.fennec/cache/libnss3.so When it fails, I get the following message: Cannot load library: reloc_library[1311]: <pid> cannot locate '__aeabi_llsl'... All libnss3.so I tested (working and failing) do have an undefined __aeabi_llsl symbol, and all have a relocation for it. In theory, if the symbol could really not be located, we should see the error on all debug builds, but we don't.
So, here is what probably happens (you've got to love the toolchain): - depending on optimization, other-licenses/android/linker_format.o requires __aeabi_llsl. - when linking libmozutils.so, the linker pulls __aeabi_llsl from libgcc.a, and includes it. - when linking libnss3.so, as some of its objects use __aeabi_llsl, the linker searches it in the various libraries, and finds it in libmozutils.so. It thus marks it undefined, generates a PLT for that function, and a corresponding GOT entry. - now, something changes that makes other-licenses/android be rebuilt, and not include __aeabi_llsl. - but libnss3.so is not rebuilt, so __aeabi_llsl is still undefined. - at runtime, the linker goes through all relocations of libnss3.so, and can't find __aeabi_llsl in any of the libraries already loaded, since it's not in libmozutils.so anymore. Workaround: clobber and rebuild. Notes: - clobber builds are unaffected. This includes nightlies. But we don't have debug nightly builds. - on opt builds, we use our custom linker to load our libs instead of the system linker. Our linker does one thing that the system linker doesn't: it does dynamic resolution of symbols. IOW, we only resolve (some) symbols when they are needed. In practice, for this problem, this makes possible problem happen when __aeabi_llsl happens to be needed at run time, which, fortunately, seems to be not that often. This is why, by modifying a failing build, with an hex editor, so that it doesn't extract libraries and thus use our linker, I could get a working fennec. It would probably have crashed kind of randomly, much later. - I had another kind of related issue on Debian powerpc, with a symbol that comes from libgcc.a and was defined by another library at link time. I think it's utterly stupid to apply common linking rules for things that are pretty much internal to the toolchain. I think the best way to avoid this would be to avoid __aeabi_* symbols to be exported from libmozutils.so.
Blocks: 438871
glandium can you take a look at this?
Assignee: nobody → mh+mozilla
it seems that doing a clobber build resolves this.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Resolution: FIXED → INVALID
Whiteboard: [orange][mobile_unittests][android_tier_1] → [mobile_unittests][android_tier_1]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: