791419 - Firefox for Android crashes on start on Kindle Fire HD 7": Couldn't get a handle to libmozsqlite3.so

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect, P2)

Description

[M8R-7hrw9m]

Attachment: FF_crash.txt

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120905151427

Steps to reproduce:

Attempted to install and start Firefox 15 on the Kindle Fire HD 7"


Actual results:

The application started, but crashed after a few seconds. This behavior is 100% reproducible after every start of Firefox, so the application is unusable.


Expected results:

Expected to be able to use Firefox for Android without it crashing a few seconds after start up.

Comment 1

[Chris Peterson [:cpeterson]]

Thanks for the bug report.

We don't officially support Amazon's Android Appstore yet, so we haven't tested the new Kindle Fire HD tablets yet. I have an original Kindle Fire I will retest.

The log file is a huge help! Firefox is failing to load our SQLite database library:

E/GeckoLinker(10618): /data/app/org.mozilla.firefox-1.apk!/libmozsqlite3.so: Warning: relocation to NULL @0x00054d10
E/GeckoLinker(10618): /data/app/org.mozilla.firefox-1.apk!/libmozsqlite3.so: Error: relocation to NULL @0x00054bf8 for symbol "memmove"
E/GeckoLibLoad(10618): Couldn't get a handle to libmozsqlite3!
E/GeckoLibLoad(10618): Throw
E/GeckoLibLoad(10618): Load sqlite done
W/dalvikvm(10618): threadid=11: thread exiting with uncaught exception (group=0x40aa11f8)
E/GeckoAppShell(10618): >>> REPORTING UNCAUGHT EXCEPTION FROM THREAD 388 ("Gecko")
E/GeckoAppShell(10618): java.lang.Exception: Error loading sqlite libraries
E/GeckoAppShell(10618): 	at org.mozilla.gecko.GeckoAppShell.loadSQLiteLibsNative(Native Method)
E/GeckoAppShell(10618): 	at org.mozilla.gecko.GeckoAppShell.loadSQLiteLibs(GeckoAppShell.java:415)
E/GeckoAppShell(10618): 	at org.mozilla.gecko.GeckoThread.run(GeckoThread.java:46)
F/libc    (10618): Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1)

Comment 2

[Lukas Blakk [:lsblakk] use ?needinfo]

Not tracking as per comment 1.

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

Please renominate if this issue with libmozsqlite3.so is reproducible on devices we do support and there is a sign of significant user impact for this issue.

Comment 4

[Chris Peterson [:cpeterson]]

Firefox 18 works on my old Kindle Fire.

Comment 5

[Chris Peterson [:cpeterson]]

btw, I have seen similar mozglue UnsatisfiedLinkError crashes (on my Kindle Fire and other devices), but I was installing a recent Firefox build over a very old build. Uninstalling and reinstalling the new build made the problem "go away".

Comment 6

[Mark Finkle (:mfinkle) (use needinfo?)]

M8R-7hrw9m

Are you installing over an old build? Can you make sure any old install has been removed and try it again?

Comment 7

[M8R-7hrw9m]

Mark, I am not installing over an old build. I tried clearing data for Firefox as well and it didn't help.

Chris, Firefox 15 does indeed work on my old Kindle Fire as well, so this seems specific to the Kindle Fire HD os. Although, I suspect the same issue will be found on the Kindle Fire 2.

I also neglected to mention I tried Firefox 17 and 18 as well they would not start as well.

Comment 8

[Fabiano]

I have the same problem with Kindle Fire HD.
The new versions install ok, but when I run them I get Firefox running for 2-5 seconds and then it takes me be back to the app menu.
I tried older versions and the "newer" one that worked so far is Firefox 10.0, all the others fail in the same way.

Comment 9

[jacewt]

I have the same problem with Kindle Fire HD. It is unrooted, I sideload and it crashes almost right away.

Comment 10

[Marcus Jenkins]

I get the same problem on my Kindle Fire HD with fennec-15.0.multi.android-arm.apk and also fennec-18.0a2.en-US.android-arm.apk

Comment 11

[AbsoluteJoe]

I'm having the same problem launching Firefox (fennec-16.0.en-US.android-arm.apk)  on Kindle Fire HD 7.

Comment 12

[Gareth]

v10 works well, but lacks flash support (Kindle fire HD)
Any version after 10 appears to close after a few seconds.

Comment 13

[varlokkur]

Just adding that I had the same experience with the latest build of the beta, updated via the Google play store. Hope this bug is squashed soon- I'd love to try out the mobile version!

Comment 14

[Aaron Train [:aaronmt]]

This needs an owner for 19.

Comment 15

[andrig.t.miller]

(In reply to Chris Peterson (:cpeterson) from comment #4)
> Firefox 18 works on my old Kindle Fire.

Just so you know, I tested Firefox 18 on my new Kindle Fire HD (8.9 in this case, instead of 7), but it still crashes.

Comment 16

[Aaron Train [:aaronmt]]

(In reply to andrig.t.miller from comment #15)
> (In reply to Chris Peterson (:cpeterson) from comment #4)
> > Firefox 18 works on my old Kindle Fire.
> 
> Just so you know, I tested Firefox 18 on my new Kindle Fire HD (8.9 in this
> case, instead of 7), but it still crashes.

That was in reference to the older device.

If there is no patch here, assume it still crashes on every channel version.

Comment 17

[Alex Keybl [:akeybl]]

(In reply to Aaron Train [:aaronmt] from comment #14)
> This needs an owner for 19.

It is a shame that we're crashing here, but it's not a supported device (we're not in the Appstore). Sending to mfinkle to see if he can find an assignee for this reproducible crash, but not tracking for release.

Comment 18

[Aaron Train [:aaronmt]]

(In reply to Alex Keybl [:akeybl] from comment #17)
> (In reply to Aaron Train [:aaronmt] from comment #14)
> > This needs an owner for 19.
> Sending to mfinkle to see if he can find an assignee for this reproducible crash, but not tracking for release.

I only say that as it is currently 19+

Comment 19

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: logcat with extra logging

Glandium, this log is from a build with MOZ_DEBUG_LINKER set. Do you see anything useful?

Comment 20

[Mike Hommey [:glandium]]

The log in comment 1 was already pretty clear: memmove can't be found. The MOZ_DEBUG_LINKER log adds this:
E/GeckoLinker( 7104): dlsym(0xb000a190 ["liblog.so"], "memmove") = 0x0
E/GeckoLinker( 7104): dlsym(0xb000e678 ["libmozglue.so"], "memmove") = 0x0
E/GeckoLinker( 7104): dlsym(0xb000a2a8 ["libc.so"], "memmove") = 0x0
E/GeckoLinker( 7104): dlsym(0xb00095b0 ["libdl.so"], "memmove") = 0x0

In the NDK, memmove is found in libc.so... That would seem to indicate the device's libc is broken.

Comment 21

[David Cook [:dncook]]

I grabbed libc.so from the Kindle Fire HD 7" software update package. When I dump the symbol tables, memmove does have an address, although it is a weak symbol.
> 0001677c  w   DF .text	00000044 memmove
Looking at the verbose log in comment 19, all the successful calls to dlsym() are for strong symbols, and the call for memmove is the first call for a weak symbol. Could it be that dlsym() has an issue with weak symbols?

Comment 22

[Mike Hommey [:glandium]]

(In reply to ssssmemyself from comment #21)
> Could it be that dlsym() has an issue with weak symbols?

Indeed, it has.

Comment 23

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Mike Hommey [:glandium] from comment #22)
> (In reply to ssssmemyself from comment #21)
> > Could it be that dlsym() has an issue with weak symbols?
> 
> Indeed, it has.

so what's the solution here?

Comment 24

[Mike Hommey [:glandium]]

Add an implementation of memmove in libmozglue.

Comment 25

[David Cook [:dncook]]

Well, there are 48 weak, non-global symbols for important functions in libc.so. (i.e. malloc, free, strcmp...) Also, the symbols do point to real function bodies.

Would it be easier to replace dlsym(...) with code that ignores the global flag? (see https://github.com/android/platform_bionic/blob/master/linker/dlfcn.cpp#L100)

Comment 26

[Mike Hommey [:glandium]]

(In reply to ssssmemyself from comment #25)
> Well, there are 48 weak, non-global symbols for important functions in
> libc.so. (i.e. malloc, free, strcmp...) Also, the symbols do point to real
> function bodies.
> 
> Would it be easier to replace dlsym(...) with code that ignores the global
> flag? (see
> https://github.com/android/platform_bionic/blob/master/linker/dlfcn.cpp#L100)

You can't replace the libc on android. The only way this would work is to allow our custom linker to resolve symbols in system libraries, which is not trivial to implement. We may want to do that on the long run, but in the meanwhile, it's just simpler to copy/paste a memmove implementation.

Comment 27

[David Cook [:dncook]]

Good news! I added all the missing functions to libmozglue.so (copied from the Android libc source) and that did the trick! I tested out the build on a Kindle Fire HD 7 device, and it seems to work well. The device wasn't rooted, so I couldn't get the test suites to run on it unfortunately.

I'll work on packaging the changes into a patch over the next couple days.

Comment 28

[Mike Hommey [:glandium]]

As mentioned to blassey on irc, you don't need to copy the functions source. Just add wrapper functions, like:
void *__wrap_memmove(void *dest, const void *src, size_t n) { return memmove(dest, src, n); }
somewhere in mozglue/build/BionicGlue.cpp (you may want to make them conditional to building for android but *not* b2g), and add -Wl,--wrap=memmove to WRAP_FLAGS in configure.in.

Comment 29

[Rafael Gieschke]

Attachment: all weak symbols in Kindle Fire HD's libc (nm libc.so | grep " W ")

Of these functions, bcopy, and bzero (from strings.h), strtotimeval (from time.h), and mallinfo (from malloc.h) are not used in Fennec.

malloc, calloc, realloc, free, memalign, and valloc (from malloc.h) and strdup and strndup (from string.h) already get wrapped on Android ( http://lxr.mozilla.org/mozilla-central/source/configure.in#7324 ).

Comment 30

[Rafael Gieschke]

Attachment: the declarations of the remaining functions of string.h

copied from Android NDK's include files

Comment 31

[Rafael Gieschke]

Attachment: a Perl script to generate wrapper functions from the declarations

Comment 32

[Rafael Gieschke]

Attachment: patch to fix Fennec on Kindle Fire HD

The patch as generated by the Perl script, pasted into Fennec's source code. Compiles and runs fine on Kindle Fire HD.

Comment 33

[Kevin Brosnan [Ex-Mozilla]]

Rafael would you set the review flags for Glandium. You can find his email at comment 28 

https://developer.mozilla.org/en-US/docs/Developer_Guide/How_to_Submit_a_Patch#Getting_the_patch_reviewed

Comment 34

[Rafael Gieschke]

Comment on attachment 696200 [details] [diff] [review]
patch to fix Fennec on Kindle Fire HD

Okay, thanks, I'll try.

Comment 35

[Mike Hommey [:glandium]]

Comment on attachment 696200 [details] [diff] [review]
patch to fix Fennec on Kindle Fire HD

Review of attachment 696200 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozglue/build/BionicGlue.cpp
@@ +126,5 @@
>  }
> +
> +/* Amazon Kindle Fire HD's libc provides most of the 
> + * functions in string.h as weak symbols, which dlsym 
> + * cannot resolve. Thus, we must wrap these functions. 

Please remove the trailing whitespaces.

Comment 36

[Rafael Gieschke]

Attachment: patch to fix Fennec on Kindle Fire HD - trailing whitespaces removed

Comment 37

[Mike Hommey [:glandium]]

Comment on attachment 696251 [details] [diff] [review]
patch to fix Fennec on Kindle Fire HD - trailing whitespaces removed

Review of attachment 696251 [details] [diff] [review]:
-----------------------------------------------------------------

No need to request another review when the previous was +ed, provided you do the mentioned changes :)

Comment 38

[Mark Finkle (:mfinkle) (use needinfo?)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b99d9b422a3d

Comment 39

[trinaldi]

Guys, my knowlodge in Java or whatever Programming Language is being used is near zero. However I have a KFD (rooted). I can test if you want me to.

Thanks for the effort.

Comment 40

[Graeme McCutcheon [:graememcc]]

https://hg.mozilla.org/mozilla-central/rev/b99d9b422a3d

Comment 41

[Kevin Brosnan [Ex-Mozilla]]

Anyone who is interested in testing this fix you can download tomorrow's (Dec-30) nightly build. It should be available for download mid-day pacific time at http://nightly.mozilla.org

Comment 42

[andrig.t.miller]

This build works fine for me on my Kindle Fire HD 8.9.  In fact, I am adding this comment with it right now.  Also, a side comment.  This build of Firefox is faster then both Silk and Chrome.  Very nice.

Comment 43

[Tom H]

I've tested today's nightly build and can verify that it runs on both Kindle Fire HD 7 and Kindle Fire HD 8.9 models. Thanks for the good work, it looks like the issue has been resolved.

Comment 44

[M8R-7hrw9m]

Reporter here; I'm happy to add my own confirmation that the issue is fixed!

Thanks for resolving this. I don't know what I would do without Firefox on my tablet.

Comment 45

[Scoobidiver (away)]

The fix should be uplifted to Aurora.

Comment 46

[noleksin]

Working great for me!  Sadly, seems like Flash Player is becoming impossible to use.  Does anyone have a copy of the Flash Player for Android Firefox?

Comment 47

[Kartikaya Gupta (email:[e-mail])]

You can find archived versions of the Flash Player at http://helpx.adobe.com/flash-player/kb/archived-flash-player-versions.html#flash_player_archives (scroll down to the Android sections)

Comment 48

[Scoobidiver (away)]

The above link is provided in Mozilla's support forum: https://support.mozilla.org/kb/how-do-i-watch-flash-videos-firefox-android

Comment 49

[noleksin]

Thank you all- everything seems to be working, now!

Comment 50

[Tom H]

I've tested 19.0b2 (latest beta) and the issue is not fixed there - still crashes on launch. Tested on Kindle Fire HD 7.0 and 8.9 devices.

Other testing is good, the fix has been in Aurora since early January and working well on the KF HD. 20.0 alpha launches and runs fine: http://www.mozilla.org/en-US/mobile/aurora/

Can anyone verify whether this fix will make it into a future 19.0 beta or 19.0 stable release?

Comment 51

[Aaron Train [:aaronmt]]

(In reply to Tom H from comment #50)
> I've tested 19.0b2 (latest beta) and the issue is not fixed there - still
> crashes on launch. Tested on Kindle Fire HD 7.0 and 8.9 devices.
> 
> Other testing is good, the fix has been in Aurora since early January and
> working well on the KF HD. 20.0 alpha launches and runs fine:
> http://www.mozilla.org/en-US/mobile/aurora/
> 
> Can anyone verify whether this fix will make it into a future 19.0 beta or
> 19.0 stable release?

The fix is on Nightly (21), Aurora (20). The target release milestone for fixing the issue will be Firefox 20. At this point it is too late for Beta testing as we're nearing Firefox 19 release in the coming weeks. I think a fix like this benefits from longer Beta testing.

Comment 52

[Aaron Train [:aaronmt]]

(wont-fix 19?)