Closed Bug 747787 Opened 13 years ago Closed 13 years ago

Intermittent Tegra crash in libdvm.so during tests

Categories

(Firefox for Android Graveyard :: General, defect)

Product:
Firefox for Android Graveyard
Component:
General
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(firefox16 affected, firefox17 affected, firefox18 affected, firefox19 affected)

Status:
RESOLVED INCOMPLETE
Tracking Flags:
Tracking Status
firefox16 --- affected
firefox17 --- affected
firefox18 --- affected
firefox19 --- affected

People

(Reporter: RyanVM, Unassigned)

References

Details

(Keywords: crash, intermittent-failure, Whiteboard: [native-crash][android_tier_1])

Crash Data

https://tbpl.mozilla.org/php/getParsedLog.php?id=11101118&tree=Mozilla-Inbound https://tbpl.mozilla.org/php/getParsedLog.php?id=11105970&tree=Mozilla-Aurora 0 libc.so + 0xa888 r4 = 0x0024f468 r5 = 0xafd438e8 r6 = 0xafd42328 r7 = 0xafd4372c r8 = 0x00100000 r9 = 0x8084f865 r10 = 0x10000000 fp = 0x00118280 sp = 0x100ffe28 lr = 0xafd0a86b pc = 0xafd0a888 Found by: given as instruction pointer in context 1 libdvm.so + 0x513d4 sp = 0x100ffe2c pc = 0x808513d5 Found by: stack scanning 2 libdvm.so + 0xa23f3 sp = 0x100ffe38 pc = 0x808a23f4 Found by: stack scanning 3 0 (deleted) + 0x34f sp = 0x100ffe3c pc = 0x40009350 Found by: stack scanning 4 libdvm.so + 0x513c4 sp = 0x100ffe48 pc = 0x808513c5 Found by: stack scanning 5 libdvm.so + 0xa23f3 sp = 0x100ffe50 pc = 0x808a23f4 Found by: stack scanning 6 libdvm.so + 0x513d4 sp = 0x100ffe58 pc = 0x808513d5 Found by: stack scanning 7 libdvm.so + 0xa23f3 sp = 0x100ffe5c pc = 0x808a23f4 Found by: stack scanning 8 libdvm.so + 0xa23f3 sp = 0x100ffe68 pc = 0x808a23f4 Found by: stack scanning 9 libdvm.so + 0xa6f97 sp = 0x100ffe6c pc = 0x808a6f98 Found by: stack scanning 10 libdvm.so + 0xa6f97 sp = 0x100ffe74 pc = 0x808a6f98 Found by: stack scanning 11 libdvm.so + 0x524a8 sp = 0x100ffe78 pc = 0x808524a9 Found by: stack scanning 12 0 (deleted) + 0x1456f
Crash Signature: [@ libdvm.so + 0x513d4]
Keywords: crash
Not sure whether this relates to bug 730890 or bug 750965 (after having seen them mentioned in the platform meeting minutes); scoobidiver would you mind possibly having a look and setting dependencies/crash stack fields as appropriate (not sure what the policy is within your team on related crashes etc). Thanks :-)
There are no open bugs in the trunk for crashes in libdvm, the latest one was fixed by bug 751262.
Crash Signature: [@ libdvm.so + 0x513d4] → [@ libdvm.so@0x513d4]
Whiteboard: [orange] → [orange][native-crash]
Blocks: 438871
Depends on: 730890
after turning on all the mochitests we get a LOT of these. If we cannot fix these crashes, we need to start turning off most of the mochitests again.
Android Tegra 250 profiling opt test mochitest-3 on 2012-06-19 07:45:04 PDT for push 3315d0f57505 slave: tegra-097 https://tbpl.mozilla.org/php/getParsedLog.php?id=12793242&tree=Profiling Android Tegra 250 mozilla-inbound opt test mochitest-8 on 2012-06-19 01:07:48 PDT for push 31e9fbf97eb1 slave: tegra-053 https://tbpl.mozilla.org/php/getParsedLog.php?id=12783000&tree=Mozilla-Inbound
Android Tegra 250 mozilla-inbound opt test mochitest-2 on 2012-06-19 08:31:03 PDT for push 2e8089e80e0e slave: tegra-134 https://tbpl.mozilla.org/php/getParsedLog.php?id=12794091&tree=Mozilla-Inbound Android Tegra 250 mozilla-inbound opt test mochitest-8 on 2012-06-19 08:48:35 PDT for push c26acecbad4d slave: tegra-265 https://tbpl.mozilla.org/php/getParsedLog.php?id=12794537&tree=Mozilla-Inbound
A lot of these crashes are "legitimate" crashes that are being tracked by other bugs. For example this one: (In reply to Ed Morley [:edmorley] from comment #60) > Android Tegra 250 mozilla-inbound opt test mochitest-8 on 2012-06-19 > 01:07:48 PDT for push 31e9fbf97eb1 > slave: tegra-053 > https://tbpl.mozilla.org/php/getParsedLog.php?id=12783000&tree=Mozilla- > Inbound is bug 749917. (In reply to Ed Morley [:edmorley] from comment #61) > Android Tegra 250 mozilla-inbound opt test mochitest-2 on 2012-06-19 > 08:31:03 PDT for push 2e8089e80e0e > slave: tegra-134 > https://tbpl.mozilla.org/php/getParsedLog.php?id=12794091&tree=Mozilla- > Inbound bug 763166 > Android Tegra 250 mozilla-inbound opt test mochitest-8 on 2012-06-19 > 08:48:35 PDT for push c26acecbad4d > slave: tegra-265 > https://tbpl.mozilla.org/php/getParsedLog.php?id=12794537&tree=Mozilla- > Inbound also bug 763166 (this one happens a lot on tbpl for some reason).
thanks kats, sounds like that is the majority of these failures. Are these priority bugs for you guys? Should we just hide the tests on tbpl until these known crashes are fixed?
They're rapidly moving up in priority for me, at least. I'm getting quite frustrated by these crashes and would like to debug them soon. I think the tests are still useful even with the random oranges because they do catch actual regressions as well, so I'd rather if the tests remained unhidden. However I realize it's pretty annoying for the sherriffs to keep having to star them so if you want to hide them I wouldn't object.
Severity: normal → critical
(In reply to Ed Morley (away 4th-6th July) [:edmorley] from comment #102) > https://tbpl.mozilla.org/php/getParsedLog.php?id=13349852&tree=Mozilla- > Inbound The crash stack for this one looks appropriate for this bug - SIGSEGV in libdvm - but the logcat suggests this is similar to - perhaps a dup of - bug 757468: I/GeckoScreenshot( 1631): rect: 207.683334, 13.000000, 513.000000, 507.683350 I/GeckoScreenshot( 1631): rect: 252.683334, 23.000000, 323.000000, 402.683350 I/GeckoDump( 1631): 130544 INFO TEST-PASS | /tests/layout/xul/test/test_bug563416.html | setting style.width changes clientWidth - 156 should equal 156 I/GeckoScreenshot( 1631): rect: 252.683334, 23.000000, 323.000000, 402.683350 I/GeckoDump( 1631): 130545 INFO TEST-END | /tests/layout/xul/test/test_bug563416.html | finished in 3877ms I/GeckoDump( 1631): 130546 INFO TEST-START | /tests/modules/libjar/test/mochitest/test_bug403331.html I/GeckoScreenshot( 1631): rect: -0.316666, 11.000000, 432.799988, 58030.683594 I/dalvikvm-heap( 1631): Forcing collection of SoftReferences for 1517084-byte allocation E/dalvikvm-heap( 1631): Out of memory on a 1517084-byte allocation. I/GeckoTabs( 1631): Got message: SessionHistory:New E/GeckoAppShell( 1631): >>> REPORTING UNCAUGHT EXCEPTION FROM THREAD 9 ("GeckoBackgroundThread") E/GeckoAppShell( 1631): java.lang.OutOfMemoryError E/GeckoAppShell( 1631): at java.nio.ByteBuffer.put(ByteBuffer.java:815) E/GeckoAppShell( 1631): at org.mozilla.gecko.gfx.ScreenshotLayer$ScreenshotImage.copyBuffer(ScreenshotLayer.java:142) E/GeckoAppShell( 1631): at org.mozilla.gecko.gfx.ScreenshotLayer$ScreenshotImage.setBitmap(ScreenshotLayer.java:148) E/GeckoAppShell( 1631): at org.mozilla.gecko.gfx.ScreenshotLayer.setBitmap(ScreenshotLayer.java:54) E/GeckoAppShell( 1631): at org.mozilla.gecko.gfx.LayerRenderer.setCheckerboardBitmap(LayerRenderer.java:138) E/GeckoAppShell( 1631): at org.mozilla.gecko.ScreenshotHandler$1.run(GeckoAppShell.java:2460) E/GeckoAppShell( 1631): at android.os.Handler.handleCallback(Handler.java:587) E/GeckoAppShell( 1631): at android.os.Handler.dispatchMessage(Handler.java:92) E/GeckoAppShell( 1631): at android.os.Looper.loop(Looper.java:123) E/GeckoAppShell( 1631): at org.mozilla.gecko.GeckoBackgroundThread.run(GeckoBackgroundThread.java:31)
Depends on: 757468
I checked several of the most recent logs for this bug -- they all showed OutOfMemory or "out of memory" in the logcat output.
these are all in mochitest suites for the OOM. Could it be that we need to split this in more than 8 chunks?
Splitting into more chunks might be an effective workaround. Of course, I would prefer to see resolution of the underlying problem.
well, reducing our memory is a much longer process. There might be a test or two which is the large offender and we could just disable those until a fix is figured out. mochitest-8 has a higher volume of these crashes and I believe that is where the layout tests (which do 100,000+ checks) are run. There are a few files which do the bulk of the work. There might not be a logical way to reduce the memory, but we shall see.
I've looked through a few of these logs and see at least two different classes of bugs. one is the out of memory crash referenced above. Thother is an illegal access error: 06-07 09:15:56.996 W/dalvikvm( 1838): threadid=1: thread exiting with uncaught exception (group=0x4001d820) 06-07 09:15:56.996 E/AndroidRuntime( 1838): FATAL EXCEPTION: main 06-07 09:15:56.996 E/AndroidRuntime( 1838): java.lang.IllegalAccessError: interface implementation not public 06-07 09:15:56.996 E/AndroidRuntime( 1838): at dalvik.system.DexFile.defineClass(Native Method) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at dalvik.system.DexFile.loadClassBinaryName(DexFile.java:209) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at dalvik.system.PathClassLoader.findClass(PathClassLoader.java:203) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at java.lang.ClassLoader.loadClass(ClassLoader.java:573) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at java.lang.ClassLoader.loadClass(ClassLoader.java:532) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.app.ActivityThread.handleCreateService(ActivityThread.java:2940) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.app.ActivityThread.access$3300(ActivityThread.java:125) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2087) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.os.Handler.dispatchMessage(Handler.java:99) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.os.Looper.loop(Looper.java:123) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at android.app.ActivityThread.main(ActivityThread.java:4627) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at java.lang.reflect.Method.invokeNative(Native Method) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at java.lang.reflect.Method.invoke(Method.java:521) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:868) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626) 06-07 09:15:56.996 E/AndroidRuntime( 1838): at dalvik.system.NativeStart.main(Native Method) In general a symbol-less crash in libdvm.so is pretty useless to report. I'd suggest marking this bug as invalid or incomplete and filing separate bugs for each of the java crashes that are in these logs. It would also be wonderful if the tinderbox parser could grab these stacks rather than the libdvm.so signature it currently reports, but that might be asking too much.
(In reply to Geoff Brown [:gbrown] from comment #103) > The crash stack for this one looks appropriate for this bug - SIGSEGV in > libdvm - but the logcat suggests this is similar to - perhaps a dup of - bug > 757468: So bug 757468 is resolved now and it turned out to be caused by robotium, which is logically unrelated to mochitest failures. I'll see if I can track down the OOM for mochitests....
Assignee: nobody → gbrown
(In reply to Brad Lassey [:blassey] from comment #109) > ... and filing separate bugs > for each of the java crashes that are in these logs. Opened bugs 775218 and 775227.
Depends on: 775218, 775227
As Brad suggested, I'm closing this as invalid. It was a catch-all for several different problems that now have their own bugs and we haven't seen any activity on this bug for several weeks.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → INVALID
Well, lack of activity in an Android bug means virtually nothing. I would have called https://tbpl.mozilla.org/php/getParsedLog.php?id=14207206&tree=Mozilla-Inbound this bug before we got warned off calling logs we couldn't comprehend this bug, so now I'm going to ignore it.
I'd probably call https://tbpl.mozilla.org/php/getParsedLog.php?id=14207206&tree=Mozilla-Inbound (which also lacks "out of memory", "outofmemory" and "implementation not public" in the log) this, too, if I wasn't ignoring it.
Whiteboard: [orange][native-crash] → [orange][native-crash][android_tier_1]
I'm not getting anywhere with this bug -- hopefully someone else will have better luck.
Assignee: gbrown → nobody
Depends on: 809065
Depends on: 810471
Whiteboard: [orange][native-crash][android_tier_1] → [native-crash][android_tier_1]
Depends on: 817518
This bug has been about a variety of issues, some fixed, some not - but it has turned into a general dumping ground for a few different problems, which isn't really helping. I've broken out the most recent issues I can find to other bugs (most of the logs here are bug 817518), in the hope that we're able to make more progress towards resolving them. (They'll also be able to be suggested by TBPL, since they have the test names in the summary). Marking this as INCOMPLETE for now.
No longer blocks: 438871
Status: REOPENED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → INCOMPLETE
Depends on: 817583
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.