java stack trace parser is producing "malformed" stack traces
Categories
(Socorro :: Processor, defect, P2)
Tracking
(Not tracked)
People
(Reporter: willkg, Assigned: willkg)
Details
Attachments
(2 files)
Here's an example:
https://crash-stats.mozilla.com/report/index/685ba6fa-ac2a-4c00-b668-c31010200107
The raw stack trace field looks like this:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:450)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:440)
... 1 more
Caused by: mozilla.appservices.places.PlacesConnectionBusy: Error executing SQL: database is locked
at mozilla.appservices.places.RustError.intoException(RustError.kt:12)
at mozilla.appservices.places.PlacesWriterConnection.noteObservation(PlacesConnection.kt:7)
at mozilla.components.browser.storage.sync.PlacesHistoryStorage$recordObservation$2.invokeSuspend(PlacesHistoryStorage.kt:11)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:15)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:15)
That's getting converted to:
malformed
This bug covers looking into that and fixing it and then reprocessing existing crashes with this problem.
|
Assignee | |
Comment 1•5 years ago
|
||
I bet it's the ... 1 more line that it's choking on. Grabbing this to look into now.
|
|
Assignee | |
Comment 2•5 years ago
|
||
Bug #1594734 covers the same issue, but doesn't identify "malformed" as a signature being a bug in Socorro.
|
|
Assignee | |
Comment 3•5 years ago
|
||
My theory is wrong. What's going on is that the stack trace is slightly different. The parser handles this:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:450)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:440)
But doesn't handle this:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:450)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:440)
I'll have to think about how I want to deal with this.
|
|
Assignee | |
Comment 4•5 years ago
|
||
|
|
Assignee | |
Comment 5•5 years ago
|
||
willkg merged PR #5146: "bug 1607519: fix Caused By handling in java stack trace parser" in 11117ea.
I'll push this out later this week. After that, I'll reprocess all the crash reports with "malformed" in the signature and see what else comes up.
|
|
Assignee | |
Comment 6•5 years ago
•
|
||
I found another malformed exception:
https://crash-stats.mozilla.org/report/index/304ffc16-f1cd-4e60-b8c2-748c10200109
mozilla.appservices.push.PushError: Error(
General Error: "No subscriptions created yet.")
at mozilla.appservices.push.RustError.intoException(RustError.kt:4)
at mozilla.appservices.push.PushManager.update(PushManager.kt:8)
at mozilla.components.feature.push.RustPushConnection.updateToken(Connection.kt:19)
at mozilla.components.feature.push.AutoPushFeature$onNewToken$1.invokeSuspend(AutoPushFeature.kt:10)
at mozilla.components.feature.push.AutoPushFeature$onNewToken$1.invoke(Unknown Source:10)
at mozilla.components.feature.push.AutoPushFeatureKt$launchAndTry$1.invokeSuspend(AutoPushFeature.kt:5)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:15)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)
This one is a lot harder to fix. I think I'm going to have to rewrite the parser.
|
|
Assignee | |
Comment 7•5 years ago
|
||
The fix for "Caused by" was deployed just now in bug #1608229.
I'll reprocess the rest of these tomorrow and see if there are other issues and think about comment #6.
|
|
Assignee | |
Comment 8•5 years ago
|
||
I reprocessed everything in December. We went from 13,000 or so of these to 600 or so. I spot checked them and the ones I saw were the same issue described in comment #6. I'll tackle that next, then reprocess again and see where we're at.
|
|
Assignee | |
Comment 9•5 years ago
|
||
|
|
Assignee | |
Comment 10•5 years ago
|
||
|
|
Assignee | |
Comment 11•5 years ago
|
||
I deployed this to prod just now in bug #1624629.
I reprocessed 16k crash reports in the last 3 months and now there are 6 that are "malformed".
They have (junky looking) JavaStackTrace values like this:
java.lang.NoSuchMethodError: No interface method oĆؿݺᎰBrWw8ʏߵD⎖ ` ĕѬ :&ɽǤ桞1u,پ mnϒԚ]͟Vｿ踁D 䬣҈˶J̄얷'$Ғ䁀ŢϤ* s~ F<ʆũӲH㾜()V in class Ljava/util/List; or its super classes (declaration of 'java.util.List' appears in /system/framework/core-oj.jar)
at org.mozilla.gecko.activitystream.homepanel.StreamRecyclerAdapter.Ғ䁀ŢϤ* s~ F<ʆũӲH㾜(StreamRecyclerAdapter.java:108)
at org.mozilla.gecko.activitystream.homepanel.StreamRecyclerAdapter.<init>(StreamRecyclerAdapter.java:104)
at org.mozilla.gecko.activitystream.homepanel.ActivityStreamPanel.<init>(ActivityStreamPanel.java:71)
at org.mozilla.gecko.activitystream.homepanel.ActivityStreamHomeScreen.<init>(ActivityStreamHomeScreen.java:28)
at java.lang.reflect.Constructor.newInstance0(Native Method)
at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
I'll do another pass.
|
|
Assignee | |
Comment 12•5 years ago
|
||
I checked and in the last week, there are now 7 crash reports that have "malformed" as a signature that look like comment 11. They're all in Fennec.
I think I'm going to call it a day and mark this FIXED and if someone needs me to fix that case, too, or other cases come up, we can fix them in new bugs.
Description
•