Open Bug 1453326 Opened 4 years ago Updated 3 years ago

Important volume of unhandled crashes by breakpad

Categories

(Toolkit :: Crash Reporting, enhancement)

enhancement
Not set
normal

Tracking

()

People

(Reporter: Sylvestre, Unassigned)

References

Details

Attachments

(3 files)

We saw recently a huge increase (+145%) of crashes catch by google play instead of breakpad.
The crash rate is also huge (2%)

This is causing a lot of information to be missing in socorro.

I guess we have to update breakpad to handle this.

Bug 1453325 is the most important crash not managed by this.
See Also: → 1450793
Liz is this bug distinct from the crashers we've been looking into, i.e., https://bugzilla.mozilla.org/show_bug.cgi?id=1450105 , https://bugzilla.mozilla.org/show_bug.cgi?id=1450793
Flags: needinfo?(lhenry)
It is distinct from bug 1450105.
Flags: needinfo?(lhenry)
That crash seems to be in our mozglue linker code. My best guess would be that it's crashing before we even get to load libxul, so we don't have Breakpad initialized at this point.
Anthony, this is causing a lot of crashes not to be sent to socorro, could you find someone to fix this issue? Thanks
Flags: needinfo?(ajones)
I'm not sure that there's anything fixable here, TBH. If we crash on Android before libxul is even loaded there's no way for us to handle that.
There is more discussion about mozglue in bug 1450793, and a link there to the Google issue we filed about this problem.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #3)
> That crash seems to be in our mozglue linker code. My best guess would be
> that it's crashing before we even get to load libxul, so we don't have
> Breakpad initialized at this point.

ted, would it be possible to have Breakpad stuff in a separate library we could load in first ?
Flags: needinfo?(ted)
Gabriele - do you have any thoughts on this?
Flags: needinfo?(ajones) → needinfo?(gsvelto)
Flags: needinfo?(ajones)
Unfortunately there is no simple way of moving crash reporting earlier in the startup process. It's already tricky enough to have it where it is right now. Besides breakpad is currently built within libxul so there's no simple way of using it earlier short of redesigning the way we do crash reporting (which we might do, it's just a lot of work).

I hope this is not relevant anymore since bug 1450793 has been fixed.
Flags: needinfo?(gsvelto)
(In reply to Calixte Denizet (:calixte) from comment #7)
> ted, would it be possible to have Breakpad stuff in a separate library we
> could load in first ?

It's certainly possible, but it would require a significant amount of work.
Flags: needinfo?(ted)
I'm going to conclude that this is too difficult for now. Let me know if you think otherwise.
Flags: needinfo?(ajones)
Attached image foo.png
Attached image foo2.png
Attached image release volume
Anthony, i understand it is hard but it is growing (my three attachments are the graphs on nightly, beta & release on google play)
Flags: needinfo?(ajones)
I'm guessing it is growing due to bug 1460989.
(In reply to Gabriele Svelto [:gsvelto] from comment #9)
> Unfortunately there is no simple way of moving crash reporting earlier in
> the startup process. It's already tricky enough to have it where it is right
> now. Besides breakpad is currently built within libxul so there's no simple
> way of using it earlier short of redesigning the way we do crash reporting
> (which we might do, it's just a lot of work).
> 
> I hope this is not relevant anymore since bug 1450793 has been fixed.

ISTR crash reporting is different on android, and is handled by java code. The main blocker remains breakpad being in libxul, but I don't see why we can't move it to libmozglue for android.
(In reply to Mike Hommey [:glandium] from comment #16)
> ISTR crash reporting is different on android, and is handled by java code.
> The main blocker remains breakpad being in libxul, but I don't see why we
> can't move it to libmozglue for android.

On Fennec the crash reporter client itself is a Java activity and we have a Java exception handler which is meant to deal with exceptions that aren't caught and bubble all the way up to the top level. Both the Java exception handler and Breakpad's exception handler launch the activity when they catch something. As far as Breakpad is concerned I believe we set the exception handler like on desktop in [1]. There's a whole bunch of stuff going on there including annotating the crash report with product information and whatnot. I don't know how hard it would be to move that stuff into mozglue (or at least the parts that we need) but I'm pretty sure it wouldn't be trivial.

[1] https://searchfox.org/mozilla-central/rev/dc6d85680539cb7e5fe2843d38a30a0581bfefe1/toolkit/xre/nsAppRunner.cpp#3360
This is a major project to schedule.
Flags: needinfo?(ajones)
You need to log in before you can comment on or make changes to this bug.