1642655 - Add support for Android mappings and retracing

Status: RESOLVED WONTFIX

(Socorro :: General, enhancement)

Description

[Manuel Martin]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

Release Android builds are usually obfuscated using Proguard. At the moment Crash-stats is not retracing the stack traces so we can't get the actual line numbers which makes fixing production crashes harder.

We should be able to get retraced stack traces for release builds uploading the mappings for the released builds.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Release Android builds of which product(s)? What's an example of a crash report that's problematic?

Comment 2

[Randall Barker [:rbarker]]

For Firefox Reality specifically, here is an example of a crash report https://crash-stats.mozilla.org/report/index/0f289b53-4ad8-4b15-acce-7c9c50200531

Comment 3

[Randall Barker [:rbarker]]

Additionally, it is R8 that is causing the issue. With proguard it was possible to disable the line number mangling but it is not possible with R8 and this is why mappings.txt file is required to determine the actual line number.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

The crash report is a crash in Java, so it doesn't include a minidump, so there's no stackwalking step.

I don't work on Firefox Reality, so I'm not familiar with how it's built or how crash reports work. Are you looking to get the Java stack trace in the JavaStackTrace field retraced? If not, what are you looking for?

Comment 5

[Randall Barker [:rbarker]]

Yes, we would like to be able to upload the mappings.txt file just like we upload the native symbols for our release builds so that the stack trace displayed on crash-stats for Java crashes can display the correct line numbers.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Ok, I think I understand this.

First off, Java stack trace value on Crash Stats isn't generated by Socorro--Crash Stats displays whatever is in the crash report. I've found parsing JavaStackTrace field to be tricky and error-prone. It's a free-form string field. It's entirely possible I won't be able to parse it well enough and we'll also need FirefoxReality crash reporting to add an additional field to the crash report containing a well-formed Java stack trace value. Maybe a JSON object?

Second, I'd love to know more about the mappings.txt file. Can it be public? Does it have to be private? Is there a parser for it somewhere?

I think this is a major project that involves:

1. Add a well-formed Java stack trace into the crash report.
2. Figure out where the mappings.txt file should get uploaded to. If it can be publicly available, it should probably go on archive.mozilla.org along with other build artifacts. If not, I'm not sure where it should go. Symbols is only for Breakpad symbols things.
3. Change the FirefoxReality build system to additionally upload a mappings.txt file to a place that Socorro can find given the information in a crash report.
4. Add a new rule to the Socorro processor to pull down the mappings.txt file when available, parse the mappings.txt file, parse the JavaStackTrace field, and fix the line numbers.

Does that sound right?

Comment 7

[Randall Barker [:rbarker]]

Attachment: mapping.zip

I've attached a mapping.txt file.

Comment 8

[Randall Barker [:rbarker]]

This sound correct to me. We can definitely help with some of these issues such as the well formed Java stack trace. Would just need to define what that means. I can also take care of 3. We currently use taskcluster to do our release builds so I can just add the upload after we do the native symbol uploads would just need to know where to send it.

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

What's the impact of implementing this? What happens if we don't do it? Can you do it in a Firefox addon?

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

Given that 4 months has passed and no one has replied to comment #9, I'm inclined to think this isn't important to do so maybe we should close it out.

On the off chance that this has impact, I wrote up an issue in the android-components tracker to get structured data for Java crashes:

https://github.com/mozilla-mobile/android-components/issues/8653

That would need to get done first before we could do work here.

Comment 11

[Will Kahn-Greene [:willkg] ET needinfo? me]

This is for FirefoxReality which isn't a thing anymore, so I'm WONTFIXing this.