Closed Bug 1626418 Opened 5 years ago Closed 5 years ago

fenix 4.2.0 has no crash reports

Categories

(Socorro :: General, defect, P1)

Product:
Socorro
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

Details

My initial guesses are:

  1. crash reports aren't getting to Socorro
  2. the Socorro collector is rejecting crash reports--maybe they've got the wrong product?
  3. crash reports are getting to Socorro, but aren't showing up in the Fenix product--they're somewhere else for some reason

Grabbing this to look into as soon as I can.

Assignee: nobody → willkg
Status: NEW → ASSIGNED
Priority: -- → P1

The Fenix releases page is here:

https://github.com/mozilla-mobile/fenix/releases/tag/v4.2.0

4.2.0 released on 2020-03-25. I looked at all the crash reports for buildid >= 20200323000000:

https://crash-stats.mozilla.org/search/?build_id=%3E%3D20200323000000&product=Fenix&date=%3E%3D2020-03-28T22%3A45%3A00.000Z&date=%3C2020-03-31T22%3A45%3A00.000Z&_facets=signature&_facets=version&_facets=build_id&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-version

It's all crash reports with "Nightly..." in the version name (I need to do a prod deploy to fix that) and then a couple of 1.0.2014 versions which Roger said were dev builds.

I looked through Socorro collector logs and didn't see evidence of the collector rejecting incoming crash reports for unsupported product that had a fishy product name. It's possible I didn't look at a wide enough range or that I'm lacking imagination. The logs are not human-readable and difficult to decipher.

I looked at crash reports for all products that had a buildid >= 20200323000000. I don't see a spike of crash reports in a different product like FennecAndroid so I don't think it's misreporting the ProductName. I only looked at that because it used to a while back.

I don't see evidence that these crash reports are making it to Socorro.

Roger: When Fenix crash reporter submits a crash report, where does it get submitted to? How does it get to Socorro?

Flags: needinfo?(royang)

As a side note, I looked at the source .tar.gz files.

I downloaded the fenix-4.2.0.tar.gz file. In that tarball, is a "version.txt" file that says that it's version 4.0.0.beta.2. That seems weird, but I can't tell if it means anything. fenix-4.1.0.tar.gz has the same thing.

The releases page lists 4.1.0 as "Firefox Preview 4.1" and 4.2.0 as "Firefox Preview v4.2". Maybe the additional "v" indicates that something in the build system changed in a way that is breaking crash reporting data flow somewhere?

Summary: fenix 4.2.0 has no crashes → fenix 4.2.0 has no crash reports

Roger downloaded the apk and did aapt2 dump badging ~/Desktop/firefox_preview_v4.2.0_arm64.apk and got this:

package: name='org.mozilla.fenix.beta' versionCode='20851902' versionName='4.2.0-beta.3' platformBuildVersionName='' platformBuildVersionCode='' compileSdkVersion='28' compileSdkVersionCodename='9'
sdkVersion:'21'
targetSdkVersion:'28'
uses-permission: name='android.permission.INTERNET'
uses-permission: name='android.permission.ACCESS_NETWORK_STATE'
...

He says it looks like it's reporting as 4.2.0-beta.3 even though on Google Play/device it says the version is 4.2.0.

I see crash reports for 4.2.0-beta.3:

https://crash-stats.mozilla.org/search/?version=4.2.0-beta.3b0&product=Fenix&date=%3E%3D2020-03-25T12%3A40%3A00.000Z&date=%3C2020-04-01T12%3A40%3A00.000Z&_facets=signature&_facets=version&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

Note how that's searching for "4.2.0-beta.3b0". The "b0" part is added by Socorro processor because the version doesn't parse right so it can't figure out what beta number the item is.

Here's an example crash report:

https://crash-stats.mozilla.org/report/index/0964d13b-e08e-4a4d-b82e-012500200401

I think we need two fixes here:

  1. fix that processor rule that's adding the b0 and make it only do it for specific products (Firefox and Fennec)
  2. add a processor rule that fixes the Version for crash reports for Fenix where Version = "4.2.0-beta.3" and change it to "4.2.0"

Also, the ReleaseChannel is set to "beta". 4.1.0 had a ReleaseChannel value of "nightly". I think that's not getting set correctly, but I don't think we need to fix it.

Flags: needinfo?(royang)

I created bug #1626618 to cover fixing the BetaVersionRule. I'll do that separately.

Roger says there should be way more than 19 crash reports. Given that, I'm not sure we've figured out what's going on enough to feel confident to go write a processor rule. He's going to keep looking. I will, too.

We figured it out!

Crash reports for 4.2.0 are coming in with version 4.2.0 and release channel "beta". The Socorro processor's BetaVersionRule is seeing the release channel and then adding a "b0" to the end because that's what it does for Firefox and Fennec because they report the version as if it was a release version (which is unhelpful).

Here's a sample crash report:

https://crash-stats.mozilla.org/report/index/d59bc662-15ee-43d2-9ade-67d540200401

The raw crash has version "4.2.0" and Socorro processor fixes it to "4.2.0b0".

I'll fix the BetaVersionRule, push that out, and reprocess the existing crash reports.

Depends on: 1626618

I deployed the fix to the BetaVersionRule and reprocessed the existing 4.2.0b0 crash reports just now.

We're all set here. Marking as FIXED.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.