1474037 - Add a category for FirefoxReality project in crash-stats

Status: RESOLVED FIXED

(Socorro :: General, task, P2)

Description

[Daosheng Mu[:daoshengmu]]

Firefox Reality team is working on Crash Reporter and wanna choose Socorro (https://crash-stats.mozilla.com/home/product/Firefox) as the dashboard.

Firefox Reality is an Android application based on GeckoView. I am worried about it's crash reports would be treated as an FennecAndroid application.

Therefore, I would like to discuss if it is possible to add a product category at Socorro / crash-stats server for Firefox Reality.

Currently, I only see Firefox, FennecAndroid, and Fennec on the list. We might could think about adding a "FirefoxReality" category for showing the collected crash reports.

Comment 1

[Lonnen :lonnen]

Hey Daosheng -- Does Firefox Reality generate a minidump when it crashes? Socorro is specialized to minidump reporting.

Have you considered using Sentry? Klar, Focus, and other GeckoView projects seem to be using Sentry instead.

see also: 
* https://bugzilla.mozilla.org/show_bug.cgi?id=1433968
* https://github.com/mozilla-mobile/focus-android/issues/2326

Comment 2

[Daosheng Mu[:daoshengmu]]

(In reply to Lonnen :lonnen from comment #1)
> Hey Daosheng -- Does Firefox Reality generate a minidump when it crashes?
> Socorro is specialized to minidump reporting.
> 
> Have you considered using Sentry? Klar, Focus, and other GeckoView projects
> seem to be using Sentry instead.
> 
> see also: 
> * https://bugzilla.mozilla.org/show_bug.cgi?id=1433968
> * https://github.com/mozilla-mobile/focus-android/issues/2326

We are trying to generate the minidump files and cooperate with http://symbols.mozilla.org/ project. I know if we use `Sentry` would be more simple. However, after some internal discussions, we choose to use Socorro instead of Sentry because we wanna our crash report and GeckoView can be placed at the same place and dashboard.

Comment 3

[Daosheng Mu[:daoshengmu]]

It seems like Bug 1433968 works on GeckoView crash reporting for Socorro.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm back! I've read through the comments here and links and I have a few questions:

1. What's the MOZ_APP_NAME and product name that you're using?

2. Where is build information for Firefox Reality going to be posted?

Comment 5

[Daosheng Mu[:daoshengmu]]

What do they look like for FennecAndroid? I think ours would be very similar with FennecAndroid's. Our app id in APK is "org.mozilla.vrbrowser" and the product name is "FirefoxReality".

The build info we post here, https://github.com/MozillaReality/FirefoxReality. For the executable files, you can jump to the bottom of the page and click "Build results", we use Taskcluster to help us build our APKs once there is any update.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

The MOZ_APP_NAME for Firefox for Android is "fennec". It's the "appname" value in the mozinfo.json file:

https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-central-android-api-16/fennec-63.0a1.multi.android-arm.mozinfo.json

Socorro currently pulls build information from archive.mozilla.org, though we're switching to Buildhub (https://mozilla-services.github.io/buildhub/) in the next few months. I don't see build information for FirefoxReality there:

https://archive.mozilla.org/pub/

Socorro pulls active version information from product-details.mozilla.org. For example, the Firefox for Android data is here:

https://product-details.mozilla.org/1.0/mobile_versions.json

I don't see Firefox Reality data there:

https://product-details.mozilla.org/1.0/

Given that you're using taskcluster to do builds, can you also get the build information and product details information into the right places?

If not, then I'm not sure how this is going to work and I'll have to think about it. It'll probably require engineering work on Socorro to support Firefox Reality since it's really different than the products we currently support.

Comment 7

[Daosheng Mu[:daoshengmu]]

Just curious, for the general case, if we don't have these two information, where will the crash reports of FirefoxReality be classified?

For the mozinfo.json, we can generate one. Do we need to upload our achieve files to 
https://archive.mozilla.org/pub/ or you can fetch it in Taskcluster?

Regarding to mobile_versions.json, does it updated by someone else? I have no idea how to update this file? Should I try to contact Release engineering team?

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

If we don't have build and product information, then we can set Socorro up to collect crash reports, but the product won't show up in the menus you're looking for it to show up in without engineering effort to support however it is that Firefox Reality is doing releases.

It's definitely worth talking to whoever is doing your releases and find out if they can produce the same artifacts that Firefox and Firefox for Android are producing that I mentioned in comment #6.

Comment 9

[Daosheng Mu[:daoshengmu]]

Thanks, I am going to chat with release engineering team to ask some assistance for producing this kind of information.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

Daosheng: How's it going? Have you made any headway or decisions or anything like that?

Comment 11

[Daosheng Mu[:daoshengmu]]

Hi Will,
Sorry for reply late, currently we start to talk to the release engineers. Hope it would be ready in August. I will make you know when we are done.


Thanks.

Comment 12

[Lars Bergstrom (:larsberg)]

:willkg We will definitely add the product info, but some of it is still in flight with the marketing GTM plan. We will also have it localized and put it up on https://l10n.mozilla-community.org/stores_l10n/.

:dbrown, do you have info on the GTM text for the stores?

Comment 13

[Will Kahn-Greene [:willkg] ET needinfo? me]

Whoops--the build info example link in comment #6 is wrong. It should be this one:

https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-central-android-api-16/fennec-63.0a1.multi.android-arm.json

For product information, I just need a product name. I don't need anything else.

Comment 14

[Will Kahn-Greene [:willkg] ET needinfo? me]

I've been doing some work and I'm pretty sure we can reduce the requirements for adding a new product to Socorro. We'll test it out on Firefox Reality (or Focus--whichever gets there first).

All we need is the ProductName value that Firefox Reality is using in its crash reports. Earlier, I said this is taken from MOZ_APP_NAME in Firefox and Fennec. That turns out to be wrong--it's from MOZ_APP_BASENAME. It's added as an annotation by the crash reporter when it's building the crash report to send to Socorro.

I think parts of the Crash Stats webapp won't work without build and active version information, but I want to suss those out and fix them.

Comment 15

[Chris Peterson [:cpeterson]]

Daosheng, GeckoView bug 1482227 will enable an app like FxR or Focus to set a custom ProductName.

Comment 16

[Daosheng Mu[:daoshengmu]]

Sounds great. The ProductName needs to wait for the API from GV to let us set it.

Chris, Do we still need ProductID currently? Does the ProductID also come from compile-time and need wait for GV's API?

Comment 17

[Dan Brown]

App store copy available at link below:

https://docs.google.com/document/d/1CRFWMoyn4cLV38Ax0fgojMc8GGwQJp849ZeC35F5d5k/edit?usp=sharing

Comment 18

[Daosheng Mu[:daoshengmu]]

If IIUC, FirefoxReality is the product name. I am going to set it via GV API for GV crash reports. (Bug 1482227)

Comment 19

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to add FirefoxReality to Socorro.

Comment 20

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4578: bug 1474037: add support for FirefoxReality

Comment 21

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 277: bug 1474037: add support for FirefoxReality (antenna)

Comment 22

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/antenna

https://github.com/mozilla-services/antenna/commit/1fa08e57cf21ddbb37e861f784ff1e33b5b326f1
bug 1474037: add support for FirefoxReality

https://github.com/mozilla-services/antenna/commit/589ddd0405a14d95b25dde4d5c08b4a99e2b3352
Merge pull request #277 from willkg/1474037-firefoxreality

bug 1474037: add support for FirefoxReality

Comment 23

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/4273c2120d34135c72ef082e76ec7731f2a3cb5c
bug 1474037: add support for FirefoxReality

https://github.com/mozilla-services/socorro/commit/85b7d4b331e8f00cc440ab505c6d293d084e8005
Merge pull request #4578 from willkg/1474037-firefoxreality

bug 1474037: add support for FirefoxReality

Comment 24

[Will Kahn-Greene [:willkg] ET needinfo? me]

The code landed, it was deployed to stage, I verified FirefoxReality crash reports are collected, processed, and supported. Then we pushed it all to prod.

So FirefoxReality should be all set on Socorro/Crash-Stats. If you encounter any problems, please let me know!

Marking as FIXED.

Comment 25

[Daosheng Mu[:daoshengmu]]

Hi Will,
I can't find any reports so far, https://crash-stats.mozilla.com/search/?product=FirefoxReality&date=%3E%3D2018-09-05T15%3A55%3A46.000Z&date=%3C2018-09-12T15%3A55%3A46.000Z&page=1&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature. Besides, there is only one report currently, https://crash-stats.allizom.org/topcrashers/?product=FirefoxReality&days=28&process_type=any, but it doesn't have very sufficient information.

I am curious if it means I still need to wait for bug 1482227 to let us rewrite the ProductName to FirefoxReality?

Comment 26

[Will Kahn-Greene [:willkg] ET needinfo? me]

The report you're seeing is a test report I sent. It has no information because it was a bare minimum set of data and included no minidump and was sent via curl.

Is FirefoxReality set up to send crash reports with "FirefoxReality" as the product name? If not, then there won't be any data in Socorro because it hasn't received any.

Comment 27

[Daosheng Mu[:daoshengmu]]

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #26)
> The report you're seeing is a test report I sent. It has no information
> because it was a bare minimum set of data and included no minidump and was
> sent via curl.
> 
> Is FirefoxReality set up to send crash reports with "FirefoxReality" as the
> product name? If not, then there won't be any data in Socorro because it
> hasn't received any.

I am curious why it is workable for Focus (Bug 1481696). Is it because you write a script to reprocess `FennecAndroid` to `Focus`? It seems like for FirefoxReality, the only way we could do is waiting for bug 1482227 lands.

Comment 28

[Will Kahn-Greene [:willkg] ET needinfo? me]

Yes, Focus crash reports are coming in as FennecAndroid and Chris said if they come in as content processes, then they're Focus crashes. There's a processor rule that converts those crashes to Focus crashes:

https://github.com/mozilla-services/socorro/blob/master/socorro/processor/mozilla_transform_rules.py#L378

Socorro doesn't have a processor rule for FirefoxReality crashes. If they're coming in as Fennec crashes, then maybe they're showing up as Focus now.

If you can find clear heuristics that I can use to differentiate, I can work on that. Otherwise you're right in that you'll have to wait until bug #1482227 lands and FirefoxReality crashes are coming in with the correct product name.

Comment 29

[Daosheng Mu[:daoshengmu]]

Our current crash reports are collected to `FennecAndroid` product. https://crash-stats.mozilla.com/search/?android_manufacturer=Oculus&date=%3E%3D2018-08-26T16%3A07%3A00.000Z&date=%3C2018-09-14T16%3A07%3A38.000Z&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature

Comment 30

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4595: fix bug 1474037: add rewrite rule for FirefoxReality

Comment 31

[Will Kahn-Greene [:willkg] ET needinfo? me]

^^^ That adds a ProductName rewrite rule to the processor similar to what we're doing with Focus. Once that lands and gets pushed to production, I can reprocess the existing crash reports.

We don't do deploys on weekends, so the earliest this can happen is Monday.

Comment 32

[Will Kahn-Greene [:willkg] ET needinfo? me]

Reopening it for the processor rewrite rule changes.

Comment 33

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/1dacd186fa771167ba8973a62c79d1c7ef16dc86
fix bug 1474037: add rewrite rule for FirefoxReality

Incoming FirefoxReality crash reports have the wrong ProductName, so they're
getting put in the FennecAndroid bin. This rewrites the ProductName for crash
reports that have Android_Manufacturer=Oculus to FirefoxReality.

https://github.com/mozilla-services/socorro/commit/7dc186fb40deb965c89b871edc5cee93bb6197b9
Merge pull request #4595 from willkg/1474037-fr-rule

fix bug 1474037: add rewrite rule for FirefoxReality

Comment 34

[Randall Barker [:rbarker]]

Crash reporting has been disabled in Firefox Reality for it's 1.0 release. We are going to look at getting it enabled next week with the correct product name. We will be shipping out on Oculus, Daydream, and HTC Wave so filtering only on a product is not going to work. Sorry for the confusion and churn around this issue.

Comment 35

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/4ecd74c606bdd81b56a0b879e9a8fae08e562b78
Revert "fix bug 1474037: add rewrite rule for FirefoxReality"

This reverts commit 1dacd186fa771167ba8973a62c79d1c7ef16dc86.

https://github.com/mozilla-services/socorro/commit/0c48b8506548293e5b1599b716956bbd5ed0d948
Merge pull request #4596 from willkg/1474037-fr-revert

fix bug 1474037: revert FirefoxReality processor rule change

Comment 36

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4596: fix bug 1474037: revert FirefoxReality processor rule change

Comment 37

[Will Kahn-Greene [:willkg] ET needinfo? me]

No problem! I want to help FirefoxReality as much as I can and doing some back-and-forth work as we collectively figure things out is ok. Let me know what else I can do to help out. :)