Closed Bug 1552898 Opened 3 months ago Closed 2 months ago

Dropdown menu for FennecAndroid topcrashes is showing the wrong options

Categories

(Socorro :: General, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: lizzard, Assigned: willkg)

Details

Attachments

(3 files)

I'm looking at what should be a long list of top crashes for FennecAndroid on Nightly,
https://crash-stats.mozilla.com/topcrashers/?product=FennecAndroid&version=68.0

There are only 3 crash signatures listed and there should be a lot more.

If I change the date range to a month, still only 3 signatures,
https://crash-stats.mozilla.com/topcrashers/?product=FennecAndroid&version=68.0&days=28

Marcia just pointed out that I should be looking at https://crash-stats.mozilla.com/topcrashers/?product=FennecAndroid&version=68.0a1

So, the issue is that the dropdown list offered "68.0" as the current version when it should be showing 68.0a1.

Summary: Nightly Fennec crash data seems wrong → Dropdown menu for FennecAndroid topcrashes is showing the wrong options

We're getting crashes for "68.0":

https://crash-stats.mozilla.com/search/?product=FennecAndroid&version=68.0&date=%3E%3D2019-05-13T17%3A06%3A00.000Z&date=%3C2019-05-20T17%3A06%3A00.000Z&_facets=signature&_facets=version&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

Since Socorro treats "68.0" as greater than "68.0a1", that's showing up as the most recent version for major version 68. I'm not sure what's sending those, but that's what's messing up the code that determines what goes in the drop downs.

This is the second time in the last month bad data is causing bad choices in those drop downs. In the previous instance, I wrote this:

https://bugzilla.mozilla.org/show_bug.cgi?id=1545426#c6

I don't know offhand how to fix it in a way that doesn't have us manually fiddling with the list of featured versions. We could go back to using product details again, but that was a bunch of code that only covers 2 of the 11 products we have. I'd rather we didn't piecemeal solve this since it creates a lot of maintenance burden. I think I'm going to push thinking about that off for now.

I don't have a better heuristic in mind for figuring out featured/active versions.

Type: task → defect
Priority: -- → P3

I was talking with Ryan (cc'd) about this. He says all the active versions for FennecAndroid are wrong and it's doing more harm than good. We talked about the circumstances which I sort of cover in the previous comment. He pointed out that it's likely the version information will always be wrong. That's probably true and frustrating.

He pointed out maybe we should force all products to update product-details and pull version data from there. I don't know where that data comes from, so I'd need to look into that. It'd be nice if product-details had a single schema--they've got multiple ones and it seems like a dumping ground at the moment. For example, these two are complete different:

I think getting releng and everyone else to establish a single schema and switch all products to publishing that information will take a long time, so we need a short-term fix that isn't automatically generated.

The short-term fix has to have these properties:

  1. The information source has to be maintained by other people--not me.
  2. The information source can't be maintained in the webapp. To add this to the webapp is a ton of work: adding a new group, new permissions, web views, tests, tables, process for requesting permissions, granting permissions, rescinding permissions, plus the ongoing maintenance of handling grant requests. That's a lot of up-front and ongoing work. I don't want to do that.
  3. This has to support all the products that Socorro supports and be easy to add new products.

I vote we go super cheesy and maintain JSON files in GitHub with the active version information. GitHub manages permissions, we can run the changes through CI, we have a process for handling PRs already, anyone can edit them.

The Crash Stats site already has code for figuring out active versions and caching that information for an hour. We can adjust that slightly to pull the data from GitHub.

I think this isn't too much work to set up. It'll be better than what we currently have. It's flexible and supports new products. We can change it if it's not working out.

One bad thing is that it requires one of us to merge the PRs. That means it'll take time for new versions to show up on the site. I think that's the trade-off here until the product-details solution works.

I'm going to make this a P2 and take this to work on. Maybe I can get this changed by the end of next week.

If anyone has thoughts, let me know soon.

Assignee: nobody → willkg
Status: NEW → ASSIGNED
Priority: P3 → P2

I think a short-term option where we can submit PRs to bump the versions makes sense. That's something we could add to our release checklists.

willkg merged PR #4965: "bug 1552898: implement manual featured versions" in 1ef1ec6.

I added a FennecAndroid.json file in this which I think has the right versions. I'll verify it on stage and make sure it works the way we want it to.

Stage is showing the values from the FennecAndroid.json file on stage:

There are instructions in that directory for updating the file. The process will be something like this:

  1. someone who is not me edits the file and submits a pr (this can be done entirely in the GitHub interface)
  2. CI will validate the structure of the changes, but not the content--if someone submits bad versions, then there's nothing that will catch that
  3. the file will get reviewed and merged by either John or myself
  4. after merging, it can take as much as an hour for the cache in prod to expire; after that prod should show the updated versions

A few interesting things:

  1. the featured versions aren't limited to 3--you can have 1, or 5, or whatever
  2. featured versions can include a "cover all the betas" version--use it if that helps
  3. we don't have to have crash reports for a featured version to be featured--you can change it before versions are released

If this doesn't work, we can figure out what to do.

Long-term, I'd prefer that the product-details site has a standard schema and covers all products. Until then, this has annoyances, but is probably good enough.

Ryan, Liz: Does stage look ok to you?

Flags: needinfo?(ryanvm)
Flags: needinfo?(lhenry)

Current Nightly version for Fennec should be 68.0a1 (it's riding the ESR68 train, so it'll always be 68.x from now on). Will look over and digest the rest and clear my NI after having a chance to do so :)

I pushed this out to prod just now and the FennecAndroid page is now showing the versions specified in the FennecAndroid.json file.

Closing this out as FIXED.

Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Flags: needinfo?(ryanvm)
Flags: needinfo?(lhenry)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.