1657668 - redo product support

Status: RESOLVED FIXED

(Socorro :: Webapp, task, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Product support in the webapp involves a db table and then a bunch of hard-coded things across the app code. This code has evolved over time as needs have changed. That makes it very difficult to add support for new products, plus it's complex and harder to maintain than it should be.

This bug covers centralizing product support into an easier-to-maintain and more flexible infrastructure.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

I spent some time looking at the code base and here's my mediocre list of all the places where the code does something product-specific:

• database model that manages products, what order to display them in, and whether they're active for the product home page
• "versions" and "featured versions" are figured out in different ways on a product-by-product basis using product_details/ files; code for figuring out versions and featured versions is in webapp-django/crashstats/crashstats/utils.py
• webapp-django/crashstats/crashstats/jinja2/crashreports/report_index.html references bugzilla_submit_url to build "file a bug" links; different products have different bug trackers and we need to support GitHub issues (bug #1657465)
• settings.VCS_MAPPINGS might be product-specific--I can't tell
• settings.DEFAULT_PRODUCT determines the default product--I'm not sure if we need this or what it's used for
• correlations are available for some products, but not others
• archivescraper scrapes builds for specific products

I think some of those should get covered by a better product settings system.

The webapp figures out featured versions by looking at the recent spat of crash reports and then determining the most recent versions of that list. That didn't work at for Fennec or Fenix for a variety of reasons, so I implemented a system where if there was a product_details/PRODUCTNAME.json file that listed the featured_versions, it would use that and not spend any time figuring things out.

Because we wanted to make it easy for non-me people to update featured versions and we wanted changes to show up on the production website quickly, we did it in a way where the system would look at those files on GitHub and cache the results with an expiry of some period of time--I think it was an hour or something.

I was thinking we could expand this system for other product-specific configuration, but I think that means we have two conflicting requirements:

1. I want to be able to test product settings changes on stage before they make it to prod.
2. We want changes to be immediately available in prod without requiring a prod deploy.

I talked to RyanVM who manages featured versions for Fennec and Roger Yang who manages featured versions for Fenix. Both of them are ok if we drop that second requirement. For Fennec, it's EOL so it's not going to get changed much. For Fenix, they've ditched one of the version numbering systems so it's possible we can go back to automated featured version figuring.

Given all that, I think the plan is something like this:

1. get rid of the crashstats_product DB table and the Django Product model
2. add files for all products in product_details/
3. implement a manager which reads the files from disk and manages them
4. rewrite all the code to use that manager

Moving forward with that now.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 5519: bug 1657668, 1558916: rework product support

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #5519: "bug 1657668, 1558916: rework product support" in 7393c7b.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I did an overhaul that covers the bulk of the product support things in comment #1:

• database model that manages products, what order to display them in, and whether they're active for the product home page
• "versions" and "featured versions" are figured out in different ways on a product-by-product basis using product_details/ files; code for figuring out versions and featured versions is in webapp-django/crashstats/crashstats/utils.py
• settings.DEFAULT_PRODUCT determines the default product--I'm not sure if we need this or what it's used for

Left over are:

• webapp-django/crashstats/crashstats/jinja2/crashreports/report_index.html references bugzilla_submit_url to build "file a bug" links; different products have different bug trackers and we need to support GitHub issues

  This work is covered in bug #1657465.

• settings.VCS_MAPPINGS might be product-specific--I can't tell; correlations are available for some products, but not others

  This should be spun off into a new bug if we need to do this.

• archivescraper scrapes builds for specific products

  I think I'm going to leave this as is. If we need to do this, we can spin off a new bug.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

This was pushed to prod in bug #1658321. Marking as FIXED. If there are problems, we'll fix those in new bugs.