Include product-details files in docker images and update with deployment

NEW
Unassigned

Status

www.mozilla.org
Bedrock
a year ago
a year ago

People

(Reporter: pmac, Unassigned)

Tracking

Production

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [PBL])

Currently product-details data are stored in the database and updates happen outside of a deployment. Because of this, it's possible for incorrect or corrupted data to enter the system and cause errors outside of our testing cycle during a deployment. To solve this we propose the following:

1. Create a Github repo[0] to hold a current copy of the JSON data available[1].
2. Have a CI process[2] that runs on a schedule (every 30min or so) that uses our product-details library to download the latest files, and if there are changes, to check them in and push them to the repo from step 1.
3. Setup the Github repo to trigger a build and deployment of the site using the new product-details data, including the full test suite that will block the production deployment on failure.

In this way we should be able to catch any problems before they get to the production servers as well as be alerted to the issue so that we can fix it and get the data updated in a timely fashion.

[0] https://github.com/mozilla/product-details-json
[1] https://product-details.mozilla.org/1.0/
[2] https://ci.us-west.moz.works/view/Bedrock/job/product-details-json-watch/

Updated

a year ago
Whiteboard: [PBL]
We've completed everything but step 3 on this. But that last step will involve a much larger change to our Jenkins setup. I'll leave this open to keep track, but add some blocking bugs as well.
Duplicate of this bug: 1294543
We may add a step 2.5 to this. We've had problems with the integrity of data on product-details.mozilla.org, and the time stamps have been a bit wonky as well. Our process of using Jenkins to update a repo on github with the files only when there is a valid update has been working well however. So we'll update the process on bedrock to use the git repo ([0] from comment #0) instead of the original source. It'll act kindof like the update of security advisories.

Comment 4

a year ago
Commit pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/8e9726a44647cce03c9f806db3931a80e962457e
Bug 1296646: Convert update_product_details_files command to use git repo

* Kept the data file validation just in case. Can be removed later.
* Add tests for update_product_details_files command
* Switch sync_all to use new product-details command
You need to log in before you can comment on or make changes to this bug.