Currently product-details data are stored in the database and updates happen outside of a deployment. Because of this, it's possible for incorrect or corrupted data to enter the system and cause errors outside of our testing cycle during a deployment. To solve this we propose the following: 1. Create a Github repo to hold a current copy of the JSON data available. 2. Have a CI process that runs on a schedule (every 30min or so) that uses our product-details library to download the latest files, and if there are changes, to check them in and push them to the repo from step 1. 3. Setup the Github repo to trigger a build and deployment of the site using the new product-details data, including the full test suite that will block the production deployment on failure. In this way we should be able to catch any problems before they get to the production servers as well as be alerted to the issue so that we can fix it and get the data updated in a timely fashion.  https://github.com/mozilla/product-details-json  https://product-details.mozilla.org/1.0/  https://ci.us-west.moz.works/view/Bedrock/job/product-details-json-watch/
We've completed everything but step 3 on this. But that last step will involve a much larger change to our Jenkins setup. I'll leave this open to keep track, but add some blocking bugs as well.
We may add a step 2.5 to this. We've had problems with the integrity of data on product-details.mozilla.org, and the time stamps have been a bit wonky as well. Our process of using Jenkins to update a repo on github with the files only when there is a valid update has been working well however. So we'll update the process on bedrock to use the git repo ( from comment #0) instead of the original source. It'll act kindof like the update of security advisories.
Commit pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/8e9726a44647cce03c9f806db3931a80e962457e Bug 1296646: Convert update_product_details_files command to use git repo * Kept the data file validation just in case. Can be removed later. * Add tests for update_product_details_files command * Switch sync_all to use new product-details command