1282494 - Please use https://product-details.mozilla.org/1.0/ instead of the svn version

Status: VERIFIED FIXED

(www.mozilla.org :: Bedrock, defect)

Description

[Sylvestre Ledru [:Sylvestre]]

Please update bedrock to use the new version of product details.

I know it requires the django-product-details to be updated. This has been reported here:
https://github.com/mozilla/django-product-details/issues/51

Comment 1

[Paul [:pmac] McLanahan]

Shouldn't require that. The URL from which to get the files is configurable and last I checked the new site is compatible. However some of the contents of the files have changed, so we will likely have to do some work to adapt the site to work with these new data.

Comment 2

[Sylvestre Ledru [:Sylvestre]]

Let me know if you need help. Do you have examples of changed content?

Comment 3

[Paul [:pmac] McLanahan]

https://product-details.mozilla.org/firefox_primary_builds.json

vs.

http://viewvc.svn.mozilla.org/vc/libs/product-details/json/firefox_primary_builds.json?view=markup

Different data types will very likely cause exceptions. We'll just have to clean all of that up.

Comment 4

[Sylvestre Ledru [:Sylvestre]]

oh yes, do you want me to bring this back?

Comment 5

[Kohei Yoshino [:kohei]]

I thought product-details.mozilla.org would be versioned so people don't have to worry about the compatibility?

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Not sure what you mean by versioned but the new file format have a version field:
https://product-details.mozilla.org/firefox.json
https://product-details.mozilla.org/mobile_android.json
https://product-details.mozilla.org/thunderbird.json
etc

Comment 7

[Kohei Yoshino [:kohei]]

I meant the *API* could be versioned, like https://product-details.mozilla.org/1.0/firefox.json

See the Twitter API, for example, which is currently 1.1: https://dev.twitter.com/rest/public/search

Comment 8

[Kohei Yoshino [:kohei]]

So, in our case, 1.0 should offer the same or backward-compatible JSON currently served from SVN, and 1.1 or 2.0 could be a somewhat better format.

Comment 9

[Paul [:pmac] McLanahan]

If it were to at least start out as exactly like what is currently in SVN then our change would be trivial.

Comment 10

[Sylvestre Ledru [:Sylvestre]]

(In reply to Paul [:pmac] McLanahan from comment #9)
> If it were to at least start out as exactly like what is currently in SVN
> then our change would be trivial.
Implemented in bug 1288661, waiting for review, thanks!

> I meant the *API* could be versioned, like https://product-details.mozilla.org/1.0/firefox.json
This is a great idea, I should have implemented this from the start.
Could you report a bug? We should do that in a v2... Thanks!

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Actually, I am working on it and I opened bug 1289801

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Paul, please wait for bug 1289801 to be live. The content will remain the same but the URL are going to change

Comment 13

[Pascal Chevrel:pascalc]

The change was deployed and the Json files are now at https://product-details.mozilla.org/1.0/

Comment 14

[Pascal Chevrel:pascalc]

Attachment: Patch on GitHub

pmac, I wrote the patch for Bedrock to override the default url in the module, tested locally and the new json files correspond to what we have in Ship It (Nightly data added to the files). Manual testing works and tests are passing.

Comment 15

[Paul [:pmac] McLanahan]

Just so I'm really clear; the new product-details.m.o data is fully up to date and will be kept updated from now on right?

Comment 16

[Sylvestre Ledru [:Sylvestre]]

Yes, it is not up to date, you can kick my ass (and report bugs) ;)

Comment 17

[Paul [:pmac] McLanahan]

So Sylvestre, you're saying we should not switch to the new product-details.m.o yet?

If that's the case, I'd still like to make the URL configurable for when it's ready (thanks Pascal!), but we need to keep the old SVN URL as the default for now.

Comment 18

[Sylvestre Ledru [:Sylvestre]]

Oups, I meant "IF it is not up to date"
it is ready now. Sorry ..

Comment 19

[Paul [:pmac] McLanahan]

Ah. Great! Thanks Sylvestre and Pascal. I'll merge that PR now and we'll test in our dev and stage environments.

Comment 20

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/b77d56001b058fd47f18ece9b685afbbed1c8280
Bug 1282494: Use http://product-details.mozilla.org/ instead of the svn version

This patch override the default PROD_DETAILS_URL variable to use https://product-details.mozilla.org/1.0/ as a source for json files

https://github.com/mozilla/bedrock/commit/b9d14b3533f13f19c6684d1d492bf5b80d5a2134
Merge pull request #4254 from pascalchevrel/bug1282494_use_ship_it_for_product_details_source

Bug 1282494: Use http://product-details.mozilla.org/ instead of the s…

Comment 21

[Paul [:pmac] McLanahan]

Just an FYI: I've also updated the django-mozilla-product-details library to use the new URL as the default.

https://github.com/mozilla/django-product-details/commit/4481a30a51deff1ad39afe135c83c1021f290ad5

Comment 22

[Paul [:pmac] McLanahan]

We're seeing errors on bedrock-stage. This seems to be because there is no entry for the latest release channel version "47.0.1" for "en-US" in https://product-details.mozilla.org/1.0/firefox_primary_builds.json. There absolutely must be an entry for all channels for all active locales on bedrock.

Comment 23

[Kohei Yoshino [:kohei]]

Android locale lists are also missing. mobile_details should have builds, beta_builds and alpha_builds properties:

https://product-details.mozilla.org/1.0/mobile_details.json
https://viewvc.svn.mozilla.org/vc/libs/product-details/json/mobile_details.json?view=co

Comment 24

[Pascal Chevrel:pascalc]

(In reply to Paul [:pmac] McLanahan from comment #22)
> We're seeing errors on bedrock-stage. This seems to be because there is no
> entry for the latest release channel version "47.0.1" for "en-US" in
> https://product-details.mozilla.org/1.0/firefox_primary_builds.json. There
> absolutely must be an entry for all channels for all active locales on
> bedrock.

This bug (bug 1290789) is fixed on master but not yet deployed.

Comment 25

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/5f18f1b7fc1236c9e4d1e157d176224d68ee5f58
Bug 1282494 - Use SVN default URL for product-details for Firefox 48 release

In bug 1282494 we asked for the default source of json files for product-details to point to https://product-details.mozilla.org/1.0/ instead of https://svn.mozilla.org/libs/product-details/json/

Several bugs were reported on our data sources since them, I believe we have fixed all of them but not all changes are  deployed to the ship it app.

As much as we'd love in release management to move entirely product-details to the data produced by ship it, we think it would be safer to switch back to the svn source for the Firefox 48 release and move to https://product-details.mozilla.org/1.0/ as a data source after the release.

Sorry for the inconveniences this may cause.

https://github.com/mozilla/bedrock/commit/9cb7438375e89d7386a5266e032571ca9d0a250a
Merge pull request #4260 from pascalchevrel/backout_ship_it_url

Bug 1282494 - Use SVN default URL for product-details for Firefox 48 …

Comment 26

[Pascal Chevrel:pascalc]

Josh, would it be possible to set up a demo server with product-details pointing to https://product-details.mozilla.org/1.0/ as a test?

Thanks

Comment 27

[Josh Mize [:jgmize]]

(In reply to Pascal Chevrel:pascalc from comment #26)
> Josh, would it be possible to set up a demo server with product-details
> pointing to https://product-details.mozilla.org/1.0/ as a test?

Yes, this is now live at https://bedrock-demo-new-product-details.us-west.moz.works/en-US/

Comment 28

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/35d66391b428cbbaf1110cbbaabc03be54904479
Bug 1282494: Please use https://product-details.mozilla.org/1.0/ instead of the svn version

https://github.com/mozilla/bedrock/commit/e865dea82120e086d1c7481189448d6ce982d1bf
Merge pull request #4267 from pascalchevrel/new_product_details

Bug 1282494: Please use https://product-details.mozilla.org/1.0/ inst…

Comment 29

[Sylvestre Ledru [:Sylvestre]]

Josh, do you know if this is in production now? Thanks

Comment 30

[Josh Mize [:jgmize]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #29)
> Josh, do you know if this is in production now? Thanks

Yes, this has been in production for several days.