679969 - Update product details on bounceradmin

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Fred Wenzel [:wenzel]]

Today a language was added to product detail, but it's not showing up on tuxedo (bounceradmin.m.o). Perhaps that app doesn't auto-update its product details data?

Please:
- check the cron jobs on that box to see if it runs "manage.py update_product_details", and at what frequency.
- if it doesn't run it automatically, please run "python26 manage.py update_product_details" in the tuxedo directory to pull the new JSON files from the server.

Thanks!

Comment 1

[Shyam Mani [:fox2mike]]

I'll poke at this.

Comment 2

[Shyam Mani [:fox2mike]]

It does have a cron :

[root@im-sentry01 ~]# cat /etc/cron.d/tuxedo 
00 */4 * * * root cd /var/www/django/tuxedo;  /data/virtualenvs/tuxedo/bin/python26 manage.py update_product_details 

If I'm reading that right, it runs this every 6 hours. 

Running that by hand :

[root@im-sentry01 ~]# cd /var/www/django/tuxedo
[root@im-sentry01 tuxedo]# /data/virtualenvs/tuxedo/bin/python26 manage.py update_product_details
[root@im-sentry01 tuxedo]#

And there are no changes to the files (last updates seem to be from  a few days ago...)

[root@im-sentry01 tuxedo]# ls -l /var/www/django/tuxedo-json
total 120
-rw-r--r-- 1 root root  3947 Aug 16 12:00 firefox_beta_builds.json
-rw-r--r-- 1 root root  1392 Aug  5 12:00 firefox_history_development_releases.json
-rw-r--r-- 1 root root   172 Aug 16 12:00 firefox_history_major_releases.json
-rw-r--r-- 1 root root  2124 Jul 12 00:00 firefox_history_stability_releases.json
-rw-r--r-- 1 root root 23810 Aug 16 12:00 firefox_primary_builds.json
-rw-r--r-- 1 root root   184 Aug 16 12:00 firefox_versions.json
-rw-r--r-- 1 root root  7889 Aug 17 12:00 languages.json
-rw-r--r-- 1 root root 22875 Aug 16 12:00 mobile_details.json
-rw-r--r-- 1 root root   339 Aug  5 20:00 mobile_history_development_releases.json
-rw-r--r-- 1 root root    96 Aug 16 12:00 mobile_history_major_releases.json
-rw-r--r-- 1 root root    43 Apr 28 20:00 mobile_history_stability_releases.json
-rw-r--r-- 1 root root     2 Jan 13  2011 thunderbird_beta_builds.json
-rw-r--r-- 1 root root   597 Aug 16 04:00 thunderbird_history_development_releases.json
-rw-r--r-- 1 root root   134 Aug 16 04:00 thunderbird_history_major_releases.json
-rw-r--r-- 1 root root  1136 Aug 16 04:00 thunderbird_history_stability_releases.json
-rw-r--r-- 1 root root  9019 Aug 16 04:00 thunderbird_primary_builds.json
-rw-r--r-- 1 root root    81 Aug 16 04:00 thunderbird_versions.json
[root@im-sentry01 tuxedo]# date
Thu Aug 18 01:24:35 PDT 2011

Comment 3

[Shyam Mani [:fox2mike]]

(In reply to Shyam Mani [:fox2mike] from comment #2)

> If I'm reading that right, it runs this every 6 hours. 

Err 6 times a day, every 4 hours. And it has been running according to the cron logs.

Comment 4

[Axel Hecht [:Pike]]

So the original data is php arrays, and this is json.

Can you say where it's getting the json files from? I tried to find the tuxedo sources earlier, and couldn't. Maybe the system that generates the json out of the php is not updated?

Comment 5

[Shyam Mani [:fox2mike]]

(In reply to Axel Hecht [:Pike] from comment #4)

> Can you say where it's getting the json files from? I tried to find the
> tuxedo sources earlier, and couldn't. Maybe the system that generates the
> json out of the php is not updated?


That cron generates the json AFAIK.

[remote "origin"]
        url = git://github.com/fwenzel/tuxedo.git

Comment 6

[Axel Hecht [:Pike]]

Didn't realize that the json is actually in svn, too, and http://svn.mozilla.org/libs/product-details/json/languages.json seems to match, data-wise. And fox2mike found "sw" in there, too.

Pascal clarified that we're only adding sw and mn to http://svn.mozilla.org/libs/product-details/json/firefox_primary_builds.json once we actually publish the builds.

Not sure what releng really needs :-?

Comment 7

[Rail Aliiev [:rail]]

(In reply to Axel Hecht [:Pike] from comment #6)
> Not sure what releng really needs :-?

We just need sw locale listed in bouncer admin. It's not there ATM.

Comment 8

[Shyam Mani [:fox2mike]]

fwenzel, 

Need a hand with this when you're up.

Comment 9

[Axel Hecht [:Pike]]

So, the exact url we need this for is https://bounceradmin.mozilla.com/api/docs/product_add, which is served by https://github.com/fwenzel/tuxedo/blob/master/apps/api/views.py#L45, which asks product_details, which only loads the json on __init__, https://github.com/fwenzel/django-mozilla-product-details/blob/master/product_details/__init__.py#L44. run once on import, and doesn't refresh until the python process of the webapp is restarted, AFAICT.

Same for https://bounceradmin.mozilla.com/admin/mirror/product/add/, that's getting its choices from

LANG_CHOICES = [(key, "%s: %s" % (key, value['English']))
                for key, value in product_details.languages.items()]

in mirror/models.py.

Comment 10

[Shyam Mani [:fox2mike]]

Good call, Pike. That fixed the issue, I can now see sw in the drop-down.

Wenzel, do you want to make code changes so this is refreshed more often? Or any other ideas to make sure this doesn't happen again?

Comment 11

[Fred Wenzel [:wenzel]]

(In reply to Shyam Mani [:fox2mike] from comment #10)
> Wenzel, do you want to make code changes so this is refreshed more often? Or
> any other ideas to make sure this doesn't happen again?

Sorry about that. I did not remember this was an issue on low-traffic websites when I filed this bug yesterday :-/

It's this issue: <https://github.com/fwenzel/django-mozilla-product-details/issues/1>