1247447 - product details cannot update

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Setup / Install, defect)

Description

[Rob Hudson [:robhudson]]

What did you do?
================
1. Built a docker set up for MDN
2. Tried to launch it
3. It failed

What happened?
==============
Traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app
    __import__(module)
  File "/app/kuma/wsgi.py", line 7, in <module>
    from whitenoise.django import DjangoWhiteNoise  # noqa
  File "/usr/local/lib/python2.7/site-packages/whitenoise/django.py", line 16, in <module>
    from django.contrib.staticfiles.storage import staticfiles_storage
  File "/usr/local/lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 12, in <module>
    from django.core.cache import (
  File "/usr/local/lib/python2.7/site-packages/django/core/cache/__init__.py", line 34, in <module>
    if DEFAULT_CACHE_ALIAS not in settings.CACHES:
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/kuma/settings/docker.py", line 1, in <module>
    from .common import *  # noqa
  File "/app/kuma/settings/common.py", line 250, in <module>
    LOCALES = get_locales()
  File "/app/kuma/settings/common.py", line 243, in get_locales
    json_locales = json.load(open(file, 'r'))
IOError: [Errno 2] No such file or directory: '/app/../product_details_json/languages.json'


When I try to run the `update_product_details` management command that fails too:

# ./manage.py update_product_details
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/kuma/settings/docker.py", line 1, in <module>
    from .common import *  # noqa
  File "/app/kuma/settings/common.py", line 250, in <module>
    LOCALES = get_locales()
  File "/app/kuma/settings/common.py", line 243, in get_locales
    json_locales = json.load(open(file, 'r'))
IOError: [Errno 2] No such file or directory: '/app/../product_details_json/languages.json'


What should have happened?
==========================
No traceback.

Is there anything else we should know?
======================================

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

It's a bootstrapping problem where if you don't have product-detail files you can't get product-detail files. Thus it'll affect new dev environments--both docker and vagrant.

Grabbing this to fix.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

In a PR: https://github.com/mozilla/kuma/pull/3779

This changes some code around so that if the product-details data isn't there, it prints a warning and rolls its eyes, but then sets LOCALES to {} and LANGUAGES to []. That allows you to run "./manage.py update_product_details" and move on with your life. This scenario should only ever happen when creating a dev environment.

Another way to solve this is to convert LOCALES and LANGUAGES into lazy structures, but that creates other problems. For example, kuma/dashboards/forms.py uses settings.LANGUAGES at module-load time. We'd probably have to lazy-ify that, too. Then suddenly we're lazy-ifying a bunch of things and then it affects me and suddenly I'm too lazy to do that work.

Comment 3

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/f2ec4b2e1aec6c13680cfc7b8d95283057b5576e
Fix bug 1247447: Fix bootstrapping problem with product-details

If you just built a dev environment and it was pristine and didn't
have any of the product-details files, then you were in a bit of a
pickle because in order to get product-details files, it'd run through
the settings code which required the product-details files which you
didn't have because that's the very thing you were trying to get. Ugh.

This fixes that by moving some code around such that LOCALES and
LANGUAGES get built together (LANGUAGES requires LOCALES anyhow) and
if the product-details files aren't there, it prints a warning and
rolls its eyes, but then returns empty structures. This is ok because
it only happens if there are no product-details files and that should
only be the case if you haven't gotten the product-details, yet.

https://github.com/mozilla/kuma/commit/2509bb586d3c25a0296b654f206bd4f493799c5d
Merge pull request #3779 from mozilla/1247447-get_locales

Fix bug 1247447: Fix bootstrapping problem with product-details