If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Add in celery to solitude

RESOLVED FIXED

Status

Cloud Services
Operations: Marketplace
RESOLVED FIXED
5 years ago
3 years ago

People

(Reporter: andym, Assigned: jason)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
I'd like solitude to have access to its own celery queue. This is completely different from any zamboni celery queue and should not be accessible from anything else.
(Assignee)

Updated

5 years ago
Assignee: server-ops-amo → jthomas
(Assignee)

Updated

5 years ago
Depends on: 818578
(Assignee)

Comment 1

5 years ago
Seeing the following errors when trying to start celery in -dev:

/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/metlog/path.py:52: UserWarning: Module django_sha2 was already imported from /data/www/payments-dev.allizom.org/solitude/vendor/src/django-sha2/django_sha2/__init__.py, but /data/www/payments-dev.allizom.org/venv/src/django-sha2 is being added to sys.path
  import pkg_resources
/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/metlog/path.py:52: UserWarning: Module dictconfig was already imported from /data/www/payments-dev.allizom.org/solitude/vendor/src/nuggets/dictconfig.py, but /data/www/payments-dev.allizom.org/venv/src/nuggets is being added to sys.path
  import pkg_resources
/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/metlog/path.py:52: UserWarning: Module commonware was already imported from /data/www/payments-dev.allizom.org/solitude/vendor/src/commonware/commonware/__init__.py, but /data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages is being added to sys.path
  import pkg_resources
/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/metlog/path.py:52: UserWarning: Module django was already imported from /data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/__init__.py, but /data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages is being added to sys.path
  import pkg_resources
/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/metlog/path.py:52: UserWarning: Module cef was already imported from /data/www/payments-dev.allizom.org/solitude/vendor/lib/python/cef.py, but /data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages is being added to sys.path
  import pkg_resources
/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)
django_paranoia.reporters.log
django_paranoia.reporters.cef_
Traceback (most recent call last):
  File "/data/www/payments-dev.allizom.org/solitude/manage.py", line 27, in <module>
    manage.main()
  File "/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/funfactory/manage.py", line 143, in main
    execute_manager(current_settings)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/__init__.py", line 459, in execute_manager
    utility.execute()
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/base.py", line 231, in execute
    self.validate()
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/base.py", line 266, in validate
    num_errors = get_validation_errors(s, app)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/core/management/validation.py", line 30, in get_validation_errors
    for (app_name, error) in get_app_errors().items():
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/db/models/loading.py", line 158, in get_app_errors
    self._populate()
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/db/models/loading.py", line 67, in _populate
    self.load_app(app_name)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/db/models/loading.py", line 88, in load_app
    models = import_module('.models', app_name)
  File "/data/www/payments-dev.allizom.org/solitude/vendor/lib/python/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/raven/contrib/django/models.py", line 209, in <module>
    register_handlers()
  File "/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/raven/contrib/django/models.py", line 195, in register_handlers
    from raven.contrib.celery import register_signal
  File "/data/www/payments-dev.allizom.org/venv/lib/python2.6/site-packages/raven/contrib/celery/__init__.py", line 13, in <module>
    from celery.signals import after_setup_logger, task_failure
ImportError: cannot import name after_setup_logger
(Assignee)

Comment 2

5 years ago
rabbitmq has been configured with a seperate vhost/queue for solitude dev. Do we need payments-proxy to have access to this?

I have added the following settings to settings/local.py

# Uncomment these to activate and customize Celery:
CELERY_ALWAYS_EAGER = False  # required to activate celeryd

BROKER_HOST = 'rabbit-dev'
BROKER_PORT = 5672
BROKER_USER = 'solitude_dev'
BROKER_PASSWORD = 'solitude_dev'
BROKER_VHOST = 'solitude_dev'

CELERY_RESULT_BACKEND = 'database'
# mysql
CELERY_RESULT_DBURI = 'mysql://<removed>'

Please merge in this pull request - https://github.com/mozilla/solitude/pull/31
(Assignee)

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Assignee)

Comment 3

5 years ago
jason | andym: one question, payments-proxy-dev.allizom.org, does this need access to celery broker?
andym | jason: no
Component: Server Operations: AMO Operations → Operations: Marketplace
Product: mozilla.org → Mozilla Services
You need to log in before you can comment on or make changes to this bug.