1088136 - Migrate Badg.us media to own server

Status: RESOLVED WONTFIX

(Participation Infrastructure :: MCWS, task)

Description

[Majken Connor [:Kensie]]

Need to move media on badg.us to its own server to make it more robust.

Comment 1

[James Boston (:jboston) <[e-mail]>]

As per meeting discussion, there's a cron job (in cron.daily) to create a snapshot of the volume on s3, but I think a different solution is needed if the intended purpose is hosting media.

Comment 2

[James Boston (:jboston) <[e-mail]>]

Perhaps a summary of what I think is happening is in order.

Problem:
The production server is too slow. 

Cause:
It's a a python server and it's serving media files.

Solution:
1. test nginx/gunicorn/django as alternative on a dev server
2. deploy that to production
3. code and test solution to serve media from s3
4. deploy that

Step 1 is what I'm working on now.

Comment 3

[James Boston (:jboston) <[e-mail]>]

badg.us isn't off the shelf django, but a derivative, playdoh. It's not working with gunicorn as it's currently setup. The wsgi file in the venv directory isn't being loaded by gunicorn.

Comment 4

[James Boston (:jboston) <[e-mail]>]

I'm using the Badgus and Playdoh documentation here:
https://wiki.mozilla.org/IT/Community/WG/Badgus
http://playdoh.readthedocs.org/en/latest/operations.html

Following the instructions, I attempt to start this server as follows:
> virtualenv ./dj
> source ./dj/bin/activate
> ln -s wsgi/playdoh.wsgi wsgi/playdoh.py
> touch wsgi/__init__.py
> gunicorn wsgi/playdoh:application

This is the error output:

> Traceback (most recent call last):
>   File "/home/ubuntu/badgus/dj/bin/gunicorn", line 11, in <module>
>     sys.exit(run())
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
>     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 185, in run
>     super(Application, self).run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 71, in run
>     Arbiter(self).run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 169, in run
>     self.manage_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 477, in manage_workers
>     self.spawn_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 542, in spawn_workers
>     time.sleep(0.1 * random.random())
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 209, in handle_chld
>     self.reap_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
>     raise HaltServer(reason, self.WORKER_BOOT_ERROR)
> gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Comment 5

[James Boston (:jboston) <[e-mail]>]

Calling 'python manage.py runserver' in the virtual env gives some useful output on missing packages. 

Fixed missing packages:
pip install python-memcached
pip install MySQL-python
pip install pillow
pip install Jinja2
pip install six

Still unable to start gunicorn by method in comment 4.

Adding gunicorn to list of installed apps in badgus/settings/base.py allows starting gunicorn by a deprecated method:
python manage.py run_gunicorn

This seems to work fine. I must be using the wrong app or module name when starting by the other method. I'm not sure. I'll loop back re-examine that after configuring nginx (which is still todo).

Comment 6

[Logan Rosen [:Logan]]

Given that we're sunsetting badg.us, I think it's safe to close this.

Comment 7

[Firefox Bug Husbandry Bot]

Bulk move of bugs