Migrate Badg.us media to own server

RESOLVED WONTFIX

Status

Infrastructure & Operations
Community IT: Infrastructure
RESOLVED WONTFIX
3 years ago
3 years ago

People

(Reporter: Kensie, Assigned: jboston)

Tracking

Details

(Whiteboard: Need help w/ gunicorn/playdoh. Server starts w/ deprecated method only., URL)

(Reporter)

Description

3 years ago
Need to move media on badg.us to its own server to make it more robust.
(Reporter)

Updated

3 years ago
Assignee: nobody → yousef
(Reporter)

Updated

3 years ago
Whiteboard: [Takeme]

Updated

3 years ago
Assignee: yousef → jamesboston
Whiteboard: [Takeme]
As per meeting discussion, there's a cron job (in cron.daily) to create a snapshot of the volume on s3, but I think a different solution is needed if the intended purpose is hosting media.

Updated

3 years ago
Depends on: 1102361
Perhaps a summary of what I think is happening is in order.

Problem:
The production server is too slow. 

Cause:
It's a a python server and it's serving media files.

Solution:
1. test nginx/gunicorn/django as alternative on a dev server
2. deploy that to production
3. code and test solution to serve media from s3
4. deploy that

Step 1 is what I'm working on now.
badg.us isn't off the shelf django, but a derivative, playdoh. It's not working with gunicorn as it's currently setup. The wsgi file in the venv directory isn't being loaded by gunicorn.
I'm using the Badgus and Playdoh documentation here:
https://wiki.mozilla.org/IT/Community/WG/Badgus
http://playdoh.readthedocs.org/en/latest/operations.html

Following the instructions, I attempt to start this server as follows:
> virtualenv ./dj
> source ./dj/bin/activate
> ln -s wsgi/playdoh.wsgi wsgi/playdoh.py
> touch wsgi/__init__.py
> gunicorn wsgi/playdoh:application

This is the error output:

> Traceback (most recent call last):
>   File "/home/ubuntu/badgus/dj/bin/gunicorn", line 11, in <module>
>     sys.exit(run())
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
>     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 185, in run
>     super(Application, self).run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 71, in run
>     Arbiter(self).run()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 169, in run
>     self.manage_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 477, in manage_workers
>     self.spawn_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 542, in spawn_workers
>     time.sleep(0.1 * random.random())
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 209, in handle_chld
>     self.reap_workers()
>   File "/home/ubuntu/badgus/dj/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
>     raise HaltServer(reason, self.WORKER_BOOT_ERROR)
> gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
Calling 'python manage.py runserver' in the virtual env gives some useful output on missing packages. 

Fixed missing packages:
pip install python-memcached
pip install MySQL-python
pip install pillow
pip install Jinja2
pip install six

Still unable to start gunicorn by method in comment 4.

Adding gunicorn to list of installed apps in badgus/settings/base.py allows starting gunicorn by a deprecated method:
python manage.py run_gunicorn

This seems to work fine. I must be using the wrong app or module name when starting by the other method. I'm not sure. I'll loop back re-examine that after configuring nginx (which is still todo).
Whiteboard: Need help w/ gunicorn/playdoh. Server starts w/ deprecated method only.

Updated

3 years ago
(Reporter)

Updated

3 years ago
Status: NEW → ASSIGNED
(Reporter)

Updated

3 years ago
Component: Community IT → Community IT: Infrastructure

Comment 6

3 years ago
Given that we're sunsetting badg.us, I think it's safe to close this.
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.