Last Comment Bug 751202 - Set up a stage/production environment for mozillaignite.org
: Set up a stage/production environment for mozillaignite.org
Status: RESOLVED FIXED
:
Product: Infrastructure & Operations
Classification: Other
Component: WebOps: Other (show other bugs)
: other
: x86 Mac OS X
: -- normal (vote)
: ---
Assigned To: Chris Turra [:cturra]
: Corey Shields [:cshields]
Mentors:
Depends on: 752726
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-02 09:10 PDT by Ross Bruniges [:boozeniges]
Modified: 2013-10-09 10:29 PDT (History)
5 users (show)
See Also:
Due Date:
QA Whiteboard:
Iteration: ---
Points: ---
Cab Review: ServiceNow Change Request (use flag)


Attachments

Description Ross Bruniges [:boozeniges] 2012-05-02 09:10:37 PDT
Please create a production instance for Mozilla Ignite.

Currently this is hosted on mozillalabs infrastructure at ignite-dev.mozillalabs.com and ignite-stage.mozillalabs.com

There is also currently a live wordpress blog at mozillaignite.org/blog - we will want to keep this when we launch. On stage I've got an alias in the httpd conf which seems to handle the URLs for the wordpress and the django app well.

A security review has been undertaken and it has been given the OK to be launched - https://bugzilla.mozilla.org/show_bug.cgi?id=737795

Any further questions please let me know

== Date Needed By ==
* 2012/05/21 - White House will be sending traffic to it on 2012/05/23 so good to be live a few days before then

== Language ==
* Language: Python
* Framework: Django/playdoh

== URL ==
* Production URL: mozillaignite.org - currently already something live here (though we can launch over it happily)

== HTTP / HTTPS ==
* Both HTTP and HTTPS are required this site.

== VPN ==
* The /admin section should be accessible only via SSL VPN.

== Code Repository ==
* Github Repo: https://github.com/rossbruniges/mozilla-ignite/tree/stage
* Github Vendor Repo: https://github.com/mozilla/playdoh-lib/tree/787b0bd7ad552a7c2b2a4f65ed37422d652082c8
* SVN Localization Repo: N/A

== Database ==
* This site requires a MySQL database.

== Settings ==
* cp settings_local.py-dist settings_local.py
* Update settings values accordingly.
* Set DEBUG to False in settings_local.py.

== Cron ==
Not currently required

== Tracebacks ==
* Please configure Django tracebacks to be sent to:
ross@mozillafoundation.org
Comment 1 Ross Bruniges [:boozeniges] 2012-05-02 09:13:44 PDT
Set to major as launch date is in less than a month.
Comment 2 Corey Shields [:cshields] 2012-05-02 12:19:51 PDT
(this will get put on the generic cluster in phx1 - this is a web hosting environment so its not so much a dedicated VM, changing the title of the bug to reflect that)
Comment 3 Ashish Vijayaram [:ashish] 2012-05-03 07:23:23 PDT
Lowering down to normal, 30d is a long enough timeframe.
Comment 4 David Ascher (:davida) 2012-05-03 08:05:12 PDT
Don't care about priority, but just pointing out that traffic will come in 20d, not 30d, and we need to test for several days before then.
Comment 5 Chris Turra [:cturra] 2012-05-09 11:06:45 PDT
while working on deploying the -dev services, i ran into the vendor library issue noted below. can you please have this reviewed?

# git submodule update --init
Submodule 'src/bleach' () registered for path 'src/bleach'
Submodule 'src/commonware' () registered for path 'src/commonware'
Submodule 'src/django' () registered for path 'src/django'
Submodule 'src/django-arecibo' () registered for path 'src/django-arecibo'
Submodule 'src/django-mozilla-product-details' () registered for path 'src/django-mozilla-product-details'
Submodule 'src/django-nose' () registered for path 'src/django-nose'
Submodule 'src/django-session-csrf' () registered for path 'src/django-session-csrf'
Submodule 'src/django-sha2' () registered for path 'src/django-sha2'
Submodule 'src/funfactory' () registered for path 'src/funfactory'
Submodule 'src/jingo' () registered for path 'src/jingo'
Submodule 'src/jingo-minify' () registered for path 'src/jingo-minify'
Submodule 'src/nuggets' () registered for path 'src/nuggets'
Submodule 'src/schematic' () registered for path 'src/schematic'
Submodule 'src/test-utils' () registered for path 'src/test-utils'
Submodule 'src/tower' () registered for path 'src/tower'
fatal: reference is not a tree: eca4737a3fb1f4645431aed6b326409ca2177d9f
Unable to checkout 'eca4737a3fb1f4645431aed6b326409ca2177d9f' in submodule path 'src/django'
Comment 6 David Ascher (:davida) 2012-05-09 11:09:43 PDT
Ross: this smells like 

  https://github.com/mozilla/playdoh/issues/107

Can you confirm that you've fixed the repo as per those instructions?
Comment 7 Ross Bruniges [:boozeniges] 2012-05-09 11:29:58 PDT
Yeah sorry guys, I have fixed a number of repos but not yet that one.

Will look at doing this ASAP (probably ready to go tomorrow morning as I'm about to eat dinner right now).

Thanks, 

Ross
Comment 8 Ross Bruniges [:boozeniges] 2012-05-10 07:06:41 PDT
Morning guys, 

Sorry about yesterday - updated a few things last night but then got caught up trying to debug browserID login errors (which have now been filed as a separate bug).

Update for the vendor-library - https://github.com/mozilla/playdoh-lib/tree/e22c58091674b92252f87742af16a22c92231c58

Also - have realised that memcached needs installing and configuring on the VM also.

Any problems please shout, 

Ross
Comment 9 Chris Turra [:cturra] 2012-05-10 09:12:41 PDT
i went ahead and removed the entire vendor/ directory and did another git submodule init / git submodule update --init --recursive. unfortunately i am still seeing an error around this...

$ git submodule init
Submodule 'vendor' () registered for path 'vendor'
Submodule 'vendor-local/src/django-browserid' () registered for path 'vendor-local/src/django-browserid'
Submodule 'vendor-local/src/django-push' () registered for path 'vendor-local/src/django-push'
Submodule 'vendor-local/src/django-taggit' () registered for path 'vendor-local/src/django-taggit'
Submodule 'vendor-local/src/django-waffle' () registered for path 'vendor-local/src/django-waffle'

$ sudo git submodule update --init --recursive
Submodule 'vendor' () registered for path 'vendor'
Submodule 'vendor-local/src/django-browserid' () registered for path 'vendor-local/src/django-browserid'
Submodule 'vendor-local/src/django-push' () registered for path 'vendor-local/src/django-push'
Submodule 'vendor-local/src/django-taggit' () registered for path 'vendor-local/src/django-taggit'
Submodule 'vendor-local/src/django-waffle' () registered for path 'vendor-local/src/django-waffle'
Submodule path 'vendor': checked out 'bb2899620eccdfbab33126bb13f15d47774bb7fe'
Submodule 'src/bleach' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/bleach'
Submodule 'src/commonware' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/commonware'
Submodule 'src/django' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django'
Submodule 'src/django-arecibo' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-arecibo'
Submodule 'src/django-mozilla-product-details' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-mozilla-product-details'
Submodule 'src/django-nose' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-nose'
Submodule 'src/django-session-csrf' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-session-csrf'
Submodule 'src/django-sha2' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-sha2'
Submodule 'src/funfactory' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/funfactory'
Submodule 'src/jingo' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/jingo'
Submodule 'src/jingo-minify' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/jingo-minify'
Submodule 'src/nuggets' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/nuggets'
Submodule 'src/schematic' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/schematic'
Submodule 'src/test-utils' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/test-utils'
Submodule 'src/tower' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/tower'
Submodule path 'src/bleach': checked out '3657f7e32a114fcee91ed795414e4debbc5a5925'
Submodule path 'src/commonware': checked out '6df3a122417ca13764b43066ccfa122ad9299995'
fatal: reference is not a tree: eca4737a3fb1f4645431aed6b326409ca2177d9f
Submodule path 'src/django-arecibo': checked out '73c21591b55c6b2b60c4ae0ae43a26b4f4d24f72'
Submodule path 'src/django-mozilla-product-details': checked out '7154e602399412a2e63d1f96cd2bccbfd804b4bc'
Submodule path 'src/django-nose': checked out 'b16178c08728243191a922212a16d3960163feff'
Submodule path 'src/django-session-csrf': checked out '2b32149525319dff19e08a16ad1a168034e6b3dc'
Submodule path 'src/django-sha2': checked out '5cb2f080c8289d2c278fcfea59f49db36db6f4ab'
Submodule path 'src/funfactory': checked out 'f8447c468865eceb3466e4d7c3f478bfd9592a5e'
Submodule path 'src/jingo': checked out '14253ef0e8c8d0899e9f9991dd954409c3959c9e'
Submodule path 'src/jingo-minify': checked out '95f24e9ecdbeb5d1d88029896b3d53ae8828e533'
Submodule path 'src/nuggets': checked out 'ce506882b6943ea45ea4f98290fc4ef23e09a083'
Submodule path 'src/schematic': checked out 'e749978cc06bda8ba655faf8d672ae270e25b5f5'
Submodule path 'src/test-utils': checked out 'ce0e9643ea3b38373823e04d8c2e5f2dc2de5665'
Submodule path 'src/tower': checked out '79b2e9d0c9f2c1a6ee8ae0453d8ca0bf3474b308'
Unable to checkout 'eca4737a3fb1f4645431aed6b326409ca2177d9f' in submodule path 'src/django'
Failed to recurse into submodule path 'vendor'
Comment 10 Ross Bruniges [:boozeniges] 2012-05-10 09:33:45 PDT
I've made the stage branch the default - this is where the code we're deploying lives, not sure if might have been pulling from that?

Also - I think that due to all the magic that the submodules does (I'm no expect) it might be easier re-cloning from scratch?
Comment 11 Chris Turra [:cturra] 2012-05-10 10:08:37 PDT
okay. that might do it -- i was pulling dev from orgin/dev. to confirm, do you want me using orgin/stage for both dev and stage environments?
Comment 12 Ross Bruniges [:boozeniges] 2012-05-10 10:11:06 PDT
On the new VMs yes please - at least for now.
Comment 13 Chris Turra [:cturra] 2012-05-10 10:43:28 PDT
looks like we're missing a module. the following is from the apache error log...

[Thu May 10 10:41:49 2012] [info] mod_wsgi (pid=27615): Create interpreter 'mozillaignite-dev.allizom.org|'.
[Thu May 10 10:41:49 2012] [info] [client 10.8.33.241] mod_wsgi (pid=27615, process='mozillaignite-dev', application='mozillaignite-dev.allizom.org|'): Loading WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] mod_wsgi (pid=27615): Exception occurred processing WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] Traceback (most recent call last):
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]   File "/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/lib/python/django/core/handlers/wsgi.py", line 250, in __call__
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]     self.load_middleware()
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]   File "/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/lib/python/django/core/handlers/base.py", line 47, in load_middleware
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]     raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] ImproperlyConfigured: Error importing middleware commons.middleware: "No module named requests"
[Thu May 10 10:41:53 2012] [debug] mod_headers.c(768): headers: ap_headers_error_filter()
Comment 14 Ross Bruniges [:boozeniges] 2012-05-10 11:25:38 PDT
I'm not sure about this one - but the errors look to be directly coming out of django opposed to any code that we've written.
Comment 15 David Ascher (:davida) 2012-05-10 11:28:49 PDT
I'm puzzled as to why there's a file:

[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] mod_wsgi (pid=27615): Exception occurred processing WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.

betafarm has nothing to do w/ this.
Comment 16 Chris Turra [:cturra] 2012-05-10 11:32:42 PDT
:davida -- it's the wsgi script alias. from the apache config...

 WSGIScriptAlias / /data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi

 WSGIDaemonProcess mozillaignite-dev processes=6 threads=1 maximum-requests=500 display-name=mozillaignite-dev
 WSGIProcessGroup mozillaignite-dev


it appears to be the only wsgi file in the project...

 $ find mozillaignite -iname \*.wsgi
 mozillaignite/wsgi/betafarm.wsgi


is there something else i should be using instead?
Comment 17 Brandon Burton [:solarce] 2012-05-10 11:48:37 PDT
(In reply to Chris Turra [:cturra] from comment #16)
> :davida -- it's the wsgi script alias. from the apache config...
> 
>  WSGIScriptAlias /
> /data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi
> 
>  WSGIDaemonProcess mozillaignite-dev processes=6 threads=1
> maximum-requests=500 display-name=mozillaignite-dev
>  WSGIProcessGroup mozillaignite-dev
> 
> 
> it appears to be the only wsgi file in the project...
> 
>  $ find mozillaignite -iname \*.wsgi
>  mozillaignite/wsgi/betafarm.wsgi
> 
> 
> is there something else i should be using instead?

That appears to come from https://github.com/rossbruniges/mozilla-ignite/commit/258b16bc07b68d0d9016dedbed06ec61bde4a9d0#wsgi
Comment 18 Brandon Burton [:solarce] 2012-05-10 11:49:26 PDT
(In reply to Ross Bruniges from comment #12)
> On the new VMs yes please - at least for now.

Just a nitpick, but the IT managed servers are physical servers, so referring to servers instead of VMs, in this context, would avoid any confusion
Comment 19 Brandon Burton [:solarce] 2012-05-10 11:50:22 PDT
(In reply to Ross Bruniges from comment #14)
> I'm not sure about this one - but the errors look to be directly coming out
> of django opposed to any code that we've written.

Since it's coming from the Django that's submoduled, it sounds like something you may need to ask Kumar about.

Since python-requests is a pure python package, it should be pulled in as part of your vendor submodule(s)
Comment 20 Ross Bruniges [:boozeniges] 2012-05-11 00:59:42 PDT
What version of python has been installed on the server?

We've been using 2.6 for local dev and mozilla labs VMs
Comment 21 Brandon Burton [:solarce] 2012-05-11 01:05:31 PDT
Yes, it's Python 2.6 on the clusters of web servers this site's environments are hosted on.

python --version
Python 2.6.6

The same servers host a number of other Playdoh apps, including Firefoxflicks, MozillaLabs/Betafarm, mozillians.org, moztrap.mozilla.org, etc
Comment 22 Ross Bruniges [:boozeniges] 2012-05-11 01:10:48 PDT
Cool - I thought that would be the case. Just trying to get to the bottom of that module missing...
Comment 23 Brandon Burton [:solarce] 2012-05-11 01:23:58 PDT
From running [bburton@genericadm.private.phx1 mozillaignite]$ grep -R requests *

and the output below

vendor-local/src/django-browserid/requirements.txt:requests==0.10.8
Binary file vendor-local/src/django-browserid/django_browserid/base.pyc matches
vendor-local/src/django-browserid/django_browserid/base.py:import requests
vendor-local/src/django-browserid/django_browserid/base.py:    r = requests.post(url, **parameters)
vendor-local/src/django-browserid/setup.py:    install_requires='requests>=0.9.1',
vendor-local/src/django-browserid/docs/details/settings.rst:    included with requests_ is used.
vendor-local/src/django-browserid/docs/details/settings.rst:.. _requests: http://docs.python-requests.org/

I would suspect your usage of django-browserid is what's wanting it, as it is listed in https://github.com/mozilla/django-browserid/blob/master/requirements.txt , but it's not included in vendor or vendor-local, so it's not found
Comment 24 Ross Bruniges [:boozeniges] 2012-05-11 08:19:44 PDT
Pushed a fix to hopefully sort this out - I think I had rather a big fail in regard to my knowledge of how the submodules were working.

Hopefully we've got what we need in there now.
Comment 25 Brandon Burton [:solarce] 2012-05-11 09:18:28 PDT
No problem. You already know more than I do.

So I redid the clone of your repo and the submodule bits and we're passed the requests dependency.

I've submitted https://github.com/rossbruniges/mozilla-ignite/pull/52/files with an additional setup step

https://mozillaignite-dev.allizom.org/en-US/ now loads, but I'm seeing some 404s on CSS because it seems like the 'media' paths in the rendered HTML don't match the layout on the filesystem, it's expecting /media/ignite/css , etc, but everything is /media/css , etc
Comment 26 Ross Bruniges [:boozeniges] 2012-05-11 09:58:14 PDT
In settings_local.py change:

if os.environ.get('DJANGO_SITE') == 'ignite':
    from settings_ignite import *
else:
    from settings import *

to

from settings_ignite import *

Sorry - this is an oversite from when, at one stage we wanted to make sure what this codebase and the betafarm codebase could be easily merged.
Comment 27 Chris Turra [:cturra] 2012-05-11 10:09:49 PDT
lookin' better! 

  https://mozillaignite-dev.allizom.org/
Comment 28 Chris Turra [:cturra] 2012-05-11 12:47:01 PDT
Ross -- i just finished setting up stage, but the style sheets don't appear to be loading with either the default or modified settings_local.py. thoughts?

  https://mozillaignite.allizom.org
Comment 29 Ross Bruniges [:boozeniges] 2012-05-14 02:52:37 PDT
Hmm - did you run ./manage.py compress_assets after installing everything? 

What happens when you set debug to True and have the non-compressed assets load through django?
Comment 30 Chris Turra [:cturra] 2012-05-14 08:45:56 PDT
Ross -- looks like you were right and i had a case of the "friday's." after re-deploying this morning with compress_assets everything looks as expected.
Comment 31 Ross Bruniges [:boozeniges] 2012-05-14 08:46:37 PDT
Haha - I know that Friday feeling :)
Comment 32 Ross Bruniges [:boozeniges] 2012-05-15 01:54:00 PDT
I started trying to do some initial smoke tests of the site and am currently unable to login with browserID, the csrf stuff is failing and throwing 403's

Has memcached been installed and correctly configured in the settings_local.py ?
Comment 33 Chris Turra [:cturra] 2012-05-16 10:46:11 PDT
Ross - memcache was not configured on the dev instance because i based the configuration on the settings_local.py-dist that was pulled down from github. 

i just added the memcached backend to dev and tested login with browserid succesfully. give it a whirl and let me know how you make it -- i will then apply this same change to stage and prod environments.

  https://mozillaignite-dev.allizom.org
Comment 34 Ross Bruniges [:boozeniges] 2012-05-16 10:51:07 PDT
Ah yes sorry - I've been meaning to try and sort out that py.dist file for a while - will try hard to get that done to ease future deploys.

I've managed to log in but only on the second attempt, first one 403'ed on me as before.

Probably good to have other people to test out to give us a better understanding on if it was just something that cleared up when I visited the site with an empty cache...

Will try and run some more smoke tests tonight/tomorrow morning.

Thanks very much!
Comment 35 Ross Bruniges [:boozeniges] 2012-05-25 09:00:08 PDT
Heya guys,

After last weeks crazy schedule I've had a chance to look at the dev site again to run some initial smoke tests through the system, I've also put a bit of work into making the deploy tasks a bit more obvious in the settings-local.py-dist.

So from what I can see we need a few directories created inside of the media_ignite/ directory (refer to settings.py or settings-local.py-dist for names) so that the admin and create views can upload images to them. We also need to ensure that the app has write access to them.

Also, wondering if you've got any plans for the wordpress blog? There is one currently installed at mozillaignite.org/blog/ - would we just want to keep it there and redirect or install it alongside the app on our new server?

Any problems please let me know, 

Ross
Comment 36 Chris Turra [:cturra] 2012-05-25 10:44:31 PDT
(In reply to Ross Bruniges from comment #35)
> So from what I can see we need a few directories created inside of the
> media_ignite/ directory (refer to settings.py or settings-local.py-dist for
> names) so that the admin and create views can upload images to them. We also
> need to ensure that the app has write access to them.

upload directory created on netapp share and symlinked to application. 

 $ ls mozillaignite/media/img/upload/
 avatars  challenges  events  projects  topics


> Also, wondering if you've got any plans for the wordpress blog? There is one
> currently installed at mozillaignite.org/blog/ - would we just want to keep
> it there and redirect or install it alongside the app on our new server?

we can give you a blog at blog.mozilla.org and should be able to import your current data into it.
Comment 37 Ross Bruniges [:boozeniges] 2012-05-29 08:30:57 PDT
Thanks Chris, still not working but hopefully we're in there.

Can you have a look in the error log to see what's happening - each time I try to upload with an image, which gives me the impression that something weird is happening, or something un-weird like the the directory reference being wrong. Do the upload paths in local.py need to be changed for this to work? Not sure what is done on other projects.

As right now I'm looking at the dev server (https://mozillaignite-dev.allizom.org/) might it be possible to set debug-True to give us hopefully some better error messages?

We're using the admin section for a few tasks - and I'm not able to log into that yet.
Could you set up ross@mozillafoundation.org as a super-user? 
Also I think a reference to the admin assets needs to be changed in the apache-config - the need for this changed when we moved django out of a git submodule and into lib.

Sorry - rather a few more points here than I was hoping, need any more info please let me know, 

Ross
Comment 38 Ross Bruniges [:boozeniges] 2012-05-29 08:32:29 PDT
> 
> we can give you a blog at blog.mozilla.org and should be able to import your
> current data into it.
Would we be able to apply a custom style to this? 
Would we also be able to send requests from mozillaignite.org/blog onto blog.mozilla.org/mozillaignite/ (assuming that's the format of URLs that we'll be getting)
Comment 39 Chris Turra [:cturra] 2012-05-29 10:15:50 PDT
(In reply to Ross Bruniges from comment #37)
> Can you have a look in the error log to see what's happening - each time I
> try to upload with an image, which gives me the impression that something
> weird is happening, or something un-weird like the the directory reference
> being wrong. Do the upload paths in local.py need to be changed for this to
> work? Not sure what is done on other projects.

i don't see an error that points to image uploads, but do see the following in the apache error logs...

[Tue May 29 08:26:38 2012] [error] 08:26:1338305198 root:DEBUG Note: funfactory monkey patches executed in /data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/funfactory/funfactory/monkeypatches.py :/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/funfactory/funfactory/monkeypatches.py:34
[Tue May 29 08:26:39 2012] [debug] mod_headers.c(743): headers: ap_headers_output_filter()
[Tue May 29 08:26:39 2012] [error] [client 10.8.33.241] File does not exist: /data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/django, referer: https://mozillaignite-dev.allizom.org/admin/


> As right now I'm looking at the dev server
> (https://mozillaignite-dev.allizom.org/) might it be possible to set
> debug-True to give us hopefully some better error messages?

looks like i missed uncommenting the debug option in the settings file. i have updated that now. sorry! 


> We're using the admin section for a few tasks - and I'm not able to log into
> that yet.
> Could you set up ross@mozillafoundation.org as a super-user? 
> Also I think a reference to the admin assets needs to be changed in the
> apache-config - the need for this changed when we moved django out of a git
> submodule and into lib.

i set the is_superuser flag on your account to true.

mysql> select * from auth_user where email='ross@mozillafoundation.org'\G
*************************** 1. row ***************************
          id: 2
    username: O_ojXJ5PhzED85OfmOcJ2LUgs3I
  first_name: 
   last_name: 
       email: ross@mozillafoundation.org
    password: !
    is_staff: 0
   is_active: 1
is_superuser: 1
  last_login: 2012-05-16 10:47:52
 date_joined: 2012-05-16 10:47:52
1 row in set (0.00 sec)
Comment 40 Ross Bruniges [:boozeniges] 2012-05-29 10:47:42 PDT
Thanks for turning DEBUG on, has provided us with the following traceback, looks like the link between where we're looking to upload the images and where the app things that they are vary a little. 

http://pastebin.com/pWQee8rG

Not sure what's going on with the django-admin (it's still not logging me in) - I thought it should be the case that when I'm logged in with browserID hitting /admin/ should drop me in, but it's not for some reason. Maybe linked to the assets not correctly loading, as mentioned in my previous email...
Comment 41 Ross Bruniges [:boozeniges] 2012-05-29 10:48:33 PDT
Actually looking at it I think I also need to be in the staff group, as well as super user!

Thanks Chris, 

Ross
Comment 42 Chris Turra [:cturra] 2012-05-29 10:55:27 PDT
(In reply to Ross Bruniges from comment #40)
> Thanks for turning DEBUG on, has provided us with the following traceback,
> looks like the link between where we're looking to upload the images and
> where the app things that they are vary a little. 
> 
> http://pastebin.com/pWQee8rG

i know what's going on here! i put the upload directory in media/img/ not media/ignite/img/

that should be fixed now if you want to give it a test.
Comment 43 Chris Turra [:cturra] 2012-05-29 10:57:20 PDT
(In reply to Ross Bruniges from comment #41)
> Actually looking at it I think I also need to be in the staff group, as well
> as super user!

mysql> update auth_user set is_staff=1 where email='ross@mozillafoundation.org';
Query OK, 1 row affected (0.07 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select * from auth_user where email='ross@mozillafoundation.org'\G
*************************** 1. row ***************************
          id: 2
    username: O_ojXJ5PhzED85OfmOcJ2LUgs3I
  first_name: 
   last_name: 
       email: ross@mozillafoundation.org
    password: !
    is_staff: 1
   is_active: 1
is_superuser: 1
  last_login: 2012-05-16 10:47:52
 date_joined: 2012-05-16 10:47:52
1 row in set (0.00 sec)
Comment 44 Ross Bruniges [:boozeniges] 2012-05-29 10:59:49 PDT
(In reply to Chris Turra [:cturra] from comment #42)
> (In reply to Ross Bruniges from comment #40)
> > Thanks for turning DEBUG on, has provided us with the following traceback,
> > looks like the link between where we're looking to upload the images and
> > where the app things that they are vary a little. 
> > 
> > http://pastebin.com/pWQee8rG
> 
> i know what's going on here! i put the upload directory in media/img/ not
> media/ignite/img/
> 
> that should be fixed now if you want to give it a test.

Not sure if I was too quick to test but just got the exact same error...
Comment 45 Chris Turra [:cturra] 2012-05-29 11:07:12 PDT
(In reply to Ross Bruniges from comment #44)
> Not sure if I was too quick to test but just got the exact same error...

you might have been a tad quick. the environment update has now finished if you want to test again.
Comment 46 Ross Bruniges [:boozeniges] 2012-05-29 11:17:48 PDT
(In reply to Chris Turra [:cturra] from comment #45)
> (In reply to Ross Bruniges from comment #44)
> > Not sure if I was too quick to test but just got the exact same error...
> 
> you might have been a tad quick. the environment update has now finished if
> you want to test again.

Still the same error :(
Comment 47 Chris Turra [:cturra] 2012-05-29 11:30:10 PDT
(In reply to Ross Bruniges from comment #46)
> (In reply to Chris Turra [:cturra] from comment #45)
> > (In reply to Ross Bruniges from comment #44)
> > > Not sure if I was too quick to test but just got the exact same error...
> > 
> > you might have been a tad quick. the environment update has now finished if
> > you want to test again.
> 
> Still the same error :(

i figured it out! i had named the directory 'upload' not 'uploads'. should be fixed now. 

feel free to hit me up on irc (cturra) if you want to chat about this more.
Comment 48 Ross Bruniges [:boozeniges] 2012-05-29 11:41:24 PDT
(In reply to Chris Turra [:cturra] from comment #47)
> (In reply to Ross Bruniges from comment #46)
> > (In reply to Chris Turra [:cturra] from comment #45)
> > > (In reply to Ross Bruniges from comment #44)
> > > > Not sure if I was too quick to test but just got the exact same error...
> > > 
> > > you might have been a tad quick. the environment update has now finished if
> > > you want to test again.
> > 
> > Still the same error :(
> 
> i figured it out! i had named the directory 'upload' not 'uploads'. should
> be fixed now. 
> 
> feel free to hit me up on irc (cturra) if you want to chat about this more.

Ha - always those typos :)

So, we've not got a permissions error on that directory, I've had some trouble with django and this before. It might have to be 777 if 755 isn't working...

What IRC rooms do you hang out in? I'm soon to be going home (am on UK time here unfortunately) but what IRC do you hang out in? I'm normally in #webdev, #london, #mofodev and am Boozeniges.

Thanks for the help today, sorry to be such a pain but feels like we've made some good progress!

Cheers, 

Ross
Comment 49 Chris Turra [:cturra] 2012-05-29 11:56:53 PDT
(In reply to Ross Bruniges from comment #48)
> So, we've not got a permissions error on that directory, I've had some
> trouble with django and this before. It might have to be 777 if 755 isn't
> working...

i changed the permissions from 755 to 766 so [w]rite was included. 


> What IRC rooms do you hang out in? I'm soon to be going home (am on UK time
> here unfortunately) but what IRC do you hang out in? I'm normally in
> #webdev, #london, #mofodev and am Boozeniges.

i am many rooms, but it would probably be easiest if you just pm me directly. 

 /msg cturra
Comment 50 Ross Bruniges [:boozeniges] 2012-05-30 05:02:41 PDT
Still not working - will try and find you in an IRC room today.

Thanks, 

Ross
Comment 51 Ross Bruniges [:boozeniges] 2012-06-06 04:50:25 PDT
Morning, 

Can we check to see if memcached is still running and configured correctly? 

It looks like the browserID login is broken and throwing 403 errors across both dev and staging.
Comment 52 Ross Bruniges [:boozeniges] 2012-06-06 05:43:53 PDT
In addition am hoping that you can set up update_site_feeds to run as a cron every 4 hours, don't mind too much about the minute - let's say 10 minutes past.

Information on cron can be found here - https://github.com/rossbruniges/mozilla-ignite/commit/3737879156c79cbef1f686c7a14ca37e10bff8d6
Comment 53 Chris Turra [:cturra] 2012-06-06 09:51:27 PDT
(In reply to Ross Bruniges from comment #51)
> Can we check to see if memcached is still running and configured correctly? 
> 
> It looks like the browserID login is broken and throwing 403 errors across
> both dev and staging.

the memcache servers we were initially using were incorrect. i have now updated -dev and -stage to use the following which has resolved these http/403s

  'LOCATION': [
    'memcache-generic01:11211',
    'memcache-generic02:11211',
  ],
Comment 54 Chris Turra [:cturra] 2012-06-07 13:42:19 PDT
closing bug because both -dev and -stage environments are up and running. production environment setup is covered by bug 762525

Note You need to log in before you can comment on or make changes to this bug.