Closed Bug 751202 Opened 12 years ago Closed 12 years ago

Set up a stage/production environment for mozillaignite.org

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: boozeniges, Assigned: cturra)

References

Details

Please create a production instance for Mozilla Ignite.

Currently this is hosted on mozillalabs infrastructure at ignite-dev.mozillalabs.com and ignite-stage.mozillalabs.com

There is also currently a live wordpress blog at mozillaignite.org/blog - we will want to keep this when we launch. On stage I've got an alias in the httpd conf which seems to handle the URLs for the wordpress and the django app well.

A security review has been undertaken and it has been given the OK to be launched - https://bugzilla.mozilla.org/show_bug.cgi?id=737795

Any further questions please let me know

== Date Needed By ==
* 2012/05/21 - White House will be sending traffic to it on 2012/05/23 so good to be live a few days before then

== Language ==
* Language: Python
* Framework: Django/playdoh

== URL ==
* Production URL: mozillaignite.org - currently already something live here (though we can launch over it happily)

== HTTP / HTTPS ==
* Both HTTP and HTTPS are required this site.

== VPN ==
* The /admin section should be accessible only via SSL VPN.

== Code Repository ==
* Github Repo: https://github.com/rossbruniges/mozilla-ignite/tree/stage
* Github Vendor Repo: https://github.com/mozilla/playdoh-lib/tree/787b0bd7ad552a7c2b2a4f65ed37422d652082c8
* SVN Localization Repo: N/A

== Database ==
* This site requires a MySQL database.

== Settings ==
* cp settings_local.py-dist settings_local.py
* Update settings values accordingly.
* Set DEBUG to False in settings_local.py.

== Cron ==
Not currently required

== Tracebacks ==
* Please configure Django tracebacks to be sent to:
ross@mozillafoundation.org
Severity: normal → major
Set to major as launch date is in less than a month.
(this will get put on the generic cluster in phx1 - this is a web hosting environment so its not so much a dedicated VM, changing the title of the bug to reflect that)
Summary: Set up a stage/production VM for mozillaignite.org → Set up a stage/production environment for mozillaignite.org
Assignee: server-ops → ashish
Lowering down to normal, 30d is a long enough timeframe.
Assignee: ashish → server-ops
Severity: major → normal
Don't care about priority, but just pointing out that traffic will come in 20d, not 30d, and we need to test for several days before then.
Assignee: server-ops → cturra
Depends on: 752726
while working on deploying the -dev services, i ran into the vendor library issue noted below. can you please have this reviewed?

# git submodule update --init
Submodule 'src/bleach' () registered for path 'src/bleach'
Submodule 'src/commonware' () registered for path 'src/commonware'
Submodule 'src/django' () registered for path 'src/django'
Submodule 'src/django-arecibo' () registered for path 'src/django-arecibo'
Submodule 'src/django-mozilla-product-details' () registered for path 'src/django-mozilla-product-details'
Submodule 'src/django-nose' () registered for path 'src/django-nose'
Submodule 'src/django-session-csrf' () registered for path 'src/django-session-csrf'
Submodule 'src/django-sha2' () registered for path 'src/django-sha2'
Submodule 'src/funfactory' () registered for path 'src/funfactory'
Submodule 'src/jingo' () registered for path 'src/jingo'
Submodule 'src/jingo-minify' () registered for path 'src/jingo-minify'
Submodule 'src/nuggets' () registered for path 'src/nuggets'
Submodule 'src/schematic' () registered for path 'src/schematic'
Submodule 'src/test-utils' () registered for path 'src/test-utils'
Submodule 'src/tower' () registered for path 'src/tower'
fatal: reference is not a tree: eca4737a3fb1f4645431aed6b326409ca2177d9f
Unable to checkout 'eca4737a3fb1f4645431aed6b326409ca2177d9f' in submodule path 'src/django'
Ross: this smells like 

  https://github.com/mozilla/playdoh/issues/107

Can you confirm that you've fixed the repo as per those instructions?
Yeah sorry guys, I have fixed a number of repos but not yet that one.

Will look at doing this ASAP (probably ready to go tomorrow morning as I'm about to eat dinner right now).

Thanks, 

Ross
Morning guys, 

Sorry about yesterday - updated a few things last night but then got caught up trying to debug browserID login errors (which have now been filed as a separate bug).

Update for the vendor-library - https://github.com/mozilla/playdoh-lib/tree/e22c58091674b92252f87742af16a22c92231c58

Also - have realised that memcached needs installing and configuring on the VM also.

Any problems please shout, 

Ross
i went ahead and removed the entire vendor/ directory and did another git submodule init / git submodule update --init --recursive. unfortunately i am still seeing an error around this...

$ git submodule init
Submodule 'vendor' () registered for path 'vendor'
Submodule 'vendor-local/src/django-browserid' () registered for path 'vendor-local/src/django-browserid'
Submodule 'vendor-local/src/django-push' () registered for path 'vendor-local/src/django-push'
Submodule 'vendor-local/src/django-taggit' () registered for path 'vendor-local/src/django-taggit'
Submodule 'vendor-local/src/django-waffle' () registered for path 'vendor-local/src/django-waffle'

$ sudo git submodule update --init --recursive
Submodule 'vendor' () registered for path 'vendor'
Submodule 'vendor-local/src/django-browserid' () registered for path 'vendor-local/src/django-browserid'
Submodule 'vendor-local/src/django-push' () registered for path 'vendor-local/src/django-push'
Submodule 'vendor-local/src/django-taggit' () registered for path 'vendor-local/src/django-taggit'
Submodule 'vendor-local/src/django-waffle' () registered for path 'vendor-local/src/django-waffle'
Submodule path 'vendor': checked out 'bb2899620eccdfbab33126bb13f15d47774bb7fe'
Submodule 'src/bleach' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/bleach'
Submodule 'src/commonware' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/commonware'
Submodule 'src/django' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django'
Submodule 'src/django-arecibo' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-arecibo'
Submodule 'src/django-mozilla-product-details' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-mozilla-product-details'
Submodule 'src/django-nose' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-nose'
Submodule 'src/django-session-csrf' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-session-csrf'
Submodule 'src/django-sha2' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/django-sha2'
Submodule 'src/funfactory' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/funfactory'
Submodule 'src/jingo' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/jingo'
Submodule 'src/jingo-minify' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/jingo-minify'
Submodule 'src/nuggets' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/nuggets'
Submodule 'src/schematic' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/schematic'
Submodule 'src/test-utils' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/test-utils'
Submodule 'src/tower' (git://github.com/mozilla/playdoh-lib.git) registered for path 'src/tower'
Submodule path 'src/bleach': checked out '3657f7e32a114fcee91ed795414e4debbc5a5925'
Submodule path 'src/commonware': checked out '6df3a122417ca13764b43066ccfa122ad9299995'
fatal: reference is not a tree: eca4737a3fb1f4645431aed6b326409ca2177d9f
Submodule path 'src/django-arecibo': checked out '73c21591b55c6b2b60c4ae0ae43a26b4f4d24f72'
Submodule path 'src/django-mozilla-product-details': checked out '7154e602399412a2e63d1f96cd2bccbfd804b4bc'
Submodule path 'src/django-nose': checked out 'b16178c08728243191a922212a16d3960163feff'
Submodule path 'src/django-session-csrf': checked out '2b32149525319dff19e08a16ad1a168034e6b3dc'
Submodule path 'src/django-sha2': checked out '5cb2f080c8289d2c278fcfea59f49db36db6f4ab'
Submodule path 'src/funfactory': checked out 'f8447c468865eceb3466e4d7c3f478bfd9592a5e'
Submodule path 'src/jingo': checked out '14253ef0e8c8d0899e9f9991dd954409c3959c9e'
Submodule path 'src/jingo-minify': checked out '95f24e9ecdbeb5d1d88029896b3d53ae8828e533'
Submodule path 'src/nuggets': checked out 'ce506882b6943ea45ea4f98290fc4ef23e09a083'
Submodule path 'src/schematic': checked out 'e749978cc06bda8ba655faf8d672ae270e25b5f5'
Submodule path 'src/test-utils': checked out 'ce0e9643ea3b38373823e04d8c2e5f2dc2de5665'
Submodule path 'src/tower': checked out '79b2e9d0c9f2c1a6ee8ae0453d8ca0bf3474b308'
Unable to checkout 'eca4737a3fb1f4645431aed6b326409ca2177d9f' in submodule path 'src/django'
Failed to recurse into submodule path 'vendor'
I've made the stage branch the default - this is where the code we're deploying lives, not sure if might have been pulling from that?

Also - I think that due to all the magic that the submodules does (I'm no expect) it might be easier re-cloning from scratch?
okay. that might do it -- i was pulling dev from orgin/dev. to confirm, do you want me using orgin/stage for both dev and stage environments?
On the new VMs yes please - at least for now.
looks like we're missing a module. the following is from the apache error log...

[Thu May 10 10:41:49 2012] [info] mod_wsgi (pid=27615): Create interpreter 'mozillaignite-dev.allizom.org|'.
[Thu May 10 10:41:49 2012] [info] [client 10.8.33.241] mod_wsgi (pid=27615, process='mozillaignite-dev', application='mozillaignite-dev.allizom.org|'): Loading WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] mod_wsgi (pid=27615): Exception occurred processing WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] Traceback (most recent call last):
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]   File "/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/lib/python/django/core/handlers/wsgi.py", line 250, in __call__
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]     self.load_middleware()
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]   File "/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/lib/python/django/core/handlers/base.py", line 47, in load_middleware
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241]     raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] ImproperlyConfigured: Error importing middleware commons.middleware: "No module named requests"
[Thu May 10 10:41:53 2012] [debug] mod_headers.c(768): headers: ap_headers_error_filter()
I'm not sure about this one - but the errors look to be directly coming out of django opposed to any code that we've written.
I'm puzzled as to why there's a file:

[Thu May 10 10:41:53 2012] [error] [client 10.8.33.241] mod_wsgi (pid=27615): Exception occurred processing WSGI script '/data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi'.

betafarm has nothing to do w/ this.
:davida -- it's the wsgi script alias. from the apache config...

 WSGIScriptAlias / /data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi

 WSGIDaemonProcess mozillaignite-dev processes=6 threads=1 maximum-requests=500 display-name=mozillaignite-dev
 WSGIProcessGroup mozillaignite-dev


it appears to be the only wsgi file in the project...

 $ find mozillaignite -iname \*.wsgi
 mozillaignite/wsgi/betafarm.wsgi


is there something else i should be using instead?
(In reply to Chris Turra [:cturra] from comment #16)
> :davida -- it's the wsgi script alias. from the apache config...
> 
>  WSGIScriptAlias /
> /data/www/mozillaignite-dev.allizom.org/mozillaignite/wsgi/betafarm.wsgi
> 
>  WSGIDaemonProcess mozillaignite-dev processes=6 threads=1
> maximum-requests=500 display-name=mozillaignite-dev
>  WSGIProcessGroup mozillaignite-dev
> 
> 
> it appears to be the only wsgi file in the project...
> 
>  $ find mozillaignite -iname \*.wsgi
>  mozillaignite/wsgi/betafarm.wsgi
> 
> 
> is there something else i should be using instead?

That appears to come from https://github.com/rossbruniges/mozilla-ignite/commit/258b16bc07b68d0d9016dedbed06ec61bde4a9d0#wsgi
(In reply to Ross Bruniges from comment #12)
> On the new VMs yes please - at least for now.

Just a nitpick, but the IT managed servers are physical servers, so referring to servers instead of VMs, in this context, would avoid any confusion
(In reply to Ross Bruniges from comment #14)
> I'm not sure about this one - but the errors look to be directly coming out
> of django opposed to any code that we've written.

Since it's coming from the Django that's submoduled, it sounds like something you may need to ask Kumar about.

Since python-requests is a pure python package, it should be pulled in as part of your vendor submodule(s)
What version of python has been installed on the server?

We've been using 2.6 for local dev and mozilla labs VMs
Yes, it's Python 2.6 on the clusters of web servers this site's environments are hosted on.

python --version
Python 2.6.6

The same servers host a number of other Playdoh apps, including Firefoxflicks, MozillaLabs/Betafarm, mozillians.org, moztrap.mozilla.org, etc
Cool - I thought that would be the case. Just trying to get to the bottom of that module missing...
From running [bburton@genericadm.private.phx1 mozillaignite]$ grep -R requests *

and the output below

vendor-local/src/django-browserid/requirements.txt:requests==0.10.8
Binary file vendor-local/src/django-browserid/django_browserid/base.pyc matches
vendor-local/src/django-browserid/django_browserid/base.py:import requests
vendor-local/src/django-browserid/django_browserid/base.py:    r = requests.post(url, **parameters)
vendor-local/src/django-browserid/setup.py:    install_requires='requests>=0.9.1',
vendor-local/src/django-browserid/docs/details/settings.rst:    included with requests_ is used.
vendor-local/src/django-browserid/docs/details/settings.rst:.. _requests: http://docs.python-requests.org/

I would suspect your usage of django-browserid is what's wanting it, as it is listed in https://github.com/mozilla/django-browserid/blob/master/requirements.txt , but it's not included in vendor or vendor-local, so it's not found
Pushed a fix to hopefully sort this out - I think I had rather a big fail in regard to my knowledge of how the submodules were working.

Hopefully we've got what we need in there now.
No problem. You already know more than I do.

So I redid the clone of your repo and the submodule bits and we're passed the requests dependency.

I've submitted https://github.com/rossbruniges/mozilla-ignite/pull/52/files with an additional setup step

https://mozillaignite-dev.allizom.org/en-US/ now loads, but I'm seeing some 404s on CSS because it seems like the 'media' paths in the rendered HTML don't match the layout on the filesystem, it's expecting /media/ignite/css , etc, but everything is /media/css , etc
In settings_local.py change:

if os.environ.get('DJANGO_SITE') == 'ignite':
    from settings_ignite import *
else:
    from settings import *

to

from settings_ignite import *

Sorry - this is an oversite from when, at one stage we wanted to make sure what this codebase and the betafarm codebase could be easily merged.
Ross -- i just finished setting up stage, but the style sheets don't appear to be loading with either the default or modified settings_local.py. thoughts?

  https://mozillaignite.allizom.org
Hmm - did you run ./manage.py compress_assets after installing everything? 

What happens when you set debug to True and have the non-compressed assets load through django?
Ross -- looks like you were right and i had a case of the "friday's." after re-deploying this morning with compress_assets everything looks as expected.
Haha - I know that Friday feeling :)
I started trying to do some initial smoke tests of the site and am currently unable to login with browserID, the csrf stuff is failing and throwing 403's

Has memcached been installed and correctly configured in the settings_local.py ?
Ross - memcache was not configured on the dev instance because i based the configuration on the settings_local.py-dist that was pulled down from github. 

i just added the memcached backend to dev and tested login with browserid succesfully. give it a whirl and let me know how you make it -- i will then apply this same change to stage and prod environments.

  https://mozillaignite-dev.allizom.org
Ah yes sorry - I've been meaning to try and sort out that py.dist file for a while - will try hard to get that done to ease future deploys.

I've managed to log in but only on the second attempt, first one 403'ed on me as before.

Probably good to have other people to test out to give us a better understanding on if it was just something that cleared up when I visited the site with an empty cache...

Will try and run some more smoke tests tonight/tomorrow morning.

Thanks very much!
Heya guys,

After last weeks crazy schedule I've had a chance to look at the dev site again to run some initial smoke tests through the system, I've also put a bit of work into making the deploy tasks a bit more obvious in the settings-local.py-dist.

So from what I can see we need a few directories created inside of the media_ignite/ directory (refer to settings.py or settings-local.py-dist for names) so that the admin and create views can upload images to them. We also need to ensure that the app has write access to them.

Also, wondering if you've got any plans for the wordpress blog? There is one currently installed at mozillaignite.org/blog/ - would we just want to keep it there and redirect or install it alongside the app on our new server?

Any problems please let me know, 

Ross
(In reply to Ross Bruniges from comment #35)
> So from what I can see we need a few directories created inside of the
> media_ignite/ directory (refer to settings.py or settings-local.py-dist for
> names) so that the admin and create views can upload images to them. We also
> need to ensure that the app has write access to them.

upload directory created on netapp share and symlinked to application. 

 $ ls mozillaignite/media/img/upload/
 avatars  challenges  events  projects  topics


> Also, wondering if you've got any plans for the wordpress blog? There is one
> currently installed at mozillaignite.org/blog/ - would we just want to keep
> it there and redirect or install it alongside the app on our new server?

we can give you a blog at blog.mozilla.org and should be able to import your current data into it.
Thanks Chris, still not working but hopefully we're in there.

Can you have a look in the error log to see what's happening - each time I try to upload with an image, which gives me the impression that something weird is happening, or something un-weird like the the directory reference being wrong. Do the upload paths in local.py need to be changed for this to work? Not sure what is done on other projects.

As right now I'm looking at the dev server (https://mozillaignite-dev.allizom.org/) might it be possible to set debug-True to give us hopefully some better error messages?

We're using the admin section for a few tasks - and I'm not able to log into that yet.
Could you set up ross@mozillafoundation.org as a super-user? 
Also I think a reference to the admin assets needs to be changed in the apache-config - the need for this changed when we moved django out of a git submodule and into lib.

Sorry - rather a few more points here than I was hoping, need any more info please let me know, 

Ross
> 
> we can give you a blog at blog.mozilla.org and should be able to import your
> current data into it.
Would we be able to apply a custom style to this? 
Would we also be able to send requests from mozillaignite.org/blog onto blog.mozilla.org/mozillaignite/ (assuming that's the format of URLs that we'll be getting)
(In reply to Ross Bruniges from comment #37)
> Can you have a look in the error log to see what's happening - each time I
> try to upload with an image, which gives me the impression that something
> weird is happening, or something un-weird like the the directory reference
> being wrong. Do the upload paths in local.py need to be changed for this to
> work? Not sure what is done on other projects.

i don't see an error that points to image uploads, but do see the following in the apache error logs...

[Tue May 29 08:26:38 2012] [error] 08:26:1338305198 root:DEBUG Note: funfactory monkey patches executed in /data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/funfactory/funfactory/monkeypatches.py :/data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/funfactory/funfactory/monkeypatches.py:34
[Tue May 29 08:26:39 2012] [debug] mod_headers.c(743): headers: ap_headers_output_filter()
[Tue May 29 08:26:39 2012] [error] [client 10.8.33.241] File does not exist: /data/www/mozillaignite-dev.allizom.org/mozillaignite/vendor/src/django, referer: https://mozillaignite-dev.allizom.org/admin/


> As right now I'm looking at the dev server
> (https://mozillaignite-dev.allizom.org/) might it be possible to set
> debug-True to give us hopefully some better error messages?

looks like i missed uncommenting the debug option in the settings file. i have updated that now. sorry! 


> We're using the admin section for a few tasks - and I'm not able to log into
> that yet.
> Could you set up ross@mozillafoundation.org as a super-user? 
> Also I think a reference to the admin assets needs to be changed in the
> apache-config - the need for this changed when we moved django out of a git
> submodule and into lib.

i set the is_superuser flag on your account to true.

mysql> select * from auth_user where email='ross@mozillafoundation.org'\G
*************************** 1. row ***************************
          id: 2
    username: O_ojXJ5PhzED85OfmOcJ2LUgs3I
  first_name: 
   last_name: 
       email: ross@mozillafoundation.org
    password: !
    is_staff: 0
   is_active: 1
is_superuser: 1
  last_login: 2012-05-16 10:47:52
 date_joined: 2012-05-16 10:47:52
1 row in set (0.00 sec)
Thanks for turning DEBUG on, has provided us with the following traceback, looks like the link between where we're looking to upload the images and where the app things that they are vary a little. 

http://pastebin.com/pWQee8rG

Not sure what's going on with the django-admin (it's still not logging me in) - I thought it should be the case that when I'm logged in with browserID hitting /admin/ should drop me in, but it's not for some reason. Maybe linked to the assets not correctly loading, as mentioned in my previous email...
Actually looking at it I think I also need to be in the staff group, as well as super user!

Thanks Chris, 

Ross
(In reply to Ross Bruniges from comment #40)
> Thanks for turning DEBUG on, has provided us with the following traceback,
> looks like the link between where we're looking to upload the images and
> where the app things that they are vary a little. 
> 
> http://pastebin.com/pWQee8rG

i know what's going on here! i put the upload directory in media/img/ not media/ignite/img/

that should be fixed now if you want to give it a test.
(In reply to Ross Bruniges from comment #41)
> Actually looking at it I think I also need to be in the staff group, as well
> as super user!

mysql> update auth_user set is_staff=1 where email='ross@mozillafoundation.org';
Query OK, 1 row affected (0.07 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select * from auth_user where email='ross@mozillafoundation.org'\G
*************************** 1. row ***************************
          id: 2
    username: O_ojXJ5PhzED85OfmOcJ2LUgs3I
  first_name: 
   last_name: 
       email: ross@mozillafoundation.org
    password: !
    is_staff: 1
   is_active: 1
is_superuser: 1
  last_login: 2012-05-16 10:47:52
 date_joined: 2012-05-16 10:47:52
1 row in set (0.00 sec)
(In reply to Chris Turra [:cturra] from comment #42)
> (In reply to Ross Bruniges from comment #40)
> > Thanks for turning DEBUG on, has provided us with the following traceback,
> > looks like the link between where we're looking to upload the images and
> > where the app things that they are vary a little. 
> > 
> > http://pastebin.com/pWQee8rG
> 
> i know what's going on here! i put the upload directory in media/img/ not
> media/ignite/img/
> 
> that should be fixed now if you want to give it a test.

Not sure if I was too quick to test but just got the exact same error...
(In reply to Ross Bruniges from comment #44)
> Not sure if I was too quick to test but just got the exact same error...

you might have been a tad quick. the environment update has now finished if you want to test again.
(In reply to Chris Turra [:cturra] from comment #45)
> (In reply to Ross Bruniges from comment #44)
> > Not sure if I was too quick to test but just got the exact same error...
> 
> you might have been a tad quick. the environment update has now finished if
> you want to test again.

Still the same error :(
(In reply to Ross Bruniges from comment #46)
> (In reply to Chris Turra [:cturra] from comment #45)
> > (In reply to Ross Bruniges from comment #44)
> > > Not sure if I was too quick to test but just got the exact same error...
> > 
> > you might have been a tad quick. the environment update has now finished if
> > you want to test again.
> 
> Still the same error :(

i figured it out! i had named the directory 'upload' not 'uploads'. should be fixed now. 

feel free to hit me up on irc (cturra) if you want to chat about this more.
(In reply to Chris Turra [:cturra] from comment #47)
> (In reply to Ross Bruniges from comment #46)
> > (In reply to Chris Turra [:cturra] from comment #45)
> > > (In reply to Ross Bruniges from comment #44)
> > > > Not sure if I was too quick to test but just got the exact same error...
> > > 
> > > you might have been a tad quick. the environment update has now finished if
> > > you want to test again.
> > 
> > Still the same error :(
> 
> i figured it out! i had named the directory 'upload' not 'uploads'. should
> be fixed now. 
> 
> feel free to hit me up on irc (cturra) if you want to chat about this more.

Ha - always those typos :)

So, we've not got a permissions error on that directory, I've had some trouble with django and this before. It might have to be 777 if 755 isn't working...

What IRC rooms do you hang out in? I'm soon to be going home (am on UK time here unfortunately) but what IRC do you hang out in? I'm normally in #webdev, #london, #mofodev and am Boozeniges.

Thanks for the help today, sorry to be such a pain but feels like we've made some good progress!

Cheers, 

Ross
(In reply to Ross Bruniges from comment #48)
> So, we've not got a permissions error on that directory, I've had some
> trouble with django and this before. It might have to be 777 if 755 isn't
> working...

i changed the permissions from 755 to 766 so [w]rite was included. 


> What IRC rooms do you hang out in? I'm soon to be going home (am on UK time
> here unfortunately) but what IRC do you hang out in? I'm normally in
> #webdev, #london, #mofodev and am Boozeniges.

i am many rooms, but it would probably be easiest if you just pm me directly. 

 /msg cturra
Still not working - will try and find you in an IRC room today.

Thanks, 

Ross
Morning, 

Can we check to see if memcached is still running and configured correctly? 

It looks like the browserID login is broken and throwing 403 errors across both dev and staging.
In addition am hoping that you can set up update_site_feeds to run as a cron every 4 hours, don't mind too much about the minute - let's say 10 minutes past.

Information on cron can be found here - https://github.com/rossbruniges/mozilla-ignite/commit/3737879156c79cbef1f686c7a14ca37e10bff8d6
(In reply to Ross Bruniges from comment #51)
> Can we check to see if memcached is still running and configured correctly? 
> 
> It looks like the browserID login is broken and throwing 403 errors across
> both dev and staging.

the memcache servers we were initially using were incorrect. i have now updated -dev and -stage to use the following which has resolved these http/403s

  'LOCATION': [
    'memcache-generic01:11211',
    'memcache-generic02:11211',
  ],
closing bug because both -dev and -stage environments are up and running. production environment setup is covered by bug 762525
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.