1116811 - Add /media/bundles share to snippets-dev

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Engagement, task)

Description

[Osmose [:osmose, :mkelly]]

In bug 1094136, we added a media directory to snippets. The discussion seems to be off-bug, but I remember we had issues where, essentially, each new subdirectory in media needed a new share added or some change made to the existing share.

I've added a feature (bug 1091939) that requires the /media/bundles directory and am getting Permission Denied errors when attempting to write to it on snippets-dev. Can we add that directory to the media share for snippets-dev? Thanks!

Comment 1

[Osmose [:osmose, :mkelly]]

Oh yeah, just filing this for now, this can wait until after the holiday change freeze. :D

Comment 2

[Chris Turra [:cturra]]

previously the files directory within media was a symlink to netapp. i have changed dev to symlink the entire media netapp mount so you should now be able to write files/directories within media.

 [[e-mail]1 snippets-dev.allizom.org]# ls -l media
 lrwxrwxrwx 1 root root 46 Dec 31 11:23 media -> /mnt/netapp_dev/snippets-dev.allizom.org/media


once you've tested this in dev (and after the change freeze) we can push this out to stage/prod for you. lets track that in a separate "to prod" deployment bug.

Comment 3

[Osmose [:osmose, :mkelly]]

Thanks!

Still getting a PermissionDenied error on dev when the app attempts to make a bundles directory in /media before saving files to it:

Traceback (most recent call last):

  File "/data/www/snippets-dev.allizom.org/vendor-local/src/django/django/core/handlers/base.py", line 90, in get_response
    response = middleware_method(request)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.11.0.55/newrelic/api/object_wrapper.py", line 216, in __call__
    self._nr_instance, args, kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.11.0.55/newrelic/hooks/framework_django.py", line 204, in wrapper
    return wrapped(*args, **kwargs)

  File "/data/www/snippets-dev.allizom.org/snippets/base/middleware.py", line 20, in process_request
    return result.func(request, *result.args, **result.kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.11.0.55/newrelic/api/object_wrapper.py", line 216, in __call__
    self._nr_instance, args, kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.11.0.55/newrelic/hooks/framework_django.py", line 475, in wrapper
    return wrapped(*args, **kwargs)

  File "/data/www/snippets-dev.allizom.org/snippets/base/views.py", line 130, in fetch_snippets
    return fetch_pregenerated_snippets(request, **kwargs)

  File "/data/www/snippets-dev.allizom.org/vendor-local/src/django/django/views/decorators/cache.py", line 75, in _cache_controlled
    response = viewfunc(request, *args, **kw)

  File "/data/www/snippets-dev.allizom.org/snippets/base/decorators.py", line 10, in _wrapped_view
    response = view_fn(*args, **kwargs)

  File "/data/www/snippets-dev.allizom.org/snippets/base/views.py", line 97, in fetch_pregenerated_snippets
    bundle.generate()

  File "/data/www/snippets-dev.allizom.org/snippets/base/models.py", line 155, in generate
    self.storage.save(self.filename, ContentFile(bundle_content))

  File "/data/www/snippets-dev.allizom.org/vendor-local/src/django/django/core/files/storage.py", line 48, in save
    name = self._save(name, content)

  File "/data/www/snippets-dev.allizom.org/vendor-local/src/django/django/core/files/storage.py", line 170, in _save
    os.makedirs(directory)

  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)

OSError: [Errno 13] Permission denied: '/data/www/snippets-dev.allizom.org/media/bundles'

Also, I noticed that the old contents of media were moved to media.orig. Can you copy the files subdirectory from the old media directory to the new media directory? We still need uploaded files there.

Comment 4

[Chris Turra [:cturra]]

oh! i know what's going on. the last update/deploy overwrote the media directory with what was in your github repo. if we want to symlink the media directory completely, we'll need to have you to add it to your .gitignore file. right now i see media/ in there, but in this case, we'd want to remove the trailing slash.

 https://github.com/mozilla/snippets-service/blob/master/.gitignore#L21


if you'd rather not do that, we'll need to create each sub directory within media/ on your behalf.

just let me know how you want to proceed.

Comment 5

[Osmose [:osmose, :mkelly]]

(In reply to Chris Turra [:cturra] from comment #4)
> oh! i know what's going on. the last update/deploy overwrote the media
> directory with what was in your github repo. if we want to symlink the media
> directory completely, we'll need to have you to add it to your .gitignore
> file. right now i see media/ in there, but in this case, we'd want to remove
> the trailing slash.
> 
>  https://github.com/mozilla/snippets-service/blob/master/.gitignore#L21
> 
> 
> if you'd rather not do that, we'll need to create each sub directory within
> media/ on your behalf.
> 
> just let me know how you want to proceed.

Oh yeaaaah, I remember now. The reason we did it this way was so that we could have an .htaccess file in the Github repo. Okay.

Yeah, let's go with an individual bundles directory just like we have with files. Sorry for the mixup!

Comment 6

[Chris Turra [:cturra]]

added as requested. you should be good to go with this :)

[[e-mail]1 snippets-dev.allizom.org]# ls -l media/
total 0
lrwxrwxrwx 1 root root 54 Jan  2 11:56 bundles -> /mnt/netapp_dev/snippets-dev.allizom.org/media/bundles
lrwxrwxrwx 1 root root 52 Jan  2 11:54 files -> /mnt/netapp_dev/snippets-dev.allizom.org/media/files