848960 - new input deprecation warnings

Status: RESOLVED FIXED

(Input :: General, defect)

Description

[Chris Turra [:cturra]]

in the new input dev and stage environments, we're seeing the following deprecation warnings for all http requests in the apache error logs. as discussed with :willkg, they seem to be three unique errors:

[error] /usr/lib64/python2.6/site-packages/jinja2/__init__.py:31: UserWarning: Module simplejson was already imported from /data/www/input.allizom.org/input/vendor-local/packages/simplejson/simplejson/__init__.py, but /usr/lib64/python2.6/site-packages is being added to sys.path
   __version__ = __import__('pkg_resources') \\


[error] /usr/lib64/python2.6/site-packages/jinja2/__init__.py:31: UserWarning: Module memcache was already imported from /data/www/input.allizom.org/input/vendor-local/packages/python-memcached/memcache.py, but /usr/lib/python2.6/site-packages is being added to sys.path
   __version__ = __import__('pkg_resources') \\


[error] /data/www/input.allizom.org/input/vendor/lib/python/celery/decorators.py:34: CDeprecationWarning:
[error] The `celery.decorators` module along with the magic keyword arguments,
[error] are deprecated, and will be removed in version 3.0.
[error]
[error] Please use the `celery.task` module instead of `celery.decorators`,
[error] and the `task.request` should be used instead of the magic keyword arguments:
[error]
[error]     from celery.task import task
[error]
[error] See http://bit.ly/celery22major for more information.
[error]
[error]
[error]   """))
[error] /data/www/input.allizom.org/input/vendor/lib/python/django/views/generic/simple.py:8: DeprecationWarning: Function-based generic views have been deprecated; use class-based views instead.
[error]   DeprecationWarning

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

The celery one is a problem with our software. I forget what's kicking up the error specifically, but I think we just fixed it in Kitsune so I think I can fix it in Fjord.

The simplejson and memcached errors, I suspect, are because those are system packages and in vendor-local.

Can you see if there are system packages installed for simplejson and the python bindings for memcache?

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this block bug #832316.

Comment 3

[Chris Turra [:cturra]]

(In reply to Will Kahn-Greene [:willkg] from comment #1)
> The simplejson and memcached errors, I suspect, are because those are system
> packages and in vendor-local.
> 
> Can you see if there are system packages installed for simplejson and the
> python bindings for memcache?

[[e-mail]1 ~]# yum list installed | grep -i simplejson
python-simplejson.x86_64         2.0.9-3.1.el6             @rhel-x86_64-server-6

[[e-mail]1 ~]# yum list installed | grep -i memcached
python-memcached.noarch          1.43-6.el6                @rhel-x86_64-server-6

Comment 4

[Chris Turra [:cturra]]

*submitted a bit quick*

in vendor-local:

 simplejson = Version: 3.0.7
 python-memcached = Version: 1.48


i can remove the system (yum) packages if you think those are getting in the way here?

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

If you could remove the system ones, that'd be great.

If not, I think I can change the packaging so they're not in vendor-local.

Comment 6

[Chris Turra [:cturra]]

the following has been added to the input puppet manifest:

+ # bug 848960
+ package {
+   [
+     'python-simplejson',
+     'python-memcached',
+   ]:
+   ensure => 'absent',
+ }


and after a puppet run:

...
notice: /Stage[main]/Webapp::Input/Package[python-simplejson]/ensure: removed
notice: /Stage[main]/Webapp::Input/Package[python-memcached]/ensure: removed
...

[[e-mail]1 ~]# yum list installed | grep -i 'memcached|simplejson' | wc -l
0

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Awesome! I'll deal with the celery thing asap.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pretty sure https://github.com/mozilla/fjord/commit/d582e01f1b073a74cd35e078d5e5d27612011afa fixes the deprecation error.

I committed it to master so it'll be on input-dev soon. We should check the web node and celery node logs to see if this is still popping up.

Comment 9

[Chris Turra [:cturra]]

looks like these previous errors have cleared up, but there is one new one in the apache error logs:

[error] /data/www/input-dev.allizom.org/input/vendor/lib/python/django/views/generic/simple.py:8: DeprecationWarning: Function-based generic views have been deprecated; use class-based views instead.
[error]   DeprecationWarning

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

PR: https://github.com/mozilla/fjord/pull/79

Landed in master in https://github.com/mozilla/fjord/commit/f93d3f935908a986cead38728d58e69b4bf6700f

I think that'll fix the deprecation warnings. I think dev is still using cron, so it'll show up there in 10-15 minutes. I just deployed it to stage using chief, so it should be there now.

Do you still see these deprecation warnings in the logs?

Comment 11

[Chris Turra [:cturra]]

the signs look good. i haven't seen this deprecated warning pop up for almost an hour!

[[e-mail]1 ~]# tail -2 /var/log/httpd/input.mozilla.org/error_log_2013-03-19-22
[Tue Mar 19 15:39:49 2013] [error] /data/www/input.allizom.org/input/vendor/lib/python/django/views/generic/simple.py:8: DeprecationWarning: Function-based generic views have been deprecated; use class-based views instead.
[Tue Mar 19 15:39:49 2013] [error]   DeprecationWarning
[[e-mail]1 ~]# date
Wed Mar 20 16:30:44 PDT 2013

i am going to mark this bug as r/fixed. 

nice work :willkg!

Comment 12

[Will Kahn-Greene [:willkg] ET needinfo? me]

Yay! Thank you for helping debug it.