798950 - TypeError: Missing "generate_reset_code" in "<class 'services.user.sql.SQLUser'>"

Status: RESOLVED FIXED

(Cloud Services :: General, defect)

Description

[arthur.lutz]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120907231657

Steps to reproduce:

upgrade sync server from hg


Actual results:

# ./bin/paster serve development.ini
Traceback (most recent call last):
  File "./bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/var/www/sync/lib/python2.6/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/var/www/sync/lib/python2.6/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/var/www/sync/lib/python2.6/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/var/www/sync/lib/python2.6/site-packages/paste/script/serve.py", line 284, in command
    relative_to=base, global_conf=vars)
  File "/var/www/sync/lib/python2.6/site-packages/paste/script/serve.py", line 321, in loadapp
    **kw)
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/util.py", line 59, in fix_call
    reraise(*exc_info)
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/compat.py", line 22, in reraise
    exec('raise t, e, tb', dict(t=t, e=e, tb=tb))
  File "/var/www/sync/lib/python2.6/site-packages/paste/deploy/util.py", line 56, in fix_call
    val = callable(*args, **kw)
  File "/var/www/sync/deps/server-core/services/baseapp.py", line 341, in make_app
    app = klass(urls, controllers, params, auth_class)
  File "/var/www/sync/deps/server-storage/syncstorage/wsgiapp.py", line 102, in __init__
    auth_class)
  File "/var/www/sync/deps/server-core/services/baseapp.py", line 85, in __init__
    self.auth = None if auth_class is None else auth_class(self.config)
  File "/var/www/sync/deps/server-core/services/wsgiauth.py", line 53, in __init__
    self.backend = get_auth(self.config)
  File "/var/www/sync/deps/server-core/services/auth/__init__.py", line 233, in get_auth
    return ServicesAuth.get_from_config(config, 'auth')
  File "/var/www/sync/deps/server-core/services/pluginreg.py", line 138, in get_from_config
    return cls.get(backend_name, **config)
  File "/var/www/sync/deps/server-core/services/pluginreg.py", line 143, in get
    klass = cls._get_backend_class(name)
  File "/var/www/sync/deps/server-core/services/pluginreg.py", line 128, in _get_backend_class
    cls.register(klass)
  File "/var/www/sync/lib/python2.6/abc.py", line 106, in register
    if issubclass(subclass, cls):
  File "/var/www/sync/lib/python2.6/abc.py", line 158, in __subclasscheck__
    ok = cls.__subclasshook__(subclass)
  File "/var/www/sync/deps/server-core/services/pluginreg.py", line 151, in __subclasshook__
    raise TypeError('Missing "%s" in "%s"' % (method, klass))
TypeError: Missing "generate_reset_code" in "<class 'services.user.sql.SQLUser'>"



Expected results:

no traceback

Comment 1

[Ryan Kelly [:rfkelly]]

(In reply to arthur.lutz from comment #0)
> User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101
> Firefox/15.0.1
> Build ID: 20120907231657
> 
> Steps to reproduce:
> 
> upgrade sync server from hg

How did you perform this update?  If you simply did `hg pull` in the server-full directory, then it's likely that some of the dependencies did not get updated correctly.

The Makefile has an 'update' target that should ensure all of the necessary dependencies are up to date, please try `make update` in the server-full directory and see if that fixes the problem.

Comment 2

[arthur.lutz]

The make update was falling because the etc/sync.conf was modified. So I put it on the side :

 mv etc/sync.conf etc/sync.conf.mine

Did the make update (seemed to work fine) and reintegrated my changes in etc/sync.conf

Maybe etc/sync.conf should not be in the depot (have a etc/sync.conf.template ? )

Now I have a different error : 

ImportError: No module named metlog_cef.cef_plugin

Comment 3

[arthur.lutz]

I manually installed metlog_cef by doing : 

./bin/activate
easy_install metlog_cef

I then went back to the error TypeError: Missing "generate_reset_code" in "<class 'services.user.sql.SQLUser'>"

the hg update didn't seem to have worked, so I forced the hg update in base directory (and in deps/server-core). Still the same error.

Got rid of the error by doing /etc/apache restart (no indication this is need anywhere in https://docs.services.mozilla.com/howtos/run-sync.html#updating-the-server )

Comment 4

[arthur.lutz]

Now, for some reason, I get a 401 when trying to sync. 

Updating to new password doesn't work. "Password incorrect, try again" (on a New password ?!!)

Trying reset is not successfull either. 

[Fri Oct 26 10:50:51 2012] [error] Uncaught exception while processing request:
[Fri Oct 26 10:50:51 2012] [error] GET /weave-password-reset
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/util.py", line 304, in __call__
[Fri Oct 26 10:50:51 2012] [error]     return self.app(environ, start_response)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/paste/translogger.py", line 68, in __call__
[Fri Oct 26 10:50:51 2012] [error]     return self.application(environ, replacement_start_response)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/webob/dec.py", line 147, in __call__
[Fri Oct 26 10:50:51 2012] [error]     resp = self.call_func(req, *args, **self.kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/webob/dec.py", line 208, in call_func
[Fri Oct 26 10:50:51 2012] [error]     return self.func(req, *args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/baseapp.py", line 225, in __notified
[Fri Oct 26 10:50:51 2012] [error]     response = func(self, request)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/baseapp.py", line 259, in __call__
[Fri Oct 26 10:50:51 2012] [error]     response = self._dispatch_request(request)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/baseapp.py", line 317, in _dispatch_request
[Fri Oct 26 10:50:51 2012] [error]     response = self._dispatch_request_with_match(request, match)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/baseapp.py", line 345, in _dispatch_request_with_match
[Fri Oct 26 10:50:51 2012] [error]     result = function(request, **params)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 154, in __call__
[Fri Oct 26 10:50:51 2012] [error]     return self._real_call(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 151, in _real_call
[Fri Oct 26 10:50:51 2012] [error]     return replacement(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/metrics.py", line 101, in metlog_call
[Fri Oct 26 10:50:51 2012] [error]     return self._fn(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 154, in __call__
[Fri Oct 26 10:50:51 2012] [error]     return self._real_call(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 151, in _real_call
[Fri Oct 26 10:50:51 2012] [error]     return replacement(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/stats.py", line 50, in metlog_call
[Fri Oct 26 10:50:51 2012] [error]     result = self._fn(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 154, in __call__
[Fri Oct 26 10:50:51 2012] [error]     return self._real_call(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/metlog_py-0.9.8-py2.6.egg/metlog/decorators/base.py", line 151, in _real_call
[Fri Oct 26 10:50:51 2012] [error]     return replacement(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-core/services/metrics.py", line 114, in metlog_call
[Fri Oct 26 10:50:51 2012] [error]     result = self._fn(*args, **kwargs)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-reg/syncreg/controllers/user.py", line 351, in password_reset_form
[Fri Oct 26 10:50:51 2012] [error]     return render_mako('password_ask_reset_form.mako', **kw)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/deps/server-reg/syncreg/util.py", line 54, in render_mako
[Fri Oct 26 10:50:51 2012] [error]     template = _lookup.get_template(template)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/mako/lookup.py", line 236, in get_template
[Fri Oct 26 10:50:51 2012] [error]     return self._load(srcfile, uri)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/mako/lookup.py", line 302, in _load
[Fri Oct 26 10:50:51 2012] [error]     **self.template_args)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/mako/template.py", line 276, in __init__
[Fri Oct 26 10:50:51 2012] [error]     module = self._compile_from_file(path, filename)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/mako/template.py", line 332, in _compile_from_file
[Fri Oct 26 10:50:51 2012] [error]     self.module_writer)
[Fri Oct 26 10:50:51 2012] [error]   File "/var/www/sync/lib/python2.6/site-packages/mako/template.py", line 624, in _compile_module_file
[Fri Oct 26 10:50:51 2012] [error]     (dest, name) = tempfile.mkstemp(dir=os.path.dirname(outputpath))
[Fri Oct 26 10:50:51 2012] [error]   File "/usr/lib/python2.6/tempfile.py", line 293, in mkstemp
[Fri Oct 26 10:50:51 2012] [error]     return _mkstemp_inner(dir, prefix, suffix, flags)
[Fri Oct 26 10:50:51 2012] [error]   File "/usr/lib/python2.6/tempfile.py", line 228, in _mkstemp_inner
[Fri Oct 26 10:50:51 2012] [error]     fd = _os.open(file, flags, 0600)
[Fri Oct 26 10:50:51 2012] [error] <type 'exceptions.OSError'>
[Fri Oct 26 10:50:51 2012] [error] OSError(13, 'Permission denied')

Comment 5

[arthur.lutz]

It's trying to make a tmpfile in /var/www/sync/deps/server-reg/syncreg/templates/tmpmJsugE

Comment 6

[Ryan Kelly [:rfkelly]]

Attachment: Makefile patch to pre-compile mako templates in server-full

(In reply to arthur.lutz from comment #5)
> It's trying to make a tmpfile in
> /var/www/sync/deps/server-reg/syncreg/templates/tmpmJsugE

It's trying to compile the .mako template files in that directory into their corresponding .mako.py files.  We really should pre-compile these as part of the build process.  I'm attaching a patch that will do that, can you please try apply this patch, running `make update` again, and see if that fixes the error?

(It sounds like there's probably another error here as well, but one thing at a time...)

Comment 7

[arthur.lutz]

In "make build"  I get : 

for TMPL in `find . -name '*.mako'`; do python -c "from mako.template import Template; Template(filename='$TMPL', module_directory='.')"; done;
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named mako.template

Should there not be python options to take into account the virtualenv ? 

I then launched the command after having sources the virtualenv (. /var/www/sync/bin/activate) Seemed to do something, no errors. 

Going back to /weave-password-reset I get the same error in my apache error log.

Comment 8

[Ryan Kelly [:rfkelly]]

Attachment: Makefile patch to pre-compile mako templates in server-full

Whoops, that patch was not using the python from the virtualenv.  Please try this updated version.

Comment 9

[Ryan Kelly [:rfkelly]]

(In reply to arthur.lutz from comment #7)
> Going back to /weave-password-reset I get the same error in my apache error
> log.

The same error about "permission denied"?

Can you please check in the directories "syncserver/templates" and "deps/server-reg/syncreg/templates", there should now be a .py and a .pyc file corresponding to each .mako template file.

Comment 10

[arthur.lutz]

rfkelly : yes same permission denied error. 

(sync)root@host:/var/www/sync# ls syncserver/templates/
base.mako     base.mako.pyc        delete_account.mako.py
base.mako.py  delete_account.mako  delete_account.mako.pyc
(sync)root@host:/var/www/sync# ls deps/server-reg/syncreg/templates/
base.mako                         password_failure.mako
base.mako.py                      password_failure.mako.py
base.mako.pyc                     password_failure.mako.pyc
captcha.mako                      password_key_sent.mako
captcha.mako.py                   password_key_sent.mako.py
captcha.mako.pyc                  password_key_sent.mako.pyc
password_ask_reset_form.mako      password_reset_form.mako
password_ask_reset_form.mako.py   password_reset_form.mako.py
password_ask_reset_form.mako.pyc  password_reset_form.mako.pyc
password_changed.mako             password_reset_mail.mako
password_changed.mako.py          password_reset_mail.mako.py
password_changed.mako.pyc         password_reset_mail.mako.pyc

Doing chown -R www-data:www-data and restarting apache don't get rid of the error.

Comment 11

[Ryan Kelly [:rfkelly]]

(In reply to arthur.lutz from comment #10)
> Doing chown -R www-data:www-data and restarting apache don't get rid of the
> error.

Can you please post an `ls -l` of ./deps/server-reg/syncreg and ./deps/server-reg/syncreg/templates?  I will try to reproduce your setup more exactly from here.

Comment 12

[Ryan Kelly [:rfkelly]]

Thinking some more, this password-reset thing may be a bit of a red-herring anyway - it's important that we get the templates working, but unless you have configured things just-so then the password reset email won't work properly anyway.

> Now, for some reason, I get a 401 when trying to sync. 
> Updating to new password doesn't work. "Password incorrect, try again" (on a New password ?!!)

Can you please take a look in your database and see what tables are there.  There should be at least one of a "user" or "users" table, and possibly both.  If there are both, which table has rows in it corresponding to your user?

Comment 13

[arthur.lutz]

(In reply to Ryan Kelly [:rfkelly] from comment #11)
> (In reply to arthur.lutz from comment #10)
> > Doing chown -R www-data:www-data and restarting apache don't get rid of the
> > error.
> 
> Can you please post an `ls -l` of ./deps/server-reg/syncreg and
> ./deps/server-reg/syncreg/templates?  I will try to reproduce your setup
> more exactly from here.

root@host:/var/www/sync# ls -l ./deps/server-reg/syncreg
total 44
drwxr-xr-x 2 www-data www-data 4096 Oct 26 10:36 controllers
-rw-r--r-- 1 www-data www-data 1758 Sep 19  2011 __init__.py
-rw-r--r-- 1 www-data www-data  259 Sep 22  2011 __init__.pyc
-rw-r--r-- 1 www-data www-data 2269 Sep 19  2011 run.py
drwxr-xr-x 2 www-data www-data 4096 Sep 19  2011 static
drwxr-xr-x 2 www-data www-data 4096 Oct 29 08:30 templates
drwxr-xr-x 3 www-data www-data 4096 Oct 26 10:05 tests
-rw-r--r-- 1 www-data www-data 2271 Sep 19  2011 util.py
-rw-r--r-- 1 www-data www-data  859 Sep 22  2011 util.pyc
-rw-r--r-- 1 www-data www-data 3412 Oct 26 10:05 wsgiapp.py
-rw-r--r-- 1 www-data www-data 1960 Oct 26 10:36 wsgiapp.pyc
root@host:/var/www/sync# ls -l ./deps/server-reg/syncreg/templates
total 96
-rw-r--r-- 1 www-data www-data 1138 Sep 19  2011 base.mako
-rw------- 1 www-data www-data 2013 Oct 29 08:30 base.mako.py
-rw------- 1 www-data www-data 2295 Oct 29 08:30 base.mako.pyc
-rw-r--r-- 1 www-data www-data  310 Sep 19  2011 captcha.mako
-rw------- 1 www-data www-data 1354 Oct 29 08:30 captcha.mako.py
-rw------- 1 www-data www-data 1531 Oct 29 08:30 captcha.mako.pyc
-rw-r--r-- 1 www-data www-data  515 Oct 26 10:05 password_ask_reset_form.mako
-rw------- 1 www-data www-data 1789 Oct 29 08:30 password_ask_reset_form.mako.py
-rw------- 1 www-data www-data 2505 Oct 29 08:30 password_ask_reset_form.mako.pyc
-rw-r--r-- 1 www-data www-data   61 Sep 19  2011 password_changed.mako
-rw------- 1 www-data www-data 1186 Oct 29 08:30 password_changed.mako.py
-rw------- 1 www-data www-data 1915 Oct 29 08:30 password_changed.mako.pyc
-rw-r--r-- 1 www-data www-data  137 Sep 19  2011 password_failure.mako
-rw------- 1 www-data www-data 1390 Oct 29 08:30 password_failure.mako.py
-rw------- 1 www-data www-data 2078 Oct 29 08:30 password_failure.mako.pyc
-rw-r--r-- 1 www-data www-data  172 Sep 19  2011 password_key_sent.mako
-rw------- 1 www-data www-data 1300 Oct 29 08:30 password_key_sent.mako.py
-rw------- 1 www-data www-data 2033 Oct 29 08:30 password_key_sent.mako.pyc
-rw-r--r-- 1 www-data www-data  862 Oct 26 10:05 password_reset_form.mako
-rw------- 1 www-data www-data 2763 Oct 29 08:30 password_reset_form.mako.py
-rw------- 1 www-data www-data 3054 Oct 29 08:30 password_reset_form.mako.pyc
-rw-r--r-- 1 www-data www-data  287 Oct 26 10:05 password_reset_mail.mako
-rw------- 1 www-data www-data 1435 Oct 29 08:30 password_reset_mail.mako.py
-rw------- 1 www-data www-data 1665 Oct 29 08:30 password_reset_mail.mako.pyc

Comment 14

[arthur.lutz]

(In reply to Ryan Kelly [:rfkelly] from comment #12)
> Thinking some more, this password-reset thing may be a bit of a red-herring
> anyway - it's important that we get the templates working, but unless you
> have configured things just-so then the password reset email won't work
> properly anyway.
> 
> > Now, for some reason, I get a 401 when trying to sync. 
> > Updating to new password doesn't work. "Password incorrect, try again" (on a New password ?!!)
> 
> Can you please take a look in your database and see what tables are there. 
> There should be at least one of a "user" or "users" table, and possibly
> both.  If there are both, which table has rows in it corresponding to your
> user?

I have both tables in the database, but only "users" has one line with my user record in it.

Comment 15

[Ryan Kelly [:rfkelly]]

(In reply to arthur.lutz from comment #14)
> 
> I have both tables in the database, but only "users" has one line with my
> user record in it.

OK, I think the update has switched you over to our new-style auth backend. In your config file, there should be an [auth] section with "backend = services.user.sql.SQLUser", does that exist?

You will need to copy your user account info across from the "users" table to the "user" table.  I think the following query should do the trick:

  INSERT INTO user (userid, username, password, accountStatus, mail, mailVerified, syncNode) SELECT id, username, password_hash, 1, email, 1, "" FROM users WHERE email = "your-email-address"

Sorry for the trouble here, I should add some notes on this to the doc.