627556 - Module to implement our split reg/sreg in server-core

Status: RESOLVED FIXED

(Cloud Services :: Server: Core, defect)

Description

[Toby Elliott [:telliott]]

We need to have a setup in server-core auth that understands our reg/sreg architecture. That means it does the expected ldap stuff except for the calls (mostly password reset code stuff and creating users) that get handed off to sreg.

Comment 1

[Toby Elliott [:telliott]]

Three files about to be attached for review:

1) mozilla.php library to implement the split (subclassing ldapsql)
2) a unit test for it. At the moment, I've used wsgi_intercept to emulate the backend. This could be better, but it's beyond the scope I can get done in the timeframe.
3) A small patch to ldapsql.py to reflect the fact that I believe the mozilla.py class doesn't need mysql.

Comment 2

[Toby Elliott [:telliott]]

Attachment: library to handle 2-tier auth functions

Comment 3

[Toby Elliott [:telliott]]

Attachment: unittests for the above

Comment 4

[Toby Elliott [:telliott]]

Attachment: Bypasses sql standup if we're not using it.

Comment 5

[Toby Elliott [:telliott]]

Attachment: Handles 2-tier auth

Comment 6

[Tarek Ziadé (:tarek)]

Comment on attachment 505845 [details] [diff] [review]
Handles 2-tier auth

>diff -r 1b026734d301 services/auth/__init__.py
>--- a/services/auth/__init__.py	Tue Jan 18 17:31:45 2011 +0100
>+++ b/services/auth/__init__.py	Fri Jan 21 10:18:39 2011 -0800
>@@ -207,6 +207,12 @@
>     except ImportError:
>         pass
> 
>+    try:
>+        from services.auth.mozilla import MozillaAuth
>+        ServicesAuth.register(MozillaAuth)
>+    except ImportError:
>+        pass
>+
>     from services.auth.dummy import DummyAuth
>     ServicesAuth.register(DummyAuth)
> 
>diff -r 1b026734d301 services/auth/ldapsql.py
>--- a/services/auth/ldapsql.py	Tue Jan 18 17:31:45 2011 +0100
>+++ b/services/auth/ldapsql.py	Fri Jan 21 10:18:39 2011 -0800
>@@ -122,13 +122,14 @@
>                  'pool_recycle': int(pool_recycle),
>                  'logging_name': 'weaveserver'}
> 
>-        if self.sqluri.startswith('mysql'):
>-            sqlkw['reset_on_return'] = reset_on_return
>-        self._engine = create_engine(sqluri, **sqlkw)
>-        for table in tables:
>-            table.metadata.bind = self._engine
>-            if create_tables:
>-                table.create(checkfirst=True)
>+        if self.sqluri is not None:
>+            if self.sqluri.startswith('mysql'):
>+                sqlkw['reset_on_return'] = reset_on_return
>+            self._engine = create_engine(sqluri, **sqlkw)
>+            for table in tables:
>+                table.metadata.bind = self._engine
>+                if create_tables:
>+                    table.create(checkfirst=True)
> 
>     def _conn(self, bind=None, passwd=None):
>         return self.pool.connection(bind, passwd)
>diff -r 1b026734d301 services/auth/mozilla.py
>--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
>+++ b/services/auth/mozilla.py	Fri Jan 21 10:18:39 2011 -0800
>@@ -0,0 +1,190 @@
>+# ***** BEGIN LICENSE BLOCK *****
>+# Version: MPL 1.1/GPL 2.0/LGPL 2.1
>+#
>+# The contents of this file are subject to the Mozilla Public License Version
>+# 1.1 (the "License"); you may not use this file except in compliance with
>+# the License. You may obtain a copy of the License at
>+# http://www.mozilla.org/MPL/
>+#
>+# Software distributed under the License is distributed on an "AS IS" basis,
>+# WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
>+# for the specific language governing rights and limitations under the
>+# License.
>+#
>+# The Original Code is Sync Server
>+#
>+# The Initial Developer of the Original Code is the Mozilla Foundation.
>+# Portions created by the Initial Developer are Copyright (C) 2010
>+# the Initial Developer. All Rights Reserved.
>+#
>+# Contributor(s):
>+#   Toby Elliott ([e-mail])
>+#
>+# Alternatively, the contents of this file may be used under the terms of
>+# either the GNU General Public License Version 2 or later (the "GPL"), or
>+# the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
>+# in which case the provisions of the GPL or the LGPL are applicable instead
>+# of those above. If you wish to allow use of your version of this file only
>+# under the terms of either the GPL or the LGPL, and not to allow others to
>+# use your version of this file under the terms of the MPL, indicate your
>+# decision by deleting the provisions above and replace them with the notice
>+# and other provisions required by the GPL or the LGPL. If you do not delete
>+# the provisions above, a recipient may use your version of this file under
>+# the terms of any one of the MPL, the GPL or the LGPL.
>+#
>+# ***** END LICENSE BLOCK *****
>+""" Mozilla Authentication using a two-tier system
>+"""
>+
>+import ldap
>+import simplejson as json
>+
>+from urllib import urlencode
>+from services.util import check_reset_code, BackendError, get_url
>+from services.auth.ldapsql import LDAPAuth
>+from services import logger
>+
>+
>+class MozillaAuth(LDAPAuth):
>+    """LDAP authentication."""
>+
>+    def __init__(self, ldapuri, sreg_location, sreg_path, sreg_scheme='https',
>+                 use_tls=False, bind_user='binduser',
>+                 bind_password='binduser', admin_user='adminuser',
>+                 admin_password='adminuser', users_root='ou=users,dc=mozilla',
>+                 users_base_dn=None, pool_size=100, pool_recycle=3600,
>+                 reset_on_return=True, single_box=False, ldap_timeout=-1,
>+                 nodes_scheme='https', check_account_state=True,
>+                 create_tables=True, ldap_pool_size=10, **kw):
>+
>+        super(MozillaAuth, self).__init__(ldapuri, None, use_tls, bind_user,
>+                                     bind_password, admin_user,
>+                                     admin_password, users_root,
>+                                     users_base_dn, pool_size, pool_recycle,
>+                                     reset_on_return, single_box, ldap_timeout,
>+                                     nodes_scheme, check_account_state,
>+                                     create_tables, ldap_pool_size, **kw)
>+
>+        self.sreg_location = sreg_location
>+        self.sreg_scheme = sreg_scheme
>+        self.sreg_path = sreg_path
>+
>+    def _proxy(self, method, url, data=None):
>+        """Proxies and return the result from the other server.
>+
>+        - scheme: http or https
>+        - netloc: proxy location
>+        """

if data is not None:

>+        if data:

This looks like a no-op. You can do: 

data = urlencode(data.items())


>+            data = urlencode([(k,v) for k,v in data.iteritems()])
>+
>+        status, headers, body = get_url(url, method, data)
>+
>+        if not status == 200:
>+            #logger.error("got status %i from sreg url %s" % (status, body))
>+            raise BackendError()
>+
>+        if body:
>+            return json.loads(body)
>+        return {}
>+
>+    @classmethod
>+    def get_name(self):
>+        """Returns the name of the authentication backend"""
>+        return 'mozilla'
>+
>+    def generate_url(self, username, path=None):

You should use the urlparse in the stdlib. It does this and checks for compliancy

>+        url = "%s://%s/%s/%s" % (self.sreg_scheme, self.sreg_location,
>+                                self.sreg_path, username)
>+        if path:
>+            url = "%s/%s" % (url, path)
>+        return url
>+
>+    def create_user(self, username, password, email):
>+        """Creates a user. Returns True on success."""
>+        payload = {'password': password, 'email': email}
>+        result = self._proxy('PUT', self.generate_url(username), payload)
>+

If result is a Response, you should do

   return result.body == 'success'

>+        return 'success' in result
>+
>+    def generate_reset_code(self, user_id):
>+        """Generates a reset code
>+
>+        Args:
>+            user_id: user id
>+
>+        Returns:
>+            a reset code, or None if the generation failed
>+        """
>+        username = self._get_username(user_id)
>+        result = self._proxy('GET',
>+                        self.generate_url(username, 'reset_code'))
>+        if not result.get('code'):
>+            return False
>+        return result['code']
>+
>+    def verify_reset_code(self, user_id, code):
>+        """Verify a reset code
>+
>+        Args:
>+            user_id: user id
>+            code: reset code
>+
>+        Returns:
>+            True or False
>+        """
>+        if not check_reset_code(code):
>+            return False
>+
>+        username = self._get_username(user_id)
>+        payload = {'reset_code': code}
>+        result = self._proxy('POST',

indentation

>+                          self.generate_url(username, 'verify_password_reset'),
>+                          payload)
>+        return 'success' in result
>+
>+    def clear_reset_code(self, user_id):
>+        """Clears the reset code
>+
>+        Args:
>+            user_id: user id
>+
>+        Returns:
>+            True if the change was successful, False otherwise
>+        """
>+        username = self._get_username(user_id)
>+        result = self._proxy('DELETE',
>+                             self.generate_url(username, 'password_reset'))

See previous comment

>+        return 'success' in result
>+
>+    def get_user_node(self, user_id, assign=True):
>+        if self.single_box:
>+            return None
>+
>+        username = self._get_username(user_id)
>+        dn = self._get_dn(username)
>+
>+        # getting the list of primary nodes
>+        with self._conn() as conn:
>+            try:
>+                res = conn.search_st(dn, ldap.SCOPE_BASE,
>+                                     attrlist=['primaryNode'],
>+                                     timeout=self.ldap_timeout)
>+            except (ldap.TIMEOUT, ldap.SERVER_DOWN, ldap.OTHER), e:
>+                #logger.debug('Could not get the user node in ldap')
>+                raise BackendError(str(e))
>+
>+        res = res[0][1]
>+
>+        for node in res['primaryNode']:
>+            node = node[len('weave:'):]
>+            if node == '':
>+                continue
>+            # we want to return the URL
>+            return '%s://%s/' % (self.nodes_scheme, node)
>+
>+        if not assign:
>+            return None
>+
>+        result = self._proxy('GET', self.generate_url(username, 'node/weave'))
>+        return result.get('node')
>diff -r 1b026734d301 services/tests/test_mozilla.py
>--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
>+++ b/services/tests/test_mozilla.py	Fri Jan 21 10:18:39 2011 -0800
>@@ -0,0 +1,163 @@
>+# ***** BEGIN LICENSE BLOCK *****
>+# Version: MPL 1.1/GPL 2.0/LGPL 2.1
>+#
>+# The contents of this file are subject to the Mozilla Public License Version
>+# 1.1 (the "License"); you may not use this file except in compliance with
>+# the License. You may obtain a copy of the License at
>+# http://www.mozilla.org/MPL/
>+#
>+# Software distributed under the License is distributed on an "AS IS" basis,
>+# WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
>+# for the specific language governing rights and limitations under the
>+# License.
>+#
>+# The Original Code is Sync Server
>+#
>+# The Initial Developer of the Original Code is the Mozilla Foundation.
>+# Portions created by the Initial Developer are Copyright (C) 2010
>+# the Initial Developer. All Rights Reserved.
>+#
>+# Contributor(s):
>+#   Tarek Ziade ([e-mail])
>+#
>+# Alternatively, the contents of this file may be used under the terms of
>+# either the GNU General Public License Version 2 or later (the "GPL"), or
>+# the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
>+# in which case the provisions of the GPL or the LGPL are applicable instead
>+# of those above. If you wish to allow use of your version of this file only
>+# under the terms of either the GPL or the LGPL, and not to allow others to
>+# use your version of this file under the terms of the MPL, indicate your
>+# decision by deleting the provisions above and replace them with the notice
>+# and other provisions required by the GPL or the LGPL. If you do not delete
>+# the provisions above, a recipient may use your version of this file under
>+# the terms of any one of the MPL, the GPL or the LGPL.
>+#
>+# ***** END LICENSE BLOCK *****
>+import unittest
>+import wsgi_intercept
>+from webob import Response
>+from services.auth.mozilla import MozillaAuth
>+from wsgi_intercept.urllib2_intercept import install_opener
>+install_opener()
>+
>+try:
>+    import ldap
>+    from services.auth.ldappool import StateConnector
>+    from services.auth.ldapsql import LDAPAuth
>+    LDAP = True
>+except ImportError:
>+    LDAP = False
>+
>+if LDAP:
>+    # memory ldap connector for the tests
>+
>+    class MemoryStateConnector(StateConnector):
>+
>+        users = {'uid=tarek,ou=users,dc=mozilla':
>+                {'uidNumber': ['1'],
>+                 'uid': ['tarek'],
>+                 'account-enabled': ['Yes'],
>+                 'mail': ['[e-mail]'],
>+                 'cn': ['tarek']},
>+                 'cn=admin,dc=mozilla': {'cn': ['admin'],
>+                 'mail': ['admin'],
>+                 'uid': ['admin'],
>+                 'uidNumber': ['100']}}
>+
>+        def __init__(self):
>+            pass
>+
>+        def simple_bind_s(self, who, *args):
>+            self.connected = True
>+            self.who = who
>+
>+        def search_st(self, dn, *args, **kw):
>+            if dn in self.users:
>+                return [(dn, self.users[dn])]
>+            elif dn in ('ou=users,dc=mozilla', 'dc=mozilla', 'md5'):
>+                key, field = kw['filterstr'][1:-1].split('=')
>+                for dn_, value in self.users.items():
>+                    if value[key][0] != field:
>+                        continue
>+                    return [(dn_, value)]
>+            raise ldap.NO_SUCH_OBJECT
>+
>+        def add_s(self, dn, user):
>+            self.users[dn] = {}
>+            for key, value in user:
>+                if not isinstance(value, list):
>+                    value = [value]
>+                self.users[dn][key] = value
>+            return ldap.RES_ADD, ''
>+
>+        def modify_s(self, dn, user):
>+            if dn in self.users:
>+                for type_, key, value in user:
>+                    if not isinstance(value, list):
>+                        value = [value]
>+                    self.users[dn][key] = value
>+            return ldap.RES_MODIFY, ''
>+
>+        def delete_s(self, dn):
>+            if dn in self.users:
>+                del self.users[dn]
>+            elif dn in ('ou=users,dc=mozilla', 'md5'):
>+                key, field = kw['filterstr'][1:-1].split('=')
>+                for dn_, value in self.users.items():
>+                    if value[key][0] == field:
>+                        del value[key]
>+                        return ldap.RES_DELETE, ''
>+            return ldap.RES_DELETE, ''
>+
>+    from contextlib import contextmanager
>+
>+    @contextmanager
>+    def _conn(self, bind=None, password=None):
>+        yield MemoryStateConnector()
>+
>+    LDAPAuth._conn = _conn
>+
>+# returns a body that has all the responses we need
>+def fake_response():
>+    return Response('{"success": 1, "code": "AAAA-AAAA-AAAA-AAAA"}')
>+
>+# returns a body that has all the responses we need
>+def bad_reset_code_response():
>+    return Response("")
>+
>+class TestLDAPSQLAuth(unittest.TestCase):
>+
>+    def test_mozilla_auth(self):
>+        if not LDAP:
>+            return
>+
>+        wsgi_intercept.add_wsgi_intercept('localhost', 80, fake_response)
>+        auth = MozillaAuth('ldap://localhost',
>+                        'localhost', 'this_path', 'http')
>+
>+        auth.create_user('tarek', 'tarek', '[e-mail]')
>+        uid = auth.get_user_id('tarek')
>+
>+        auth_uid = auth.authenticate_user('tarek', 'tarek')
>+        self.assertEquals(auth_uid, uid)
>+
>+        # reset code APIs
>+        code = auth.generate_reset_code(uid)
>+
>+        wsgi_intercept.add_wsgi_intercept('localhost', 80,
>+                                          bad_reset_code_response)
>+        self.assertFalse(auth.verify_reset_code(uid, 'beh'))
>+        self.assertFalse(auth.verify_reset_code(uid, 'XXXX-XXXX-XXXX-XXXX'))
>+
>+        wsgi_intercept.add_wsgi_intercept('localhost', 80, fake_response)
>+        self.assertTrue(auth.verify_reset_code(uid, code))
>+
>+        auth.clear_reset_code(uid)
>+
>+def test_suite():
>+    suite = unittest.TestSuite()
>+    suite.addTest(unittest.makeSuite(TestLDAPSQLAuth))
>+    return suite
>+
>+if __name__ == "__main__":
>+    unittest.main(defaultTest="test_suite")

Comment 7

[Toby Elliott [:telliott]]

Attachment: Revised files

Comment 8

[Tarek Ziadé (:tarek)]

Comment on attachment 505895 [details] [diff] [review]
Revised files

Looks good.

As discussed on irc, the proxy returns json so you can replace "'success' in result" by "result == 'success'"

Comment 9

[Toby Elliott [:telliott]]

hmm, now I'm confused again. 

the proxy returns {"success": 1}, which gets translated into a dict by the proxy. I don't think your proposed matcher works

Comment 10

[Tarek Ziadé (:tarek)]

is that what the PHP app returns ? If so, it looks like invalid JSON...

If the server return the success string in Json, it's "success" (quotes included)

if it returns a plain string (no quote), then json.loads() will fail on this, thus we'd need to have an option to avoid the conversion.

Comment 11

[Toby Elliott [:telliott]]

the reg server returns success.

the sreg server returns {"success": 1}

Comment 12

[Tarek Ziadé (:tarek)]

ok, not sure why it's different but I guess it's not a big deal :)

Comment 13

[Toby Elliott [:telliott]]

fixed in http://hg.mozilla.org/services/server-core/rev/983e6c3e1794