875461 - Audit MozLDAP's queries

Status: RESOLVED FIXED

(Infrastructure & Operations :: Infrastructure: LDAP, task)

Description

[Peter Bengtsson [:peterbe]]

git clone --recursive https://github.com/mozilla/moz-ldap.git
cd moz-ldap
virtualenv -p python2.6 testingmozldap
source testingmozldap/bin/activate
pip install -r requirements/compiled.txt
cp mozldap/settings/local.py-dist mozldap/settings/local.py
edit mozldap/settings/local.py

The keys to edit are:

1. SECRET_KEY = 'anything'
2. LDAP_SERVER_URI
3. LDAP_BIND_DN  (I've been using the "bind-mozldap" one)
4. LDAP_BIND_PASSWORD

Lastly:

./manage.py runserver

Then go to http://localhost:8000/ and you can test the queries using the form presented.

Comment 1

[Justin Dow [:jabba]]

I can't get this to work on an ubuntu VM that I just set up for this. It only has python 2.7 and 3 though, not sure if that's the issue. Do you maybe have a working VM I can try? I'm not good at debugging playdoh apps.

Comment 2

[Peter Bengtsson [:peterbe]]

It should work in python 2.7.

Comment 3

[Justin Dow [:jabba]]

jabba@ubuntu:~/git/moz-ldap$ ./manage.py runserver 10.0.0.73:8000
Validating models...

0 errors found
Django version 1.4.4, using settings 'mozldap.settings'
Development server is running at http://10.0.0.73:8000/
Quit the server with CONTROL-C.
Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/contrib/staticfiles/handlers.py", line 67, in __call__
    return self.application(environ, start_response)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/core/handlers/wsgi.py", line 241, in __call__
    response = self.get_response(request)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/core/handlers/base.py", line 179, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/views/generic/base.py", line 48, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/utils/decorators.py", line 25, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/jabba/git/moz-ldap/mozldap/base/utils.py", line 14, in wrapper
    response = f(*args, **kw)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/utils/decorators.py", line 21, in bound_func
    return func(self, *args2, **kwargs2)
  File "/home/jabba/git/moz-ldap/mozldap/base/utils.py", line 57, in new_dispatch
    return orig_dispatch(self, request, *args, **kwargs)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/utils/decorators.py", line 25, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/jabba/git/moz-ldap/mozldap/base/utils.py", line 46, in wrapper
    return view(request, *args, **kwargs)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/utils/decorators.py", line 21, in bound_func
    return func(self, *args2, **kwargs2)
  File "/home/jabba/git/moz-ldap/mozldap/base/utils.py", line 57, in new_dispatch
    return orig_dispatch(self, request, *args, **kwargs)
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/views/generic/base.py", line 69, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/jabba/git/moz-ldap/mozldap/base/views.py", line 82, in get
    rs = self.connection.search_s(
  File "/home/jabba/git/moz-ldap/mozldap/base/views.py", line 39, in connection
    for opt, value in settings.LDAP_GLOBAL_OPTIONS.items():
  File "/home/jabba/git/moz-ldap/vendor/lib/python/django/utils/functional.py", line 185, in inner
    return func(self._wrapped, *args)
AttributeError: 'Settings' object has no attribute 'LDAP_GLOBAL_OPTIONS'
[12/Jun/2013 13:13:47] "GET /exists?mail=[e-mail] HTTP/1.1" 500 59

Comment 4

[Peter Bengtsson [:peterbe]]

My bad. I left that out of the instructions. You'll either:

1. `git pull origin master` because this just landed [0]

2. Add `LDAP_GLOBAL_OPTIONS = {}` to your settings/local.py

3. Add these lines to your settings/local.py::

 import ldap
 LDAP_GLOBAL_OPTIONS = {
     ldap.OPT_DEBUG_LEVEL: 4095
 }


[0] https://github.com/mozilla/moz-ldap/commit/c947a6393e38900391778e421b6e3f4f7ff389b2

Comment 5

[Peter Bengtsson [:peterbe]]

Note this new change last week:
https://github.com/mozilla/moz-ldap/commit/b2db8f972629eccce554999b8437f115ae76ae59

Comment 6

[Justin Dow [:jabba]]

I think I found a small bug. When I test myself against the posix_sysadmins group, which is a "normal" posixGroup (as opposed to the "weird" posixGroups that start with scm_), I see this query:

"(&(cn=posix_sysadmins)(|(member=mail=[e-mail],o=com,dc=mozilla)(member=mail=[e-mail],o=org,dc=mozilla)(member=mail=[e-mail],o=net,dc=mozilla)(memberUid=mail=[e-mail],o=com,dc=mozilla)(memberUid=[e-mail])))"

I believe it should instead be:

"(&(cn=posix_sysadmins)(|(member=mail=[e-mail],o=com,dc=mozilla)(member=mail=[e-mail],o=org,dc=mozilla)(member=mail=[e-mail],o=net,dc=mozilla)(memberUid=jdow)(memberUid=[e-mail])))"

And I'm not sure on efficiency, but it might be worth doing some testing to see if we could split the group queries into three seperate ones, based on some rules. If we can detect the objectClasses of the group up front, we have three categories:

1) groupOfNames
2) regular posixGroup
3) posixGroup, but group name starts with "scm_"

Then our queries can be simplified down to:

1) "(&(cn=groupname)(|(member=mail=[e-mail],o=com,dc=mozilla)(member=mail=[e-mail],o=org,dc=mozilla)(member=mail=[e-mail],o=net,dc=mozilla)))" *** (see note below)

2) "(&(cn=groupname)(memberUid=jdow))"

3) "(&(cn=groupname)(memberUid=[e-mail]))"

*** In the example above, an even better query to do, instead of trying all three trees, it would be better if we can just detect the DN of the user instead, so option 1) above would end up as:

"(&(cn=groupname)(member=mail=$DN))"

I'd see if we can split out this logic a little better. Basically if we can detect a user's DN up front and a group's objectClasses up front, then we take out a lot of guesswork and basically eliminate the ability to fail in various edge cases that I can think of.

Can you see if you can rework the logic based on these suggestions? Summary:

First, get user's DN, mail and uid attributes.
Second, get a group's objectClass attributes

Then:

If group's objectclass contains groupOfNames, then query is:
1)"(&(cn=$GROUP)(member=mail=$DN))"

If group's objectClass contains posixGroup and cn=scm_*, then query is:
3) "(&(cn=$GROUPNAME)(memberUid=$MAIL))"

If group's objectClass contains posixGroup and does not start with scm_, then query is:
"(&(cn=$GROUPNAME)(memberUid=$UID))"

It might be possible to wrap all of these into a single LDAP query. Would need some testing. Something with a lot of & and | and !.

(|(&(objectClass=groupOfNames)(cn=$GROUPNAME)(memberUid=$MAIL))(&(obejctClass=posixGroup)(cn=scm_*)(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGroup)(!(cn=scm_*))(cn=$GROUPNAME)(memberUid=$UID))

That last one needs someone to check the logic, but it might just work to take care of all the cases.

Comment 7

[Justin Dow [:jabba]]

(In reply to Justin Dow [:jabba] from comment #6)

> (|(&(objectClass=groupOfNames)(cn=$GROUPNAME)(memberUid=$MAIL))(&(obejctClass
> =posixGroup)(cn=scm_*)(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGr
> oup)(!(cn=scm_*))(cn=$GROUPNAME)(memberUid=$UID))


Might need someone with an affinity for LISP to double check those parentheses as well. :)

Comment 8

[Justin Dow [:jabba]]

:atoll says I'm missing a trailing parenthesis and that one of the "objectClass" is misspelled as obejctClass. So, try:


(|(&(objectClass=groupOfNames)(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGroup)(cn=scm_*)(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGroup)(!(cn=scm_*))(cn=$GROUPNAME)(memberUid=$UID)))

Comment 9

[Justin Dow [:jabba]]

Found another typo. Here's the fixed version:

(|(&(objectClass=groupOfNames)(cn=$GROUPNAME)(member=$DN))(&(objectClass=posixGroup)(cn=scm_*)(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGroup)(!(cn=scm_*))(cn=$GROUPNAME)(memberUid=$UID)))

Comment 10

[Peter Bengtsson [:peterbe]]

First of all, the first bug fixed:
https://github.com/mozilla/moz-ldap/commit/7b59c9d8187e1ab93712bf389b46f47aaf965b89

Now I'm going to look into that optimization.

Comment 11

[Peter Bengtsson [:peterbe]]

Your query from comment 9 worked, except one!
This one now returns false::

 (|(&(objectClass=groupOfNames)(cn=svn_mozilla)(member=mail=[e-mail],o=com,dc=mozilla))(&(objectClass=posixGroup)(cn=scm_*)(cn=svn_mozilla)(memberUid=[e-mail]))(&(objectClass=posixGroup)(!(cn=scm_*))(cn=svn_mozilla)(memberUid=axel)))

Comment 12

[Justin Dow [:jabba]]

Ah interesting. Yeah, it looks like all the svn_* groups are just like the scm_* groups, so we'll need this:

(|(&(objectClass=groupOfNames)(cn=$GROUPNAME)(member=$DN))(&(objectClass=posixGroup)(|(cn=scm_*)(cn=svn_*))(cn=$GROUPNAME)(memberUid=$MAIL))(&(objectClass=posixGroup)(!(|(cn=scm_*)(cn=svn_*))(cn=$GROUPNAME)(memberUid=$UID)))

Basically turning the cn=scm_ filters into OR of cn=scm_ and cn=svn_.

Comment 13

[Peter Bengtsson [:peterbe]]

https://github.com/mozilla/moz-ldap/commit/b6ee769d7a8c8d1a5426aedcc7202af714f49dc6

All tests pass!