905880 - LDAP Queries failing for registration on python26-syncreg-1.0-3 on SL6.2

Status: RESOLVED FIXED

(Cloud Services :: Server: Registration, defect)

Description

[Bob Micheletto [:bobm]]

LDAP Queries are failing for registration server python26-syncreg-1.0-3 running under SL6.2, but working correctly on Centos 5.5 with an identical configuration.

Comment 1

[Bob Micheletto [:bobm]]

Attachment: packages.diff

There are some key differences between the relevant package versions on these servers that may explain the issue.

Comment 2

[Bob Micheletto [:bobm]]

Traceback example from the /var/log/syncreg.log file:
2013-08-14 13:02:21,450 ERROR [syncserver] 542d0f1862d622cf462bf29f636bb92a
2013-08-14 13:02:21,451 ERROR [syncserver] Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/services/util.py", line 304, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/paste/translogger.py", line 68, in __call__
    return self.application(environ, replacement_start_response)
  File "/usr/lib/python2.6/site-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/services/baseapp.py", line 187, in __notified
    response = func(self, request)
  File "/usr/lib/python2.6/site-packages/services/baseapp.py", line 221, in __call__
    response = self._dispatch_request(request)
  File "/usr/lib/python2.6/site-packages/services/baseapp.py", line 273, in _dispatch_request
    response = self._dispatch_request_with_match(request, match)
  File "/usr/lib/python2.6/site-packages/services/baseapp.py", line 299, in _dispatch_request_with_match
    result = function(request, **params)
  File "/usr/lib/python2.6/site-packages/syncreg/controllers/user.py", line 96, in user_exists
    uid = self.auth.get_user_id(request.user)
  File "/usr/lib/python2.6/site-packages/services/user/mozilla_ldap.py", line 75, in get_user_id
    self._get_dn(user)
  File "/usr/lib/python2.6/site-packages/services/user/mozilla_ldap.py", line 354, in _get_dn
    with self._conn() as conn:
  File "/usr/lib64/python2.6/contextlib.py", line 16, in __enter__
    return self.gen.next()
  File "/usr/lib/python2.6/site-packages/services/ldappool.py", line 294, in connection
    conn = self._get_connection(bind, passwd)
  File "/usr/lib/python2.6/site-packages/services/ldappool.py", line 247, in _get_connection
    conn = self._create_connector(bind, passwd)
  File "/usr/lib/python2.6/site-packages/services/ldappool.py", line 227, in _create_connector
    self._bind(conn, bind, passwd)
  File "/usr/lib/python2.6/site-packages/services/ldappool.py", line 184, in _bind
    conn.simple_bind_s(bind, passwd)
  File "/usr/lib/python2.6/site-packages/services/ldappool.py", line 82, in simple_bind_s
    clientctrls)
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 781, in simple_bind_s
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 207, in simple_bind_s
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 422, in result
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 426, in result2
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 432, in result3
  File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 96, in _ldap_call
INVALID_CREDENTIALS: {'info': '', 'desc': 'Invalid credentials'}

Comment 3

[Ryan Kelly [:rfkelly]]

Comment on attachment 791056 [details] [diff] [review]
packages.diff

So they're both using the same version of python-ldap.  IIRC this is an RPM we build ourselves, correct?  If so, what configuration is the build machine and which of web2 or web3 does it more closely resemble?

"python26-1.0" seems like a very strange package name, I'd expect something more like "python26-2.6.2" as seen in the web2 package list.

What about differences in the system-level LDAP libraries?  It could be some binding issue between the ldap lib and the system lib.

Comment 4

[Ryan Kelly [:rfkelly]]

Latest release of python-ldap package is 2.4.13, we should try upgrading to that as a first step (requires building against OpenLDAP 2.4.11 or greater)

Comment 5

[Bob Micheletto [:bobm]]

LDAP Packages on the current hosts: (Note they are Centos 5)
openldap-clients-2.3.43-12.el5_6.7
nss_ldap-253-25.el5
nss_ldap-253-25.el5
php-ldap-5.2.9-2.rhel5
openldap-2.3.43-12.el5_6.7
python26-ldap-2.3.13pre-1moz
openldap-2.3.43-12.el5_6.7
openldap-devel-2.3.43-12.el5_6.7

LDAP Packages on the replacement hosts: (Note: Scientific Linux 6.2)
openldap-2.4.23-26.el6_3.2.x86_64
openldap-devel-2.4.23-26.el6_3.2.x86_64
pam_ldap-185-11.el6.x86_64
compat-openldap-2.3.43-2.el6.x86_64
nss-pam-ldapd-0.7.5-14.el6.x86_64
python26-ldap-2.3.13pre-1moz.x86_64
openldap-clients-2.4.23-26.el6_3.2.x86_64

Comment 6

[Bob Micheletto [:bobm]]

Note: ldap queries from ldapsearch do not work from the current hosts, but do from the replacement hosts.

Current Host:
ldapsearch -LLL -D uid=USERNAME,ou=logins,dc=mozilla -w PASSWORD -H ldap://master.ldap.phx1.svc.mozilla.com -b 'dc=mozilla' '(mail=[e-mail])'
ldap_sasl_interactive_bind_s: No such attribute (16)

Replacement Host:
ldapsearch -LLL -D uid=USERNAME,ou=logins,dc=mozilla -w PASSWORD -H ldap://maste
r.ldap.phx1.svc.mozilla.com -b 'dc=mozilla' '(mail=[e-mail])'
dn: uidNumber=XXXX, etc.
etc.

Comment 7

[Bob Micheletto [:bobm]]

Running this script works from both the current and replacement hosts:
import ldap
from services.ldappool import ConnectionManager

# Get these from the config file
ldapuri = "ldap://master.ldap.phx1.svc.mozilla.com"
bind = "uid=USER,ou=logins,dc=mozilla"
passwd = "PASSWORD"

dn = 'cn=maxuid,ou=users,dc=mozilla'
c = ConnectionManager(ldapuri, bind=bind, passwd=passwd)

     with c.connection() as conn:
      print conn.search_st(dn, ldap.SCOPE_BASE)


Current Host:
[('cn=maxuid,ou=users,dc=mozilla', {'uidNumber': ['NUMBER'], 'cn': ['maxuid']})]

Replacement Host:
[('cn=maxuid,ou=users,dc=mozilla', {'uidNumber': ['NUMBER'], 'cn': ['maxuid']})]

Comment 8

[Ryan Kelly [:rfkelly]]

Wait, is "current host" the one that is working and "replacement host" the one that is not, or is it the other way around?  Or something even more complicated than that? :-)

Comment 9

[Ryan Kelly [:rfkelly]]

Looks like the new hosts have a fresh enough version of openldap, that we could try building python-ldap 2.4.13 on them and see if it makes a different.

Comment 10

[Bob Micheletto [:bobm]]

(In reply to Ryan Kelly [:rfkelly] from comment #8)
> Wait, is "current host" the one that is working and "replacement host" the
> one that is not, or is it the other way around?  Or something even more
> complicated than that? :-)

That's correct.

Comment 11

[Bob Micheletto [:bobm]]

Puppet was not running on these servers, Bug 908036 has been opened to resolve that issue.  The credentials were not populated by Puppet, but by hand and were almost but not quite right.  Once set to the proper credential testing an account worked.

[THISISYOURSHELLPROMPTSPEAKING]$ curl http://web3.reg/user/1.0/USER
1
[THISISYOURSHELLPROMPTSPEAKING]$ curl http://web4.reg/user/1.0/USER
1
[THISISYOURSHELLPROMPTSPEAKING]$ curl http://web5.reg/user/1.0/USER
1

Comment 12

[Ryan Kelly [:rfkelly]]

Comment on attachment 791056 [details] [diff] [review]
packages.diff

clearing f? flag since bug is resolved