669917 - LDAP phonebook search for IRC nick should be case insensitive

Status: RESOLVED FIXED

(Infrastructure & Operations :: Infrastructure: LDAP, task)

Description

[Kumar McMillan [:kumar]]

To reproduce: search for boozeniges if you want to find out whose IRC nick it is.  This search should be case insensitive but it's not:

https://ldap.mozilla.org/phonebook/#search/boozeniges (no results)
https://ldap.mozilla.org/phonebook/#search/Boozeniges (results!)

Small issue but *might* be easy to fix, I don't know.  I know the community phonebook is the works so feel free to punt on this.

Comment 1

[:Atoll]

:rtucker, is this still an issue?

Comment 2

[Justin Dow [:jabba]]

This is still an issue, and the problem is in LDAP. I've tried to figure out why it's not case insensitive. This is the "im" attribute in LDAP, which has the following schema:

attributetype ( 1.3.6.1.4.1.13769.1.1.7 NAME 'im'
        DESC 'Various IM logins'
        EQUALITY caseExactIA5Match
        SUBSTR caseIgnoreIA5SubstringsMatch
        SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )

I *think* the SUBSTR caseIgnoreIA5SubstringsMatch bit should make it case insensitive, but perhaps the EQUALITY caseExactIA5Match conflicts with that? We should research and see if we can make it case insensitive.

Comment 3

[Brian Hourigan [:digi]]

I looked into rfc 4517 and I think the appropriate EQUALITY attribute is caseIgnoreIA5Match. [1] Further to that we may want to look into caseIgnoreIA5SubstringsMatch [1], due to a bug identified in 707209. Although it's not clear if that is a bug in phonebook itself, or the ldap schema.

1. http://tools.ietf.org/html/rfc4517#page-30

Comment 4

[Justin Dow [:jabba]]

It's a bug in LDAP, phonebook just passes an LDAP search query back directly to LDAP.

So, if we update the EQUALITY line of that schema entry to caseIgnoreIA5Match, this should work? I can test on dev.

Comment 5

[Justin Dow [:jabba]]

So far no luck. My test cases here are:

[e-mail]1:~> ldapsearch -x -D "mail=[e-mail],o=com,dc=mozilla" -W -h 127.0.0.1 -b "dc=mozilla" '(im=*boozeniges*)' im
(this returns nothing, but we want it to)

and

[e-mail]1:~> ldapsearch -x -D "mail=[e-mail],o=com,dc=mozilla" -W -h 127.0.0.1 -b "dc=mozilla" '(im=*Boozeniges*)' im
(this returns the user properly)

When I changed the schema's EQUALITY attribute from caseExactIA5Match to caseIgnoreIA5Match, it did not help, in fact both test cases no longer worked anymore. I don't understand why this dosn't work, since we are searching a substring (*<search term>*) because of the wildcards, unless I'm misunderstanding what is meant by substring and equality here.

Note that the search in phonebook app just uses this same syntax with the asterisks on either side of the user-input search term, so I think if we can solve it form the command line, it will be fixed in phonebook.

Comment 6

[Justin Dow [:jabba]]

We are making progress on this. We've narrowed it down to an issue with OpenLDAP's indexing. If I remove the index for the 'im' attribute, everything works.

Comment 7

[Justin Dow [:jabba]]

After some more research and testing, I think the following patch to our slapd.conf file will resolve this issue:

-index uid,memberUid,im                      eq,pres,sub
+index uid,memberUid                         eq,pres,sub

This will remove the "im" attribute from being indexed. The index doesn't seem to be particularly useful in the first place, particularly not in this context ( see the Index portion of http://www.openldap.org/doc/admin24/tuning.html - We definitely aren't querying that particular search term, so this index is useless). The index itself seems to not understand case insensitivity, so removing it altogether should resolve this issue. I think setting EQUALITY to caseIgnoreIA5Match could be useful as well, but not necessarily required, since I believe we only ever do substring searches with phonebook (the only app using this attribute), and we already have caseIgnoreIA5SubstringsMatch specified, so the EQUALITY part doesn't matter for this case. Changing it won't hurt anything either.

I'm PTO next two weeks, but someone can push this change in my absence. I've tested on dev.

Comment 8

[Justin Dow [:jabba]]

I've pushed this change, but it looks like it won't take effect until the backend Berkeley DB is rebuilt from scratch. I'm filing a change request window bug to do this.

Comment 9

[Justin Dow [:jabba]]

Ok, this has been completed. I rebuilt the database on the LDAP master after doing both the index change and the schema change (caseExactIA5Match -> caseIgnoreIA5Match). We now have case insensitive search for nicks. Boozeniges is no longer here, but you can test/verify by searching for jabba and JABBA or JaBbA, and should get same results each time.