507712 - LDAP searches from address book are poorly formed and cause performance issues

Status: NEW

(MailNews Core :: LDAP Integration, defect)

Description

[John Egan]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Build Identifier: 2.0.0.2.1

When you type a character string into a new email directly from Thunderbird, it executes the following query against LDAP:

BEGIN
2009/07/31:14:15:03 * ServerWorker (REG):7
  ConnID:280926 * mesgID:2 * OpID:1 * OpName:search         
  ConnIP:141.144.177.98 ConnDN: Anonymous
 INFO :gslfseADoSearch BASE = dc=oracle, dc=com FILTER = (|(cn=testresourcex*)(mail=testresourcex*)(sn=testresourcex*)) #REQDATTR = 2 scope =2
14:15:03 * INFO : gsleswrASndResult2 RESULT = 0 nentries=1 
END

This is a very reasonable "BEGINS WITH" search (i.e. the asterik wildcard comes AFTER the string.

However, when you go directly into the Address Book and query against your LDAP director from there, you get:

BEGIN
2009/07/31:14:13:12 * ServerWorker (REG):7
  ConnID:280906 * mesgID:4 * OpID:3 * OpName:search         
  ConnIP:141.144.177.98 ConnDN: Anonymous
 INFO :gslfseADoSearch BASE = dc=oracle, dc=com FILTER = (|(mail=*testresourcex*)(cn=*testresourcex*)(givenName=*testresourcex*)(sn=*testresourcex*)) #REQDATTR = 64 scope =2
14:13:14 * INFO : gsleswrASndResult2 RESULT = 0 nentries=1 
END


With a very inefficient CONTAINS search.

Why this different behavior?

Against Oracle Intenet Directory (the LDAP server that we have deployed) this CONTAINS search causes extra, unnecessary work on the database.  In a company with well over 100K user accounts, this search can take a very long time to come back and overall slowness of the database.

Netscape had this same limitation.  Outlook gives the end user the option to choose the scope of his/her search but default to BEGINS WITH searches, which are ideal.

Reproducible: Always

Steps to Reproduce:
1.  Go to your Address Book from Thunderbird (Tool/Address Book or use CNTRL+2)
2.  If you're using an LDAP directory, select this from your left panel.
3.  In the search box, enter your search critera.
4.  View your LDAP logs and see the CONTAINS search.
Actual Results:  
These steps cause Thunderbird to execute a poorly-formed CONTAINS search against the LDAP server, which causes at best longer time to return a result to the end user and at worse causes an overall slowdown of the DB.

Expected Results:  
The query should be the same in address book as it is in a normal email:

BEGIN
2009/07/31:14:15:03 * ServerWorker (REG):7
  ConnID:280926 * mesgID:2 * OpID:1 * OpName:search         
  ConnIP:141.144.177.98 ConnDN: Anonymous
 INFO :gslfseADoSearch BASE = dc=oracle, dc=com FILTER = (|(cn=testresourcex*)(mail=testresourcex*)(sn=testresourcex*)) #REQDATTR = 2 scope =2
14:15:03 * INFO : gsleswrASndResult2 RESULT = 0 nentries=1 
END

Comment 1

[Nikolay Shopik]

Same here
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2pre) Gecko/20090727 Shredder/3.0b4pre

Comment 3

[John Lancaster]

This is also a problem for me with 100K+ user in AD "(|(mail=*testresourcex*)(cn=*testresourcex*)(givenName=*testresourcex*)(sn=*testresourcex*))" style quires are an issue.

I would be happy if the address book used an advanced preference like "ldap_2.servers.MYSERVER.autoComplete.filterTemplate" maybe it could be called "ldap_2.servers.MYSERVER.addressbook.filterTemplate"

Comment 4

[:aceman]

The AB search on LDAP supposedly uses mail.addr_book.quicksearchquery.format query (just translates fields to LDAP using some *.attrmap.* prefs). Can you try that out?

Comment 5

[John Egan]

I'm not sure what you're asking me to try out.  Can you be more specific?

Comment 6

[:aceman]

As comment 3 said, there is a preference to control how the autocomplete query looks like for the LDAP server (which fields it should look into).

I just added that instead of adding a ldap_2.servers.MYSERVER.addressbook.filterTemplate pref, there already exists mail.addr_book.quicksearchquery.format that you can tweak.

If you do not like that the queries have CONTAINS operators, you can try to change it in those preferences.