401955 - LDAP directory search in address book always returns zero matches

Status: RESOLVED FIXED

(MailNews Core :: Address Book, defect)

Description

[Andy Boze]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a9pre) Gecko/2007103002 SeaMonkey/2.0a1pre equivalent to Firefox/2.0.0.8
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a9pre) Gecko/2007103002 SeaMonkey/2.0a1pre

I have my University LDAP directory configured in the Address Book. I'm not sure for how long now, but it consistently return zero results for every search I do. It did return results at one time, but it seems to have broken several weeks ago. When addressing an e-mail, though, the look-up works when I type a name in the address area. If I click the address button in the mail compose window toolbar and try to search the LDAP directory, I get no search results, the same as in the address book.

Reproducible: Always

Steps to Reproduce:
1.Configure the Address Book with and LDAP directory.
2.Search for a name in the LDAP directory.
3.
Actual Results:  
There are never any matches found.

Expected Results:  
If a name exists in the LDAP directory, one or more matches should be returned.

Comment 1

[Mark Banner (:standard8)]

Could you try a nightly build from the 21st or 22nd? This could be related to bug 401847, though admittedly it appears to be different on first inspection.

Comment 2

[Andy Boze]

I still have several old installers from a month ago. The best I can tell you at this point is that the address book search worked in the Sept 28 nightly and did not work in the October 4 nightly and later. I can download some of the other nightlies in that week range if it would help to pinpoint the exact nightly.

Comment 3

[Mark Banner (:standard8)]

(In reply to comment #2)
> I still have several old installers from a month ago. The best I can tell you
> at this point is that the address book search worked in the Sept 28 nightly and
> did not work in the October 4 nightly and later. I can download some of the
> other nightlies in that week range if it would help to pinpoint the exact
> nightly.

I don't need you to do that at the moment - there are several bugs committed in that time frame, but having just re-read your description, I probably just need to test this again.

Let me just confirm though:

- Addressing a message from LDAP by typing it into the address bar works
- Using an LDAP directory doesn't work from "Select Addresses"
- Using an LDAP directory doesn't work from the Address Book window

Are you using LDAP authentication e.g. bind DN/password?

Comment 4

[Andy Boze]

(In reply to comment #3)
> Let me just confirm though:
> 
> - Addressing a message from LDAP by typing it into the address bar works
Correct

> - Using an LDAP directory doesn't work from "Select Addresses"
Correct

> - Using an LDAP directory doesn't work from the Address Book window
Correct

> 
> Are you using LDAP authentication e.g. bind DN/password?

No. However I just tried setting up the Bind DN plus SSL and I get the same results. (I didn't realize our directory server even supported that.)

Comment 5

[Mark Banner (:standard8)]

(In reply to comment #4)
> No. However I just tried setting up the Bind DN plus SSL and I get the same
> results. (I didn't realize our directory server even supported that.)

So you're using SSL in both cases? In theory we shouldn't have affected that, but you never know.

Could you try two things?

Firstly, before you search, open up the Error Console (Tools->Web Development), select Clear, then do the search and see if any errors are displayed.

Secondly, take a look at this page:

http://wiki.mozilla.org/MailNews:LDAP_Address_Books#LDAP_Logging

If you could turn on LDAP logging for a build, and attach a log of attempting to search to this bug it would be useful (mangling hostnames etc is acceptable).

Comment 6

[Andy Boze]

(In reply to comment #5)
> So you're using SSL in both cases? In theory we shouldn't have affected that,
> but you never know.

Sorry I wasn't clear. No, I don't usually use SSL. I just enabled it when I added the Bind DN.
> 
> Could you try two things?
> 
> Firstly, before you search, open up the Error Console (Tools->Web Development),
> select Clear, then do the search and see if any errors are displayed.

No errors.
> 
> Secondly, take a look at this page:
> 
> http://wiki.mozilla.org/MailNews:LDAP_Address_Books#LDAP_Logging
> 
> If you could turn on LDAP logging for a build, and attach a log of attempting
> to search to this bug it would be useful (mangling hostnames etc is
> acceptable).
> 
I enabled the logging and I'll attach the log file. I did three LDAP searches. The first two were from the Address Book on the names "boze" and "sill", and both those searches failed to return matches. The third search was from the message compose window using address autocomplete on the name "boze", and it succeeded. 

If I'm interpreting the log correctly, it looks like there's a difference in the way the searches are constructed.

Comment 7

[Andy Boze]

Attachment: LDAP log file

Comment 8

[Mark Banner (:standard8)]

(In reply to comment #6)
> I enabled the logging and I'll attach the log file. I did three LDAP searches.
> The first two were from the Address Book on the names "boze" and "sill", and
> both those searches failed to return matches. The third search was from the
> message compose window using address autocomplete on the name "boze", and it
> succeeded. 

Thanks for that.

> If I'm interpreting the log correctly, it looks like there's a difference in
> the way the searches are constructed.

Yes there is a difference in the searches, but that hasn't changed recently and I don't see why it would vary between those two builds in comment 2 if its the search that is the problem.

The interesting thing is that we're getting an LDAP_DECODING_ERROR, which is strange, but again, I don't think anything has changed things in that area in recent builds which would affect that.

I'm cc'ing some people who might have a better idea about what is going on than I do at the moment.

Comment 9

[Anton Bobrov]

Mark, i just tried it with the latest nightly and got the same result [ no
result ] with adddressbook. i can see that the search actually works and
all matching entries being returned however the problem as indicated in
the log has something to do with attributes you are trying to get the
values for. i cant get a hold of debug build and strangely enough nightly
builds have debug disabled and since it takes awhile to build one from
scratch locally for me can you point where to get debug enabled build/s ?
what i think should help by having debug build is this log line here :
http://lxr.mozilla.org/mozilla/source/directory/xpcom/base/src/nsLDAPMessage.cpp#486
so we can see which attributes its looking for, there is a good chance to
see some garbage or non existent attributes in this case, judging by
DECODING_ERROR being returned constantly for each search entry returned.

Comment 10

[Mark Banner (:standard8)]

Attachment: Temporary Enable the debug line

ok, here's the plan:

1) Land this patch
2) Let Mac cycle once or twice. Anton grabs a build from ftp://ftp.mozilla.org/pub/seamonkey/tinderbox-builds/cb-xserve02-trunk
3) Once Anton has a build, I back the patch out.

David - would you be happy with that?

Comment 11

[David :Bienvenu]

Comment on attachment 287413 [details] [diff] [review]
Temporary Enable the debug line

sure, that's fine.

I'd also be OK with temporarily enabling PR_Logging in release builds in general.

Comment 12

[Mark Banner (:standard8)]

ok, I've landed the patch. I'll put another note once mac has cycled and done a build. (and its actually ftp://ftp.mozilla.org/pub/thunderbird/tinderbox-builds/bm-xserve09-trunk/ that you want to watch).

Comment 13

[Mark Banner (:standard8)]

(In reply to comment #12)
> ok, I've landed the patch. I'll put another note once mac has cycled and done a
> build. (and its actually
> ftp://ftp.mozilla.org/pub/thunderbird/tinderbox-builds/bm-xserve09-trunk/ that
> you want to watch).

Anton, the build has cycled and pushed out a new version - can you download & give it a try please? (let me know if its working and I'll back the patch out again).

Comment 14

[Anton Bobrov]

yep, got it and it works, sent you email about that already. you can
back it off now and i will update this bug with my findings later on.

Comment 15

[Mark Banner (:standard8)]

(In reply to comment #14)
> yep, got it and it works, sent you email about that already. you can
> back it off now and i will update this bug with my findings later on.
> 
This is now backed out. Thanks Anton.

Comment 16

[Anton Bobrov]

it looks like the problem is caused by
http://lxr.mozilla.org/mozilla/source/mailnews/addrbook/src/nsAbLDAPCard.cpp#306
or similar invocation [dunno for sure, havent ran it under debugger yet] which
has objectclass attribute hardcoded and it has to be present in the search entry
to ensure successful code flow in sucha case. in the addressbook search however
nothing requests objectclass attribute to be returned, heres how addressbook 
search request looks like on the server side :

- SRCH base="ou=people,dc=mcom,dc=com" scope=2 filter="(|(mail=*jsmith*)(cn=*jsmith*)(givenName=*jsmith*)(sn=*jsmith*))" attrs="o company mail mozillacustom3 custom3 mozillausehtmlmail xmozillausehtmlmail mozillacustom2 custom2 mozillahomecountryname ou department departmentNumber orgunit mobile cellphone carphone telephoneNumber title mozillacustom1 custom1 mozillanickname xmozillanickname pager pagerphone mozillaworkurl workurl facsimileTelephoneNumber facsimileTelephoneNumber mozillasecondemail xmozillasecondemail mozillacustom4 custom4 nsaimid nscpaimscreenname street street postOfficeBox l l mozillahomeurl homeurl mozillahomestreet givenName mozillahomepostalcode mozillahomelocalityname birthyear mozillaworkstreet2 mozillahomestreet2 postalCode zip homePhone c c sn sn mozillahomestate st region description notes modifyTimestamp cn cn"

and this is what such request gets back for some sample entry here :

uid=JSmith,ou=People, dc=mcom,dc=com
mail=[e-mail]
telephoneNumber=911999222
facsimileTelephoneNumber=911999111
givenName=John
sn=Smith
modifyTimestamp=20051110143619Z
cn=John Smith

things under directory/xpcom seem to handle that well because they first
fetch attributes returned by the search and only then fetch their values
so they could never try to fetch a value for non existing search attr,
see :

http://lxr.mozilla.org/mozilla/source/directory/xpcom/base/src/nsLDAPChannel.cpp#859
http://lxr.mozilla.org/mozilla/source/directory/xpcom/base/src/nsLDAPMessage.cpp#331
http://lxr.mozilla.org/mozilla/source/directory/xpcom/base/src/nsLDAPMessage.cpp#480

which on LDAP C SDK side of things maps to ldap_first/next_attribute and
only then followed by ldap_get_values.

Comment 17

[Jeremy Lainé]

Ouch, in that case I'm to blame for the regression as I introduced the code you mentioned in nsAbLDAPCard.

As an extra test to confirm that the problem is related to the objectClass attribute, you could patch nsAbLDAPAttributeMap.js like this (nice thing is it doesn't require being able to build any code):

RCS file: /cvsroot/mozilla/mailnews/addrbook/src/nsAbLDAPAttributeMap.js,v
retrieving revision 1.5
diff -u -8 -r1.5 nsAbLDAPAttributeMap.js
--- nsAbLDAPAttributeMap.js     30 Sep 2007 06:53:38 -0000      1.5
+++ nsAbLDAPAttributeMap.js     6 Nov 2007 06:03:05 -0000
@@ -124,16 +124,18 @@
     for each (var prop in this.mPropertyMap) {
       attrs.push(prop);
     }

     if (!attrs.length) {
       throw Components.results.NS_ERROR_FAILURE;
     }

+    attrs.push('objectClass');
+
     return attrs.join(",");
   },

   getAllCardProperties: function getAllCardProperties(aCount) {

     var props = [];
     for (var prop in this.mPropertyMap) {
       props.push(prop);

Comment 18

[Jeremy Lainé]

Attachment: Request objectClass attribute in LDAP searches

Assuming the problem is indeed the fact that the objectClass is not being requested/returned, attached is a patch that should fix the problem. 

Some notes:

* I think it's appropriate to add objectClass in nsAbLDAPDirectoryQuery (rather than nsAbLDAPAttributeMap), because that's where the call to nsAbLDAPCard::SetMetaProperties is made. I don't think patching nsAbLDAPAttributeMap::getAllCardProperties would be appropriate as objectClass is not mapped to any card property. Comments welcome.

* Appending ",objectClass" (note the comma) to returnAttributes should always be OK, as the LDAP attribute mapper cannot return an empty string:

http://lxr.mozilla.org/mailnews/source/mailnews/addrbook/src/nsAbLDAPAttributeMap.js#128 

* Should we have a more explicit error message in nsAbLDAPCard::SetMetaProperties if the objectClass is missing?

Comment 19

[Anton Bobrov]

i can confirm that nsAbLDAPAttributeMap.js one-liner works. as for requesting
the objectClass attribute it is safe to assume it is always there since each
ldap entry must have one.

Comment 20

[Mark Banner (:standard8)]

Comment on attachment 287512 [details] [diff] [review]
Request objectClass attribute in LDAP searches

Yep, nsAbLDAPDirectoryQuery will do for now. We can always move it if we find some other reason.

Comment 21

[Mark Banner (:standard8)]

I've just checked this in:

Checking in mailnews/addrbook/src/nsAbLDAPDirectoryQuery.cpp;
/cvsroot/mozilla/mailnews/addrbook/src/nsAbLDAPDirectoryQuery.cpp,v  <--  nsAbLDAPDirectoryQuery.cpp
new revision: 1.51; previous revision: 1.50

Thanks for the patch Jeremy and to Anton for finding the problem.

Comment 22

[Joe Sabash [:JoeS1]]

Not knowing what modules are common and which aren't,
Does this checkin pertain to Thunderbird also ?

Comment 23

[Mark Banner (:standard8)]

(In reply to comment #22)
> Not knowing what modules are common and which aren't,
> Does this checkin pertain to Thunderbird also ?

Yes it does, sorry I should have moved it to be in Core (Mailnews: Address Book) as it covers both - I'll do that now.