1277166 - no or incomplete list of contacts returned for searching LDAP addressbook

Status: RESOLVED DUPLICATE of bug 1277195

(Thunderbird :: Address Book, defect)

Description

[[e-mail]]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20160507231935

Steps to reproduce:

If I enter a search string in the addressbook search bar, I got an result using 38-8-0, but in all releases of 45-... I've tried, the same search on an LDAP addressbook didn't return the correct list. Instead, it returned no list, or an minimal list, i.e. only one or two entries.

Comment 1

[Wayne Mery (:wsmwk)]

what happens if you install 45.1.1 from download of http://getthunderbird.com/

Comment 2

[[e-mail]]

I've trieb all the latest versions starting from 45.0 to 45.1.0, 45.1.1 and so on, even the beta's, nothing worked :( Today I downgraded again to 38.8.0, everything works, except that I'm not using the latest 45'er release...

Comment 3

[:aceman]

Do you use any special characters in the search string? Like brackets, apostrophes, etc.

Do you have the specific LDAP addressbook selected, do are you on the "All Addressbooks" entry in the list?

Comment 4

[[e-mail]]

No, no special characters! I use simple search string, e.g. "al" for "alexander, alex, alan, ...", and I sure do not use special characters.

I do have the addressbook selected, but it returns only a "testapp" entry and "my" entry. However, it seems that in autocompletion of a mail, the autocompletion of addresses from the LDAP still works!! Hm... If I choose autocompletion from LDAP addressbook, I can look for the addresses in the Mail editor, but at the same time, I get no feedback for a search in the addressbook GUI?!? Well, I would appreciate to look for contacts in the addressbook GUI, in order to see additional contact info (like phone number or such).

Comment 5

[Wayne Mery (:wsmwk)]

consider this list http://mzl.la/1WYMdc5

Comment 6

[[e-mail]]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #5)
> consider this list http://mzl.la/1WYMdc5

Sorry for coming back late on this topic. Please, maybe I don't understand well, but could you please tell me what I should "consider" in the mentioned list (where even this bug is listed (infinite loop!))?

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to [e-mail] from comment #6)
> (In reply to Wayne Mery (:wsmwk, NI for questions) from comment #5)
> > consider this list http://mzl.la/1WYMdc5
> 
> Sorry for coming back late on this topic. Please, maybe I don't understand
> well, but could you please tell me what I should "consider" in the mentioned
> list (where even this bug is listed (infinite loop!))?

Sorry for not being clear. I am suggesting you just view the list of bugs to see which, if any, might match your issue.  Perhaps https://mzl.la/2aa4Xof is a better list

Also, in the absence of developers, the quickest way to make progress is continue the great work you did on testing other versions of thunderbird to find which is the first that fails. That involves testing nightly builds, chosen by binary tree, to find a one-day regression range.  It would involve at most 10 downloads. http://mozilla.github.io/mozregression/ is a great tool for doing this.

Comment 8

[Martin Bremer]

I can confirm that I also see this issue. At the moment I am using TB 45.2.0 on a Mac (OSX 10.8.5), I see the same thing on OSX 10.6 and 10.11. On Windows 7 with the latest TB it does work!

I know that this problem exists for several versions of TB, in the past it did work for me as well, so I agree that the only way to find out what changed is to use older versions of TB and check where it failed. I can do that maybe in the coming days.

The interesting thing is that it only fails in the AddressBook window in the search bar. When I open the AddressBook and select in the menu Edit->Search Addresses it does work. Also address completion using my AddressBook when writing an email works fine. The bug is only in the search bar.
 
Since I have access to the LDAP server logs I can actually see what is happening in LDAP:

11/Aug/2016:16:24:07 +0200] conn=120591 op=-1 msgId=-1 - fd=22 slot=22 LDAP connection from 172.17.75.62:50933 to 172.17.75.17
[11/Aug/2016:16:24:07 +0200] conn=120591 op=0 msgId=1 - BIND dn="uid=myusername,ou=People,o=myorganisation" method=128 version=3
[11/Aug/2016:16:24:07 +0200] conn=120591 op=0 msgId=1 - RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=myusername,ou=people,o=myorganisation"
[11/Aug/2016:16:24:07 +0200] conn=120591 op=1 msgId=2 - SRCH base="ou=people,o=myorganisation" scope=2 filter="(&(|(cn=*witte*)(givenName=*witte*)(sn=*witte*)(mozillanickname=*witte*)(mail=*witte*)(mozillasecondemail=*witte*)(&(description=*witte*))(o=*witte*)(ou=*witte*)(title=*witte*)(mozillaworkurl=*witte*)(mozillahomeurl=*witte*)))" attrs="c c description notes modifyTimestamp telephoneNumber title mozillahomelocalityname o company givenName mozillahomestate mail mozillanickname xmozillanickname nsaimid nscpaimscreenname mobile cellphone carphone mozillahomepostalcode birthmonth facsimileTelephoneNumber facsimileTelephoneNumber birthyear ou department departmentNumber orgunit mozillasecondemail xmozillasecondemail postalCode zip mozillacustom1 custom1 mozillahomecountryname homePhone st region mozillacustom2 custom2 pager pagerphone mozillacustom3 custom3 mozillausehtmlmail xmozillausehtmlmail sn sn mozillahomestreet2 birthday street street postOfficeBox mozillacustom4 custom4 mozillahomeurl homeurl l l mozillahomestreet mozillaworkurl workurl labeledUri cn cn mozillaworkstreet2 objectClass"
[11/Aug/2016:16:24:07 +0200] conn=120591 op=1 msgId=2 - RESULT err=0 tag=101 nentries=0 etime=0

As you can see the BIND with my connection details works fine (I've removed my username/organisation details), after the bind it says err=0. This should exclude the possibility that I actually bind anonymously, which would also result in no results due to ACIs set on the search results.

Then it does the search with filter="(&(|(cn=*witte*)(givenName=*witte*)(sn=*witte*)(mozillanickname=*witte*)(mail=*witte*)(mozillasecondemail=*witte*)(&(description=*witte*))(o=*witte*)(ou=*witte*)(title=*witte*)(mozillaworkurl=*witte*)(mozillahomeurl=*witte*)))"

I can use an LDAP browser to do the same search and then it works fine.

The rest of the search line shows which attributes need to be retrieved. Although the list is huge and it contains many duplicates (somebody needs to clean this, but that is a different discussion), I can also request the same long list from an LDAP browser and again it works fine. I compared both search requests in the logs and they look identical. But the one used by the AddressBook search bar gives no result. This is reported in the LDAP logs: nentries=0

So at least we can see that it is LDAP that is not returning the result, the question is why the search fails while an identical search with a different tool does work. I wonder if there is a non-printable character in the search string.

Anyway, I will try to find out in which version this behavior was introduced.

Comment 9

[Martin Bremer]

I've tested earlier versions of TB and in 9 iterations I found out that this functionality was broken in 45.0. It still worked in 44.0b1.

Comment 10

[eugene.tsuno]

This is related to 
https://bugzilla.mozilla.org/show_bug.cgi?id=1277195

The aFilter for ldap searches was simpler, then somehow the filter for the local address book was used for  non-local queries.

Here is the snippet from someone's testing:

38.8.0:
aFilter = '(&(|(mail=*crock*)(cn=*crock*)(givenName=*crock*)(sn=*crock*)))';

45.1.1
aFilter = '(&(|(cn=*crock*)(givenName=*crock*)(sn=*crock*)(mozillaNickname=*crock*)(mail=*crock*)(mozillaSecondEmail=*crock*)(&(description=*crock*))(o=*crock*)(ou=*crock*)(title=*crock*)(mozillaWorkUrl=*crock*)(mozillaHomeUrl=*crock*)))';

I get the same aFilter on 45.2.

So depending on how your ldap server responds, that extra mozilla entries can make it fail to respond.  On mine, it takes other referrals and slows it down to the point it is unusable in real time, it takes a few minutes to populate the entries.

Comment 11

[Martin Bremer]

After giving a full report what is happening, this bug is still set to UNCONFIRMED after a year. I would be nice if someone would start working on this.