1730874 - LDAP regression: search using userAccountControl Filter gets empty results

Status: RESOLVED FIXED

(MailNews Core :: Address Book, defect)

Description

[Christian Fertig]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

For two decades (even in the old mozilla before thunderbird) we're using this search filter in the advanced LDAP settings:

(&(objectclass=User)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

to display only activated (not deactivated) users

Actual results:

since TB 91 (in fact TB91.0.3) it is not working anymore.

(objectclass=User) alone is working

(!(userAccountControl:1.2.840.113556.1.4.803:=2)) alone is not working

(userAccountControl:1.2.840.113556.1.4.803:=65536)) some example from
https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx?Sort=MostRecent
just for testing is NOT working

in my Opinion it must be something with the syntax of this query (bitwise ANDs or something like that)

I was not able to receive a log as decribed here: https://wiki.mozilla.org/MailNews:Logging
Logfile for LDAP stays empty

Comment 1

[Geoff Lankow (:darktrojan)]

We rebuilt the LDAP code in this version. I'll pass this bug on to the developer in charge of that. In the meantime you can still use the old version by flipping the pref mailnews.ldap.jsmodule.

Comment 2

[Ping Chen [:rnons] (away)]

Attachment: Bug 1730874 - Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan

An extensibleMatch filter is in the form of <type>:dn:<rule>:=<value>.

Comment 3

[Ping Chen [:rnons] (away)]

Thanks for reporting. Can you help me test the fix by downloading an artifact build from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/QUIrjhD7TJSP83rkIDfJ-Q/runs/0/artifacts/public/build/target.zip? It's built here https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=03847779c7a81ff6bf9954497f4dcde260d863c5

Comment 4

[Christian Fertig]

I've tested against your daily build 94.0a1 and I can confirm, that it's working again :)

Comment 5

[Christian Fertig]

(In reply to Geoff Lankow (:darktrojan) from comment #1)

you can still use the old version by flipping the pref mailnews.ldap.jsmodule.

this is working fine for me, a good workaround waiting for TB94. Thanks

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f72fa1a50e19
Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan

Comment 7

[Ping Chen [:rnons] (away)]

Comment on attachment 9241496 [details]
Bug 1730874 - Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan

[Approval Request Comment]
Regression caused by (bug #): bug 1696625
User impact if declined: extensibleMatch filter returns no result
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): low

Comment 8

[Wayne Mery (:wsmwk)]

Comment on attachment 9241496 [details]
Bug 1730874 - Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan

[Triage Comment]
Approved for beta

Comment 9

[Rob Lemley [:rjl]]

Thunderbird 93.0b5:
https://hg.mozilla.org/releases/comm-beta/rev/30b960df1f29

Comment 10

[Wayne Mery (:wsmwk)]

Comment on attachment 9241496 [details]
Bug 1730874 - Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan

[Triage Comment]
Approved for esr91

Comment 11

[Rob Lemley [:rjl]]

Thunderbird 91.2.0:
https://hg.mozilla.org/releases/comm-esr91/rev/eafbf6e20976

Comment 12

[muncubus]

I have changed mailnews.ldap.jsmodule to false in 91.6.
(&(sAMAccountType=805306368)(!useraccountcontrol:1.2.840.113556.1.4.803:=2)) still not working.
"Support extensibleMatch filter in LDAPMessage.jsm. r=darktrojan"- I don't understand what to with this

Comment 13

[Ping Chen [:rnons] (away)]

Hi muncubus, according to https://datatracker.ietf.org/doc/html/rfc4515#section-3, a valid LDAP filter follows this pattern

filter         = LPAREN filtercomp RPAREN
not            = EXCLAMATION filter

This means (!useraccountcontrol:1.2.840.113556.1.4.803:=2) is invalid, but (!(useraccountcontrol:1.2.840.113556.1.4.803:=2)) is valid. Do you mind changing the filter string and try again? Thanks

Comment 14

[muncubus]

(In reply to Ping Chen (:rnons) from comment #13)

Hi muncubus, according to https://datatracker.ietf.org/doc/html/rfc4515#section-3, a valid LDAP filter follows this pattern

filter         = LPAREN filtercomp RPAREN
not            = EXCLAMATION filter

This means (!useraccountcontrol:1.2.840.113556.1.4.803:=2) is invalid, but (!(useraccountcontrol:1.2.840.113556.1.4.803:=2)) is valid. Do you mind changing the filter string and try again? Thanks

Hi. Thanks a lot. Now it's working, but only with mailnews.ldap.jsmodule in false.

Comment 15

[Ping Chen [:rnons] (away)]

Can you get some debug logs

1. set mailnews.ldap.loglevel to All
2. open DevTools, clear existing logs in the Console
3. trigger a ldap search

Comment 16

[muncubus]

When mailnews.ldap.jsmodule in false - no any messages and filtering works fine.
When mailnews.ldap.jsmodule in true I've got:
mailnews.ldap: Connecting to ldap://-----:389 LDAPClient.jsm:43:18
mailnews.ldap: Connected LDAPClient.jsm:144:18
mailnews.ldap: Binding b-----a LDAPClient.jsm:64:18
mailnews.ldap: C: [1] BindRequest LDAPClient.jsm:254:18
mailnews.ldap: S: [1] BindResponse resultCode=0 message="" LDAPClient.jsm:191:20
mailnews.ldap: Searching dn="dc=-----" filter="(&(&(objectCategory=Person)(objectclass=user)(mail=*)(!(useraccountcontrol:1.2.840.113556.1.4.803:=2)))(&(|(cn=стан)(givenName=стан)(sn=стан)(mozillaNickname=стан)(mail=стан)(mozillaSecondEmail=стан)(&(description=стан))(o=стан)(ou=стан)(title=стан)(mozillaWorkUrl=стан)(mozillaHomeUrl=стан))))" LDAPClient.jsm:124:18
mailnews.ldap: C: [2] SearchRequest LDAPClient.jsm:254:18
mailnews.ldap: S: [2] SearchResultReference <empty string> 3 LDAPClient.jsm:191:20
mailnews.ldap: S: [2] SearchResultDone resultCode=0 message="" LDAPClient.jsm:191:20
And filtering not working. about "-----" don't mind .

​

Comment 17

[Ping Chen [:rnons] (away)]

Thanks, looks like SearchResultReference is not correctly handled by ldap-js. With mailnews.ldap.jsmodule set to false, can you get some logs by starting TB with MOZ_LOG=LDAP:5 thunderbird. If you're on Windows, refer to https://wiki.mozilla.org/MailNews:Logging#Generating_a_Protocol_Log

Comment 18

[Ping Chen [:rnons] (away)]

Created bug 1756383 for SearchResultReference handling.

Comment 19

[muncubus]

Yes. Here https://mega.nz/file/S5IUVRLB#1XjhPFKc7ExZIginjG37HZJZAHYdq1Xk7VGsjzNnw00, please.

Comment 20

[Ping Chen [:rnons] (away)]

(In reply to Christian Fertig from comment #4)

I've tested against your daily build 94.0a1 and I can confirm, that it's working again :)

Hi, can you help me test if this works on TB 91 with mailnews.ldap.jsmodule set to true (a restart is needed)? Seems other people still see this problem.

Comment 21

[Christian Fertig]

Hi,

ok, I tested both settings with my 91.7.0 32bit and it seems not to work with ldap-jsmodule enabled.
Attached the screenshots with enabled and disabled setting, the LDAP directory is only searched with the jsmodule false.

Christian

Comment 22

[Christian Fertig]

Attachment: Thunderbird 91.7 ldap-jsmodule-disabled

Comment 23

[Christian Fertig]

Attachment: Thunderbird 91.7 ldap-jsmodule-enabled - no LDAP hit

Comment 24

[Christian Fertig]

(In reply to Christian Fertig from comment #21)

my Filter is still

(&(objectclass=User)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

Comment 25

[Ping Chen [:rnons] (away)]

Thanks a lot, can you also test a beta or daily build? Last time 94.0a1 seemed to work for you, I'm trying to find if there is some code we forgot to uplift to 91.

Comment 26

[Ping Chen [:rnons] (away)]

I think I've found the problem, will make a fix.