1728942 - Searching ldap address book with Chinese character doesn't work

Status: RESOLVED FIXED

(Thunderbird :: Address Book, defect)

Description

[Ryan Ho]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

1. Open Address Book
2. Select Ldap Directory from left panel
3. Type some Chinese character like "王" in search field, then press Enter

Actual results:

Display no result, can't find anything

Expected results:

It should have found some names

Comment 1

[Ryan Ho]

Ldap server log

[03/Sep/2021:13:13:38.678841267 +0800] conn=5089 op=0 RESULT err=0 tag=97 nentries=0 wtime=0.000047041 optime=0.000064271 etime=0.000108957 dn=""
[03/Sep/2021:13:13:38.688631527 +0800] conn=5089 op=1 SRCH base="dc=ds1,dc=mydomain,dc=com,dc=tw" scope=2 filter="(&(|(cn=*%E7%8E%8B*)(givenName=*%E7%8E%8B*)(sn=*%E7%8E%8B*)(mozillaNickname=*%E7%8E%8B*)(mail=*%E7%8E%8B*)(mozillaSecondEmail=*%E7%8E%8B*)(&(null=TRUE)(description=*%E7%8E%8B*))(o=*%E7%8E%8B*)(ou=*%E7%8E%8B*)(title=*%E7%8E%8B*)(mozillaWorkUrl=*%E7%8E%8B*)(mozillaHomeUrl=*%E7%8E%8B*)))" attrs=ALL
[03/Sep/2021:13:13:38.705923772 +0800] conn=5089 op=1 RESULT err=0 tag=101 nentries=0 wtime=0.000129649 optime=0.017291965 etime=0.017419271 notes=U details="Partially Unindexed Filter" - Invalid attribute in filter - results may not be complete.```

Comment 2

[Ryan Ho]

Same query filter using Apache Directory Studio and actually find some names.

    (|
        (cn=*王*)
        (givenName=*王*)
        (sn=*王*)
        (mozillaNickname=*王*)
        (mail=*王*)
        (mozillaSecondEmail=*王*)
        (&
            (null=TRUE)
            (description=*王*)
        )
        (o=*王*)
        (ou=*王*)
        (title=*王*)
        (mozillaWorkUrl=*王*)
        (mozillaHomeUrl=*王*)
        (null=*王*)
        (null=*王*)
    )
)```

Ldap server log
```[03/Sep/2021:13:19:13.033318306 +0800] conn=5094 op=10 RESULT err=0 tag=101 nentries=1 wtime=0.000133907 optime=0.000174020 etime=0.000305324
[03/Sep/2021:13:19:24.669603850 +0800] conn=5094 op=12 SRCH base="dc=ds1,dc=mydomain,dc=com,dc=tw" scope=2 filter="(&(|(cn=*\E7\8E\8B*)(givenName=*\E7\8E\8B*)(sn=*\E7\8E\8B*)(mozillaNickname=*\E7\8E\8B*)(mail=*\E7\8E\8B*)(mozillaSecondEmail=*\E7\8E\8B*)(&(null=TRUE)(description=*\E7\8E\8B*))(o=*\E7\8E\8B*)(ou=*\E7\8E\8B*)(title=*\E7\8E\8B*)(mozillaWorkUrl=*\E7\8E\8B*)(mozillaHomeUrl=*\E7\8E\8B*)(null=*\E7\8E\8B*)(null=*\E7\8E\8B*)))" attrs="objectClass"
[03/Sep/2021:13:19:24.674232814 +0800] conn=5094 op=12 RESULT err=0 tag=101 nentries=8 wtime=0.000255632 optime=0.004630283 etime=0.004882239 notes=U details="Partially Unindexed Filter" - Invalid attribute in filter - results may not be complete.```

Comment 3

[Ryan Ho]

TB is encoding "王" into %E7%8E%8B, and Apache Directory Studio is \E7\8E\8B.
I think that made difference.

Comment 4

[Ping Chen [:rnons] (away)]

Attachment: Bug 1728942 - Fix searching ldap with Chinese character. r=darktrojan

The queryString is percent encoded, decode it before parsing.

Comment 5

[Wayne Mery (:wsmwk)]

Rob, if this doesn't land before you do the merge on Monday, can we get this as an uplift for your 93.0b1 build?

Comment 6

[Rob Lemley [:rjl]]

I'm building tomorrow, so hopefully it lands before then.

Comment 7

[Rob Lemley [:rjl]]

Please request uplift once this is reviewed and landed in comm-central.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/54f4e2c5d697
Fix searching ldap with Chinese character. r=darktrojan

Comment 9

[Ping Chen [:rnons] (away)]

Comment on attachment 9239554 [details]
Bug 1728942 - Fix searching ldap with Chinese character. r=darktrojan

[Approval Request Comment]
Regression caused by (bug #): bug 1696625
User impact if declined: Searching ldap with Chinese characters doesn't work
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): low

Comment 10

[Wayne Mery (:wsmwk)]

Comment on attachment 9239554 [details]
Bug 1728942 - Fix searching ldap with Chinese character. r=darktrojan

[Triage Comment]
Approved for beta

Comment 11

[Rob Lemley [:rjl]]

Thunderbird 93.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/143666c870d6

Comment 12

[Wayne Mery (:wsmwk)]

Comment on attachment 9239554 [details]
Bug 1728942 - Fix searching ldap with Chinese character. r=darktrojan

[Triage Comment]
Approved for esr91

Comment 13

[Rob Lemley [:rjl]]

Thunderbird 91.1.1:
https://hg.mozilla.org/releases/comm-esr91/rev/9df75f85df70