1604773 - Unable to retrieve s/mime certificate from LDAP directory (v68.3.0)

Status: RESOLVED FIXED

(Thunderbird :: Security, defect)

Description

[Nasos]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.4 Safari/605.1.15

Steps to reproduce:

Firstly I want to note that this was working in 68.2.2 and stopped working in 68.3.0.
-Configure LDAP directory
-Compose new mail message and select from Composition toolbar "Security"->"Encrypt this message".
-Click Security from Composition toolbar or select from that "View Security Info"

other notes:
Autocomplete works, the LDAP directory is configured with no authentication, tested it in Mac and Windows and in case it helps I 'm finding in v68.3.0 Release Notes a fix having description:
"LDAP lookup not working when SSL was enabled. LDAP search not working when "All Address Books" was selected."

Actual results:

The error message is:

Unable to save your message as a draft.
You specified encryption for this message, but the application failed to find an encryption certificate for 

Expected results:

Retrieve the recipient's certificate from LDAP

Comment 1

[Kai Engert (:KaiE:)]

Is anyone aware of a public LDAP directory with S/MIME certificates available, so we could test/reproduce this bug?

Comment 2

[Nasos]

I used also the one mentioned here https://bugzilla.mozilla.org/show_bug.cgi?id=430422

Comment 3

[Kai Engert (:KaiE:)]

thanks Nasos. Setting needinfo for myself to analyze.

Comment 4

[Nasos]

thank you for helping with that Kai. I re-checked just to confirm that the ldap records have the "userCertificate;binary" entry
ldapsearch -LLL -x -H ldap://ldap.pca.dfn.de -b o=DFN-Verein,c=DE "(mail=[e-mail])"

Comment 5

[Kai Engert (:KaiE:)]

I've investigated, and I found the following details. I'm not sure if that can be considered a bug, it might only be a misunderstanding of how the feature is working.

First, configuring an LDAP directory is insufficient to trigger the LDAP lookup.
The code checks the value of preference "ldap_2.autoComplete.directoryServer".

This is empty in my test, even after having added the directory server.

I found that this pref must be set of the preference prefix of the configured LDAP server, which might not be obvious, but can be looked up in config editor or profile file prefs.js.

I had to set ldap_2.autoComplete.directoryServer to this value: ldap_2.servers.DFNPKI

This triggers the lookup, which can be seen by a brief popup window showing.
I found that the code indeed obtains data from the server, and attempts to import it.
However, the certificate cannot be found in certificate manager.

This could be caused by another behavior of the certificate import code. Based on my memory of that code, I consider the following likely:

The import code will attempt to "verify" the certificate. This means, it will attempt to build a chain of trust to a known root CA. If that isn't possible (no chain is found), or if the certificate is considered "not valid", it will not be imported.

I looked at the certificate stored in the directory. It was issued by the following CA:
Issuer: C = DE, O = Verein zur Foerderung eines Deutschen Forschungsnetzes e. V., OU = DFN-PKI, CN = DFN-Verein Global Issuing CA

This might be an intermediate CA, or a root CA, I don't know. My profile doesn't have that CA, so no chain can be found, likely causing the import attempt to be rejected.

Try to obtain that CA and import it. Import all additional intermediate CAs that might be required, if there is more than one level to the root CA. If it chains to a root CA that isn't trusted for email certificates, you'll have to mark the respective CA as trusted.

Then try again.

Comment 6

[Kai Engert (:KaiE:)]

Well, ok, the first part (ldap_2.autoComplete.directoryServer not set) can probably be accomplished by selecting the configured directory server in the composition/addressing prefs, after having edited directories and added a server.

Comment 7

[Nasos]

I confirm that to me too, the autocomplete process works by selecting the appropriate server in composition/addressing prefs. That was never an issue from the first place but it's good we 're excluding that code wise.

Regarding the verification of the recipient certificate I tried the following:

• I downloaded the user's Certificate using ldapsearch i.e.
  ldapsearch -LLL -x -t -H ldap://ldap.pca.dfn.de -b o=DFN-Verein,c=DE "(mail=arlinghaus --at-- fh-muenster.de)"
  and imported that in TB Certificate Manager/People. Compose a new message, require encryption, click Security from Composition toolbar and the status of the certificate in the popup window is "Not Trusted".

• I imported the root CA I found in https://info.pca.dfn.de/dfn-ca-global-g2/ in Certificate Manager/Authorities and after repeating the above procedure I'm getting a status "Valid". Bare in mind that the user's certificate has been imported manually in People tab in previous step.

• I tried to use another recipient using the autocomplete in a new message i.e. Olaf Koch <o.koch --at-- hszg.de>, that is, having the root CA installed and forcing TB to search, download and import the certificate. I'm getting the same behaviour (same as in my first post) from TB reporting:
  You specified encryption for this message, but the application failed to find an encryption certificate for
  and the status of certificate is "Not found".

Comment 8

[Kai Engert (:KaiE:)]

Thanks for your testing. I was able to identify the failure and have a fix.

This regressed as part of bug 1562157, which was a fix uplifted for Thunderbird 68.3.0

That code changed interface nsILDAPBERValue worked, but nobody detected that the JavaScript code for LDAP certificate fetching still used the old API. Because we don't have strong typing in JS, and because there isn't any automated test, this went unnoticed.

In the fix, I'll add a comment that mentions type name nsILDAPBERValue, so future developers who search for uses of that type have a chance to find this place.

Comment 9

[Kai Engert (:KaiE:)]

Attachment: 1604773-v1.patch

[Approval Request Comment]
Regression caused by (bug #): 1562157
User impact if declined: LDAP S/MIME certificate fetching remains broken
Testing completed (on c-c, etc.): not yet
Risk to taking this patch (and alternatives if risky): no risk

Comment 10

[Magnus Melin [:mkmelin]]

Comment on attachment 9120782 [details] [diff] [review]
1604773-v1.patch

Review of attachment 9120782 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, thx! r=mkmelin

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/a9d1bbebb67f
Restore ability to retrieve S/MIME certificates from LDAP. r=mkmelin

Comment 12

[Jorg K (CEST = GMT+2)]

This regressed as part of bug 1562157, which was a fix uplifted for Thunderbird 68.3.0

Sorry 'bout that :-( - I had to uplift the lot since otherwise the LDAP fix patches from bug 1576364 didn't apply.

Comment 13

[Kai Engert (:KaiE:)]

(In reply to Jorg K (GMT+1) (PTO to 26th Jan 2020, sporadically reading bugmail) from comment #12)

This regressed as part of bug 1562157, which was a fix uplifted for Thunderbird 68.3.0

Sorry 'bout that :-( - I had to uplift the lot since otherwise the LDAP fix patches from bug 1576364 didn't apply.

We cannot blame you! The blame is on the missing tests. Just wanted to document the cause.

Comment 14

[Geoff Lankow (:darktrojan)]

73 beta 1:
https://hg.mozilla.org/releases/comm-beta/rev/7e2733b1e871d5bcdd4e850889a44a865ff5a92c

Comment 15

[Kai Engert (:KaiE:)]

Hello Aryx, could you please land on THUNDERBIRD_68_VERBRANCH? TIA.

Comment 16

[Kai Engert (:KaiE:)]

(In reply to Kai Engert (:KaiE:) from comment #15)

Hello Aryx, could you please land on THUNDERBIRD_68_VERBRANCH? TIA.

Ah sorry, not in gecko.

Comment 17

[Geoff Lankow (:darktrojan)]

68.4.2:
https://hg.mozilla.org/releases/comm-esr68/rev/2c52c943f7d3d234ae4ec35356f387b5daa557cd