1833651 - Unexpected "Searching the directory for recipients' certificates. This may take a few minutes." pop-up blocks typing while composing new e-mail

Status: RESOLVED FIXED

(MailNews Core :: Security: S/MIME, defect)

Description

[aandre]

Attachment: Capture d'écran 2023-05-17 164412.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42

Steps to reproduce:

1. Open Thunderbird
2. Click "Write"
3. Fill the "To" section with any e-mail address
4. Press "Enter"

The next pop-up appears :
Downloading Certificates
Searching the directory for recipients certificates. This may take a few minutes.

Actual results:

The pop-up is stuck on the screen until you click "Stop Searching".
You cannot type until doing this action.

Expected results:

No pop-up was displayed until Thunderbird 100 has been released.

Comment 1

[Magnus Melin [:mkmelin]]

Fallout from suggesting encryption in s/mime?

Comment 2

[aandre]

EDIT
The pop-up only appears if you have an S/MIME certificate configured and in use :
Tools > Account settings > End-To-End Encryption > S/MIME > Select…

Comment 3

[Kai Engert (:KaiE:)]

(In reply to Magnus Melin [:mkmelin] from comment #1)

Fallout from suggesting encryption in s/mime?

This dialog usually is shown when the user:

• has enabled/configured S/MIME
• has configured an LDAP directory for looking up contacts

In that scenario, we've always looked for missing certificates on the LDAP server.

Therefore I'm guessing that "S/MIME reminder enabled" should NOT be responsible for this bug. I'm guessing that you'd still get that popup if you set setting mail.smime.remind_encryption_possible to false (using config editor). @aandre Can you confirm?

However, prior to fixing bug 1811298
( Enable OCSP for S/MIME recipient encryption certificates,
https://hg.mozilla.org/releases/comm-esr102/rev/36ca6aa96424 )
that was done after the user hit "send" for the email (and certificates were still missing).

Because we could no longer do blocking OCSP requests, the behavior was changed to perform S/MIME checks (including OCSP) earlier, to ensure we already have all information at the time we need to send the message.

Comment 4

[Kai Engert (:KaiE:)]

It's unexpected that the popup remains on screen.

@aandre is it possible that your LDAP configuration refers to a server that is offline or has problems?

See thunderbird settings, composition, section "addressing". You probably have a directory server configured and enabled.

Can you please open the Thunderbird error console (CTRL/COMMAND + SHIFT + J), before you add the recipient in the composer window, and check if entering the composer window triggers any error messages on the console?

Comment 5

[aandre]

Hi Kai,
We do use a company LDAP, configured and in use. It is working fine for addressing.

While using the Thunderbird console, these errors are displayed :

LDAPClient.jsm:252:18
mailnews.ldap:
error { target: TCPSocket, isTrusted: true, name: "ConnectionRefusedError", message: "Network", errorCode: 2152398861, srcElement: TCPSocket, currentTarget: TCPSocket, eventPhase: 2, bubbles: false, cancelable: false, … }

LDAPOperation.jsm:186
NS_ERROR_XPC_JSOBJECT_HAS_NO_FUNCTION_NAMED: JavaScript component does not have a method named: "onLDAPError"'JavaScript component does not have a method named: "onLDAPError"' when calling method: [nsILDAPMessageListener::onLDAPError]

Thank you

Comment 6

[Magnus Melin [:mkmelin]]

Looks like onLDAPError is indeed not implemented there. We can fix that but I don't know if it should do anything more than log that error.

Comment 7

[Kai Engert (:KaiE:)]

I'm trying to reproduce to better understand what's happening. We need to find out if the missing implementation of onLDAPError is the cause for the operation to stop, and the dialog remaining on screen.

I think in general we should attempt to find a solution that no longer uses that popup when downloading certificates.

However, for the immediate fix, if the reason the dialog is stuck is because of a JS exception, it would be the easiest and quickest solution to the fix the exception and handle it gracefully.

Comment 8

[Kai Engert (:KaiE:)]

(In reply to aandre from comment #5)

We do use a company LDAP, configured and in use. It is working fine for addressing.

It's confusing that you say it's working for addressing, but the error on the JS console reports that it cannot connect to an LDAP server.

Comment 9

[Magnus Melin [:mkmelin]]

Attachment: Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

Comment 10

[Kai Engert (:KaiE:)]

Based on Magnus' findings, I note that we have to places in which an LDAP error can occur. Apparently TB in your environment is able to create a connection, but then runs into a failure when submitting the request, and we don't handle that scenario.

Because you are using stable TB 102, I think we should create a minimal patch that only fixes that scenario, avoids the JS exception, and closes the window when that situation happens.

Comment 11

[Kai Engert (:KaiE:)]

Attachment: Bug 1833651 - Handle additional error scenarios when fetching S/MIME certificates from LDAP. r=mkmelin

Comment 12

[Kai Engert (:KaiE:)]

@aandre Could you help us to test this fix, prior to adding it to the stable release, to ensure it's indeed fixing the issue for you?

What operating system would you prefer to use for testing the fix?

Comment 13

[aandre]

@Kai, sure no problem, I'm running Windows 11 actually.

Comment 14

[Kai Engert (:KaiE:)]

The Windows test build should be ready in about 45-60 minutes. I'll add a link once it's ready.
Or you could find it yourself, on https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=a58ce27eb6f5dc1b73e96d5bdf17398734a276f3
once the letter B behind "Windows 2012" turns into green, click it, in the lower area click "artifacts", and click "target.zip" to download, extract into a separate directory and run it. You might want to run it with parameter -P to be able to select your usual profile.
That build will be based on 102.11 plus this fix.

Comment 15

[Kai Engert (:KaiE:)]

Hmm, there's also a signed build, that might be easier to use. Direct link to the target.zip file:
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eq_ySF6hQKSF5I3_TdsqYA/runs/0/artifacts/public/build/target.zip

As said above, if you run it without parameters, it will probably create a separate profile. Use parameter -P to select your usual profile, with existing settings.

Comment 16

[aandre]

I just had a try with the version you've sent here.
After entering the e-mail address, I still can see the small pop-up but it disapears after a second.
Then I can type as in the past.
Works fine as expected !

Comment 17

[Kai Engert (:KaiE:)]

(In reply to aandre from comment #16)

Works fine as expected !

Thank you very much for testing and the feedback!

Comment 18

[aandre]

Thanks a lot for your help and investigation !

Comment 19

[Kai Engert (:KaiE:)]

Comment on attachment 9335392 [details]
Bug 1833651 - Handle additional error scenarios when fetching S/MIME certificates from LDAP. r=mkmelin

I recommend to uplift this to stable esr102 on the next occassion. Landing into c-c will happen today. The patch is very minimal and very safe, and won't affect anyone excepts users who already use the potential failure scenario.

[Approval Request Comment]
Regression caused by (bug #):
User impact if declined: stuck UI
Testing completed (on c-c, etc.): manually
Risk to taking this patch (and alternatives if risky): low

Comment 20

[Magnus Melin [:mkmelin]]

(In reply to aandre from comment #16)

Works fine as expected !

Out of interest, what do you get logged as warning in the error console?

Comment 21

[aandre]

This is the warning log displayed now :

LDAP error: 2152398861
certFetchingStatus.js:260:13

Comment 22

[Kai Engert (:KaiE:)]

Do you use a self-signed certificate, or expired cert, on the LDAP server?

Comment 23

[aandre]

Yes we do use a self-signed cert

Comment 24

[Kai Engert (:KaiE:)]

That's very likely the reason why Thunderbird reports a failure connecting to it.

While TB has some code to allow overriding certificates, the code for connecting to an LDAP server may not allow triggering that, so you aren't prompted, and the connection is silently refused.

I don't understand why LDAP lookups are working for addressing in your environment.

Comment 25

[Kai Engert (:KaiE:)]

The patch from comment 11, https://phabricator.services.mozilla.com/D178766
was landed into comm-central, plus lint fixes:

https://hg.mozilla.org/comm-central/rev/27392bad94952159ff8a2e2547e4f7a79c975d6f
https://hg.mozilla.org/comm-central/rev/1f9b90cff3a17ed76c6e3d1fb0b3e7a955396772

There were some whitespace mismatches that caused the phabricator patch to not apply on c-c cleanly, that's why I merged manually, and then I forgot to include the phabricator revision ID in the commit, which causes the automatic tracking to not work - sigh.

Comment 26

[aandre]

Weird, we do use self-signed certificate, connect using SSL with port 636 and the only thing we aren't able to do since the upgrade from Thunderbird 91.9.1 to 102.0 is the offline sync (Failure).
Otherwise in online mode we have no apparent issue, contacts and certificates can be found by typing first letters into the "To" section.

Comment 27

[Kai Engert (:KaiE:)]

Maybe addressing still works because you did an offline sync in the past.

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/d47ff6010cec
Lint follow up with latest linting config. rs=bustage DONTBUILD

Comment 29

[aandre]

It also works fine on fresh installed computers.
Also with newly created accounts.

Comment 30

[Kai Engert (:KaiE:)]

On the working machines, are you prompted to add an override for the cert?

Have a look at file cert_override.txt in the profile directory on the broken system. You could try to delete the line (while thunderbird is stopped) that refers to the LDAP server. Just an idea.

Comment 31

[aandre]

No, never. We add the cert authority and allow it, configure the directory and that's all.
Nothing prompted.
I cannot find the cert_override.txt file, either in profile directory or progfiles. Is this something created only if we are prompted to add an override ?

Comment 32

[Kai Engert (:KaiE:)]

(In reply to aandre from comment #31)

We add the cert authority and allow it, configure the directory and that's all.

If you installed and trusted the matching CA cert in Thunderbird that issued the LDAP server's certificate, then I don't understand why Thunderbird wouldn't connect. Maybe a missing intermediate.

I cannot find the cert_override.txt file, either in profile directory or progfiles. Is this something created only if we are prompted to add an override?

Yes, probably only created if necessary.
When you said self-signed, I thought the server certificate is self signed, but it sounds like only your root CA is self signed, and your LDAP server certificate issued/signed by your CA cert.

Comment 33

[Kai Engert (:KaiE:)]

Comment on attachment 9335392 [details]
Bug 1833651 - Handle additional error scenarios when fetching S/MIME certificates from LDAP. r=mkmelin

Should also go to beta

Comment 34

[Magnus Melin [:mkmelin]]

(In reply to aandre from comment #21)

LDAP error: 2152398861

That means you got NS_ERROR_CONNECTION_REFUSED.

Theory: you have some old LDAP server also configured and that's used for a particular identity (but not working of course, server offline perhaps?).
Note that you can set the LDAP server per identity under Account Settings | Composition and Addressing.

Comment 35

[Wayne Mery (:wsmwk)]

Comment on attachment 9335392 [details]
Bug 1833651 - Handle additional error scenarios when fetching S/MIME certificates from LDAP. r=mkmelin

[Triage Comment]
Approved for beta

Comment 36

[Wayne Mery (:wsmwk)]

Comment on attachment 9335392 [details]
Bug 1833651 - Handle additional error scenarios when fetching S/MIME certificates from LDAP. r=mkmelin

[Triage Comment]
Approved for esr102

Comment 37

[Kai Engert (:KaiE:)]

Please land both patches. I accidentally forgot to request approval on the second patch, too.

The second patch is the one that fixes the bug !!!
Sorry

Comment 38

[Wayne Mery (:wsmwk)]

Comment on attachment 9335388 [details]
Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

[Triage Comment]
Approved for beta
Approved for esr102

Comment 39

[Kai Engert (:KaiE:)]

argh, sorry. I was wrong.

I misread the patches. Sorry.

Comment 40

[Daniel Darnell [:dandarnell]]

114.0b6:
https://hg.mozilla.org/releases/comm-beta/rev/8d31a12bd7f6

Comment 41

[Kai Engert (:KaiE:)]

I recommend to take the additional patch (modernized) - revision D178765 - for c-c and possibly 115.
I think we don't need it on 102.

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/a5c479e86dc7
Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

Comment 43

[Kai Engert (:KaiE:)]

I see this landed on comm-esr102 for 102.12.0
https://hg.mozilla.org/releases/comm-esr102/rev/9d42734e12597ccdb59fee178bf369d8c328dcad

Comment 44

[Rob Lemley [:rjl]]

Thunderbird 102.12.0:
https://hg.mozilla.org/releases/comm-esr102/rev/9d42734e1259

Comment 45

[Kai Engert (:KaiE:)]

Comment on attachment 9335388 [details]
Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

I suggest to uplift the second patch (the more thorough code cleanup) to 115, because I'd prefer to avoid the manual merging for a backported version of bug 1841348.

Comment 46

[Wayne Mery (:wsmwk)]

(In reply to Kai Engert (:KaiE:) from comment #45)

Comment on attachment 9335388 [details]
Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

I suggest to uplift the second patch (the more thorough code cleanup) to 115, because I'd prefer to avoid the manual merging for a backported version of bug 1841348.

But patch 2 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version - landed in comment 42. https://hg.mozilla.org/comm-central/rev/a5c479e86dc7

Comment 47

[Kai Engert (:KaiE:)]

(In reply to Wayne Mery (:wsmwk) from comment #46)

But patch 2 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version - landed in comment 42. https://hg.mozilla.org/comm-central/rev/a5c479e86dc7

The c-c landing from comment 42 only covered 116.

When I requested beta on 2023-07-03, beta was still version 115.
In the meantime beta has changed to 116.

This means I need to change my uplift request to comm-esr115.

Comment 48

[Kai Engert (:KaiE:)]

Comment on attachment 9335388 [details]
Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

Changing my approval request to comm-esr115, as explained above.

Comment 49

[Wayne Mery (:wsmwk)]

Comment on attachment 9335388 [details]
Bug 1833651 - Add unimplemented LDAPMessageListener.onLDAPError - modernized version. r=kaie

[Triage Comment]
Approved for esr115

Indeed, comment 42 was a few days after the merge.

Comment 50

[Rob Lemley [:rjl]]

Thunderbird 115.0:
https://hg.mozilla.org/releases/comm-esr115/rev/d3f16130171e