Closed Bug 1574248 Opened 6 years ago Closed 6 years ago

Certificate was not updated from LDAP Directory server

Categories

(Thunderbird :: Security, defect)

Product:
Thunderbird
Component:
Security
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(thunderbird_esr68+)

Status:
RESOLVED DUPLICATE of bug 1576364
Tracking Flags:
Tracking Status
thunderbird_esr68 + ---

People

(Reporter: Tyson, Unassigned)

References

Details

(Keywords: regression)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

Composed a message to a recipient for whom my TB's db of certificates has an expired cert and specified that the message should be encrypted. My Directory Server is specified to be an LDAP server that has the current valid certificate for that recipient.
[When switching between 69.0b2 and 60.8.0, the directory server setting appears to go to an empty value. I made sure it was set.]

Actual results:

The security status showed the person's certificate as expired. Eventually TB consults the Directory Server (visible via a momentary alert pop-up that says it is checking the directory server). Moments later, a notice appears that it can't autosave the message because the certificate for the recipient is not valid.

I find this error (created earlier than the actual search) in the Error Console
NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIAbDirectoryQuery.doQuery]

Expected results:

When it consults the Directory Server, it should have gotten the updated certificate and replaced it in my TB for this recipient. The security status should be updated to show my TB has a valid cert for the recipient, and the autosave (and sending) should succeed.

I quit TB 69.0b2 and started TB 60.8.0. I set the directory server to our site's directory server. I composed a message to the same recipient, set it to be encrypted, checked the Security Status and it showed his cert as Expired. I waited for the system to consult the directory server. I checked again and found the recipient's cert was now valid. No related error appears in the Error Log.

I go back to TB 69.0b2 and find that recipient's cert is now valid. However, the problem still occurs if I send a message to another person with an expired cert.

I consider this a serious defect. If we can't update certificate info, we can't send encrypted mail, which we must do.

So I guess that's broken in TB 68, too. Can you please try a pre-release version from here:
http://ftp.mozilla.org/pub/thunderbird/candidates/68.0-candidates/build2/

If I read the report correctly, TB 60 did update the certificate from the LDAP server, TB 69 and likely TB 68 don't. That makes it a regression.

We have a person who routinely finds the cause for regressions, but this is too hard to set up. Over to the security team.

tracking-thunderbird_esr68: --- → +
Flags: needinfo?(mkmelin+mozilla)
Flags: needinfo?(kaie)
Keywords: regression

Rather hard to test :(

Looks like (part of) the code is around here: https://searchfox.org/comm-central/rev/cb71ec93ae822bce37fb00e321ef201cce24ff1e/mailnews/extensions/smime/content/msgCompSecurityInfo.js#44

xref bug 1275180 and bug 1509586 comment 6.

Component: Untriaged → Security
Flags: needinfo?(mkmelin+mozilla)

We need an LDAP server instance for debugging.

Do you want me to setup and maintain a server?

Flags: needinfo?(kaie) → needinfo?(mkmelin+mozilla)

(In reply to Jorg K (GMT+2) from comment #1)

So I guess that's broken in TB 68, too. Can you please try a pre-release version from here:
http://ftp.mozilla.org/pub/thunderbird/candidates/68.0-candidates/build2/

In 68.0 (64-bit) on OS X (add-ons disabled), the bug happens (the cert is not updated). It appears that when TB 68 looks up an address in LDAP (well before any need for the certificate), this error appears
NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIAbDirectoryQuery.doQuery] 2 nsAbLDAPAutoCompleteSearch.js:261

I tried to turn on LDAP logging, but the log file was empty.
export MOZ_LOG=LDAP:5,timestamp
I have successfully logged IMAP with the script as in https://wiki.mozilla.org/MailNews:Logging
I only changed "IMAP" to "LDAP".

[My bad on the logging. I suspect I didn't get the log because I was cd'ed to a directory I had no write permissions when I ran my script (even though the script writes the log file to $HOME. Or possibly, using both LDAP:5 and ldap:5 made it work.]

The log was
2019-08-16 18:09:42.362195 UTC - [(null) 32534: Main Thread]: D/LDAP nsLDAPOperation::SearchExt(): called with aBaseDn = '/ldap_2.servers.SRIDirectory_1'; aFilter = '(|(cn=min**)(mail=min**)(sn=min**))'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 100
2019-08-16 18:09:43.469239 UTC - [(null) 32534: Main Thread]: D/LDAP nsLDAPOperation::SearchExt(): called with aBaseDn = '/ldap_2.servers.SRIDirectory_1'; aFilter = '(|(cn=min.)(mail=min.)(sn=min.**))'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 100
2019-08-16 18:10:48.149042 UTC - [(null) 32534: Main Thread]: D/LDAP unbinding
2019-08-16 18:10:48.149056 UTC - [(null) 32534: Main Thread]: D/LDAP unbound

The first two lines correspond to my adding the recipient to the TO: line.
The second two correspond to the time when the LDAP did the lookup later for the certificate.

I was surprised this was all the LDAP logging that occurred. Perhaps something is failing before it actually does the queries, so there is no query to log.

Related but apparently not the root problem:

In TB 69.0, another bug prevents me from accessing the LDAP server to download the data from the LDAP server for offline use. However, that bug doesn't happen in 68.0. Thus in TB 68.0, I can use that (under Preferences > Composition > Addressing > Edit Directories > Edit > Offline > Download now) to make this query. That query fails, with "Replication failed" shown in the window. No error message appears in the log.
In 60.8.0, the replication succeeds as something over 1000 records are downloaded.

For the server, Base DN = ou=People,dc=SRI,dc=COM, port = 636, Bind DN = (no entry), Use SSL is checked.

I realized the problem could be SSL. If TB has updated its requirements for SSL, but our internal-only LDAP server doesn't support them, then that could be the cause.

So, I unchecked "Use SSL". The replication succeeded. I checked "Use SSL" and it failed. I unchecked "Use SSL" and again tried to send an encrypted message to a recipient for which my TB had an expired certificate. I saw TB pop up the alert to say it was querying the server, but the cert info didn't update, and TB complained it couldn't autosave.

(NOTE: I do not depend upon the TB LDAP server for autocompletion of addresses. That may be working or not.. It is rather difficult for me to distinguish previously used email address (in local address book) from those provided solely by LDAP. As I recall there is a different icon, but I'd have to figure it out again.)

Thanks for testing TB 68 which will be our next major release, planned for next week. If there are any issues, we'll try to get them fixed before upgrading everyone from TB 60 to TB 68.

We found two issues related to LDAP: Bug 1574712 and bug 1574590. Let's get those fixed first.

Depends on: 1574712, 1574590

I guess this issue will be resolved when bug 1574590 lands. Fingers crossed. So automated testing in the area wouldn't go astray though.

Can you please retest with TB 69 beta 4 from here http://ftp.mozilla.org/pub/thunderbird/candidates/69.0b4-candidates/build1/ and TB 68.0 ESR (pre-release) from here http://ftp.mozilla.org/pub/thunderbird/candidates/68.0-candidates/build5/.

Flags: needinfo?(Tyson)

Fudge! Apparently TB69 beta 4 made an incompatible change to my profile, so I can't immediately go back to 68 or earlier.
While I'm figuring that out, here's my initial experiences. I'll add another comment when to confirm or correct what I find.

Earlier tonight, I downloaded 68.0 ESR from 68.0-candidates/build5/. It initially appeared to still have (this) bug 157428. However, that likely was due to the fact that LDAP SSL worked for us in 60.8 but apparently does not work in 68+. I did not try LDAP without SSL.

When I had time later, not realizing that if I ran TB 69, I was going to burn my bridges for going back to 68, I tried 69.0b4.

TB69.0b4: If I disable LDAP SSL, I do not experience this bug. I get no errors in the console window.
TB 69.0b4 seems to have bug 1574249 . regardless of whether SSL is set for LDAP or not. (As reported there, I'm in Mac OS X 10.14.6, if it matters.)

If I enable LDAP SSL, I do have the bug. (In TB 60.8, I do not have the bug with LDAP SSL enabled.)
I get these errors in the console window. First, when I bring up the compose window.

NS_NOINTERFACE: Component returned failure code: 0x80004002 (NS_NOINTERFACE) [nsISupports.QueryInterface] stack-trace-collector.js:90

Then when I start to type in the TO: field, at the 2nd character, I get
: Component returned failure code: 0x80590051 [nsIAbDirectoryQuery.doQuery] 3 nsAbLDAPAutoCompleteSearch.js:261

I get no further error message when it tries to go to the LDAP server, but the certificate does not get updated.

Yes, maybe my LDAP server is incompatible with changes made to TB's SSL. However, at the very least I can't QA LDAP+SSL. If you know that LDAP+SSL works on your test servers, I still would suggest that release notes should point out that change.

Back to 68.0esr attempt.
I realized I had an old laptop that had an older TB config on it. It last ran TB 45.8. I upgraded it to TB 60.8 and all seemed to go fine.

But then I tried to run 68.0esr. I could never get it to work. It appeared never to communicate properly with our internal IMAP server or with Google's IMAP server, as it reported timeouts (after several minutes).

The weird thing is that my newer laptop did talk to the server with TB 68.0esr. Why should one work and not the other? I thought maybe there was something different about the configs. Either something problematic on the non-working system, or some preference that enabled it on the working system.

So I tried to make a new config on the old laptop in 68.0esr. It learned from our server that we offer IMAP through a particular host and that TB should use StartTLS. All that went fine, until it needed to test the password. That again seemed as if it never got a response back.
When I start a new message (Command-N), the error console gets this before I do anything more. That seems like more evidence that something isn't right, and I can't blame it on the LDAP server. Now TB can't talk to the IMAP server. (and remember, it timed out talking to Googlemail as well)

When I went back to 60.8, I had no problem making the new config nor reading mail from our IMAP server or from Google.

So I guessed that the problem again might be SSL/TLS. I looked at all the configuration settings that include "ssl" or "tls". On the system that did work for a while with TB 68.0esr, there were settings that weren't in the never-working system. These preferences were marked as modified.
These may or may not be relevant, but here they are.
mail.smtpserver.smtp1.try_ssl 2
mail.smtpserver.smtp10.try_ssl 3
mail.smtpserver.smtp2.try_ssl 2
mail.smtpserver.smtp3.try_ssl 1
mail.smtpserver.smtp4.try_ssl 2
mail.smtpserver.smtp5.try_ssl 0
mail.smtpserver.smtp6.try_ssl 3
mail.smtpserver.smtp7.try_ssl 2
mail.smtpserver.smtp8.try_ssl 3
mail.smtpserver.smtp9.try_ssl 2
There is also an unmodified preference.
mail.smtpserver.default.try_ssl 0

At this point, I feel I've stumbled into another difference with 68.0esr (and presumably 69+) that will cause grief for users and may be a bug.
I do hope it is just that I'm doing something dumb...

As for QA'ing 68.0esr, I can not yet. Maybe in 12-18 hours or so (it is the end of my day), I can recover my profile & such from a backup and try that on the laptop that did work with 68.0esr until it ran 69.

You can go back with -allow-downgrade. Let me read your post now.

Thanks for testing. So in summary:

  • 68 and 69 work without SSL, both don't work with SSL in your particular case. 69 additionally exposes bug 1574249.
  • The error in stack-trace-collector.js:90 sadly happens all the time an is unrelated.
  • Let's not mix other account setup problems into this report here.

So to me the question remains: What changes to SSL were made between 60 and 68 and will that affect other users. Kai, are you aware of such changes?

We should also set up our own testing infrastructure for this.

Flags: needinfo?(Tyson) → needinfo?(kaie)

OK, I tried SSL with the server from Adams.edu and it doesn't work for address-autocomplete. I get
Component returned failure code: 0x80590051 [nsIAbDirectoryQuery.doQuery] 2 nsAbLDAPAutoCompleteSearch.js:261
I filed bug 1576364 for that. This is about address entry and auto-complete, it's still not about updating certificates.

I guess there is no chance fixing this without fixing bug 1576364 first.

Depends on: 1576364

(In reply to Kai Engert (:kaie:) from comment #3)

We need an LDAP server instance for debugging.

Do you want me to setup and maintain a server?

That could be useful. Please take a look and see if you can get it all set up without too much work.

Flags: needinfo?(mkmelin+mozilla)

Jörg, Mabry, when upgrading to a new major ESR release, it's possible that the NSS library has changed its behavior in its SSL/TLS implementation.

Usually, they try to be as backwards compatible as possible, but sometimes some some older server configurations might get deprecated by the new NSS client.

I'm not aware of any major changes during the past year, but I'd have to go read all the release notes for the past year, to be certain.

If TB 60.x worked with SSL/TLS, but TB 68.x doesn't work SSL/TLS, then we should analyze what's going on. I suggest that you use Wireshark, and with each of 60.x and 68.x, you create a log of connecting with TB to your LDAP server. (Wireshark allows you to set filters, to only log the connections of interest, and you should set a filter to the LDAP server hostname and the relevant port. If you haven't used Wireshark previously, maybe you could find someone who could help you with that.)

I'm guessing you are using an LDAP server that is behind a firewall, so we cannot connect it, right?

Flags: needinfo?(kaie)

We'd have to check release notes for NSS versions 3.37 to 3.44.

I skimmed through release notes, but couldn't find anything major.

Note that we occasionally experienced in the past that an old server is intolerant, when seeing unexpected (newer) features being offered in the TLS handshake, and disconnects. In other words, it's possible that's a bug on the server side of your LDAP server, which only happens with more recent clients.

Comparing Wireshark logs of old and new TB, and then using a Wireshark option to display the log as a SSL/TLS protocol session, would likely tell us more.

Does certificate downloading work, if you don't have any certificate yet? (delete the cert, then restart, so instead of an update attempt, it will be an initial download attempt)

Version: 69 → 68

Jörg just pointed me to comment 14. I guess that means you think that bug 1576364 is the underlying cause.

Mabry, you could temporarily try a setting, which is less secure than the default. If making the change fixes your issue, then this is a duplicate of bug 1576364. The experimental and insecure change is to open preferences / advanced / certificates, disable "query OCSP", then restart TB.

Yes, this appears to be a duplicate of bug 1576364.
In 68.0, I tested & confirmed the bug. Then I disabled the query of OCSP servers and tried the same thing (I did not restart TB). The certificate was updated (the bug did not happen). I then re-enabled the query of OCSP servers and tried the same thing (for a new recipient with expired certificate). The bug again happened.

Good luck in chasing it down. It was good to see how diligently all of you work at making TB work well. Thanks!

Thanks for testing. You helped us to find a lot of issues in LDAP.

Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.