SMTP CRAM-MD5 authentication fails when server challenge contains non-ASCII bytes due to incorrect UTF-8 encoding
Categories
(MailNews Core :: Networking: SMTP, defect)
Tracking
(thunderbird_esr153 wontfix, thunderbird160 wontfix)
People
(Reporter: aazh64, Assigned: mkmelin)
Details
Attachments
(7 external files)
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:160.0) Gecko/20100101 Firefox/160.0
Steps to reproduce
-
Configure an SMTP server supporting AUTH CRAM-MD5. The server must issue a challenge containing at least one byte above 0x7F after Base64 decoding.
-
In Thunderbird, configure the outgoing SMTP server:
- Connection security: SSL/TLS
- Authentication method: Encrypted password (CRAM-MD5)
- Port: 465
- Username and password: valid credentials
-
Attempt to send an email.
-
Observe that SMTP authentication fails, although the same credentials work with AUTH PLAIN/LOGIN over TLS and with CRAM-MD5 in Claws Mail.
The issue is reproducible in Thunderbird 153.0.3 (Windows) and 158.0.1 (Linux).
Minimal reproducible challenge (Base64):
PDQyNDYzNjI0MjYuMTQ4MzUyODhAgXcugW0+
Decoded challenge: 27 bytes, including two 0x81 bytes.
In modules/MailAuthenticator.sys.mjs, function getCramMd5Token(password, challenge), the decoded challenge is incorrectly passed through TextEncoder, changing the binary data from 27 to 29 bytes.
Replacing:
new TextEncoder().encode(atob(challenge))
with:
Uint8Array.from(atob(challenge), c => c.charCodeAt(0))
fixes the issue. The fix was tested against a real Sendmail/Cyrus SASL SMTP server and resulted in 235 2.0.0 OK Authenticated.
Actual results
Thunderbird fails SMTP CRAM-MD5 authentication with:
535 5.7.0 authentication failed
The Sendmail/Cyrus SASL server logs:
AUTH failure (CRAM-MD5): authentication failure (-13)
SASL(-13): authentication failure: incorrect digest response
Root cause:
TextEncoder converts the binary string returned by atob(challenge) into UTF-8, corrupting challenge bytes above 0x7F.
For the provided challenge, both 0x81 bytes become 0xC2 0x81, increasing its length from 27 to 29 bytes.
As a result, Thunderbird calculates an incorrect HMAC-MD5 digest, which the server rejects.
Expected results
Thunderbird should preserve the exact bytes of the Base64-decoded CRAM-MD5 challenge when calculating HMAC-MD5, including bytes above 0x7F.
With valid credentials, the SMTP server should accept authentication and respond:
235 2.0.0 OK Authenticated
CRAM-MD5 authentication should work regardless of whether the server challenge contains only ASCII bytes or arbitrary binary bytes.
|
Assignee | |
Comment 1•17 days ago
|
||
Confirmed. (Not a regression.)
|
|
Assignee | |
Comment 2•17 days ago
|
||
For testing, this is called "Encrypted Password" in the settings.
|
|
Assignee | |
Comment 3•17 days ago
|
||
|
|
Assignee | |
Comment 4•17 days ago
|
||
If the server sent a CRAM-MD5 challenge that isn't valid base64, atob() threw
inside the SMTP and POP3 auth handlers, and the login stalled until the
connection timed out.
Instead, send "*" to cancel the exchange, as RFC 4954 (SMTP) and RFC 5034
(POP3) allow. The server rejects the cancellation, and the login then fails
through the usual path: the next auth method is tried, or the user is asked
what to do.
|
|
Assignee | |
Comment 5•17 days ago
|
||
SmtpAuthenticator.getPlainToken() fetched the password itself rather than
through SmtpClient._getPassword(), so cancelling the password prompt threw
NS_ERROR_ABORT out of the auth code: no QUIT was sent, no error was reported,
and sending stalled until the connection timed out.
Get the password with _getPassword() and return when there is none, as the
LOGIN, CRAM-MD5 and NTLM paths do, and pass it to getPlainToken() like the
POP3/IMAP authenticator already does.
|
|
Assignee | |
Comment 6•17 days ago
|
||
LOGIN and CRAM-MD5 asked for the password only after the server's 334 reply,
so the server sat waiting for the next line while the password prompt was
open. A slow user could hit the server's command timeout, and when the prompt
was cancelled, the server took the QUIT that followed as the password or
digest and rejected it. The connection then stayed open until the client's
QUIT close timeout expired.
Get the password before sending AUTH, like PLAIN and NTLM already do, and pass
it to the handlers. A cancelled prompt now sends no AUTH at all.
|
|
Assignee | |
Comment 7•17 days ago
|
||
test_smtpAuthPlain.js, test_smtpAuthLogin.js and the cancellation task in
test_smtpAuthCramMd5.js were the same test for different auth methods. Replace
them with test_smtpAuthPasswordCancelled.js, which runs it for PLAIN, LOGIN and
CRAM-MD5.
|
|
Assignee | |
Comment 8•17 days ago
|
||
SmtpAuthenticator and IncomingServerAuthenticator had identical copies of
getPlainToken(password). Keep a single one in MailAuthenticator.
The callers in SmtpClient and Pop3Client don't need any change: they call the
method on SmtpAuthenticator and Pop3Authenticator instances with the same
arguments, and both classes inherit it from the base class.
|
||
Updated•17 days ago
|
|
|
Assignee | |
Updated•17 days ago
|
Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/7f7efc7c83ad
Convert the decoded CRAM-MD5 challenge to bytes as is instead of UTF-8 encoding it. r=babolivier
https://hg.mozilla.org/comm-central/rev/2e9b349d4837
Cancel the CRAM-MD5 exchange when the server challenge isn't valid base64. r=babolivier
https://hg.mozilla.org/comm-central/rev/4b927ebb7681
Fail the SMTP AUTH PLAIN login when the password prompt is cancelled. r=babolivier
https://hg.mozilla.org/comm-central/rev/e5727164abc9
Ask for the SMTP password before AUTH LOGIN and AUTH CRAM-MD5. r=babolivier
https://hg.mozilla.org/comm-central/rev/d38e059c638a
Merge the SMTP password prompt cancellation tests. r=babolivier
https://hg.mozilla.org/comm-central/rev/1d0a2d91a544
Move getPlainToken() to the MailAuthenticator base class. r=babolivier
|
|
Assignee | |
Comment 10•16 days ago
|
||
After the cancelled password prompt the client sends QUIT, but stays in the
SmtpServer connection lists until it notices the server closing the
connection. closeCachedConnections() then sent it a second QUIT, which the
server recorded when it arrived before the connection was closed.
Only close the cached connections once the transaction has been checked.
|
||
Comment 11•16 days ago
|
||
Description
•