923043 - Thunderbird doesn't handle IDN correctly (From/To/Cc address not sent as punycode)

Status: NEW

(MailNews Core :: Internationalization, defect)

Description

[Vincent MAZARD]

User Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; Tablet PC 2.0)

Steps to reproduce:

1 - Create an IMAP account with an email address whose domain name contains one or more accents
2 - send an email to anybody


Actual results:

1 - Thundebird doesn't translate the 'from:' field to punycode.
It sends it as-is during the smtp negociation.It doesn't even encode it to a 7-bits format like quoted part.
As a result, the mail server doesn't transfer it properly : the resulting email, read from the receiver's email client, contains unreadable characters in the "from:" filed instead of accents.

2 - Sometimes, users must enter their complete email address to authenticate to their mail server. If the user configures his login with the original email, then Thunderbird doesn't encode it to punycode when trying to authenticate.
As a result, the user can't login to the IMAP server. The problem is the same in the outgoing SMTP server configuration.
To solve this issue, the user must set his login to the punycode encoded email address. This is not really annoying, but not very "sexy" either.
Users rarely know the punycode translation for their domain name.
And there's no reason they should.


Expected results:

Thunderbird should translate every email to its punycode format if the domain part contains 8-bits characters :
- in the from: header when sending an email (and any other header related to the sender's email address like reply-to: etc...)
- in the login processes (pop, imap, smtp, etc..)

Comment 1

[Mark Banner (:standard8)]

I'm pretty sure this is a sub-set of bug 235312.

Comment 2

[Magnus Melin [:mkmelin]]

I'm not so sure about authentication. This message suggests both UTF-8 and punycode are in theory correct:
http://dovecot.org/list/dovecot/2012-February/081369.html

Comment 3

[teddy]

I can confirm that mails are not deliverable if the TO address field contains country specific characters. The adress is definitly NOT converted to punycode in TB 38.4.0. They are sent "as is". Tested with "info@päpsy-erbach.de" as recipient. In the mail server logs the TO address is as entered in TB.
The punycode discussion lasts for at least five years now. Is it just broken in recent versions or is it not avaliable at all?

Comment 5

[Francesco (no longer active, use PM for contact)]

As stated in comment 3, it doesn't only affect the From: field, but also other addressing fields. I've just come across an issue with an umlaut domain being hosted at Ionos. There the server rejects the message based on the umlaut being present in the To: header.

Comment 6

[Francesco (no longer active, use PM for contact)]

Sending a message to a umlaut domain via a Microsoft server (smtp-mail.outlook.com) works fine. The MS server advertises SMTPUTF8 and so TB doesn't use ACE:
https://searchfox.org/comm-central/rev/7a9bc53bfb291ce128e6fa60e0b231ec267834ae/mailnews/compose/src/SmtpClient.sys.mjs#225-267

However, the MS server seems to do the "right thing" by downgrading the SMTP envelope (RCPT TO) and mail headers to ACE/Punycode/ASCII. So in light of that, it's a bit questionable whether TB should uglify headers just to cater for MTAs which advertise SMTPUTF8 but don't fully support it (RFC 6531), and MTAs what don't implement UTF-8 headers (RFC 6532).