1596377 - Implement RFC 6532 to allow UTF-8 in the "references" IDs to not break threads using utf-8 in the "references" header

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[dr0i]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

Answering emails which uses UTF-8 in the "references"-header breaks threads.

Actual results:

TB doesn't handle UTF-8 encoded headers. This may break things, especially "references", which breaks threads if other email clients uses this and TB is answering such an email.

Expected results:

As https://en.wikipedia.org/wiki/International_email states:

"MIME-encoding must not be used to encode other information in a header, such
as an email address, or header fields like Message-ID or Received. Moreover,
the MIME-encoding requires extra processing of the header to convert the data
to and from its MIME-encoded word representation, and harms readability of a
header section.

The 2012 standards RFC 6532 and RFC 6531 allow the inclusion of Unicode
characters in a header content using UTF-8 encoding, and their transmission via
SMTP - but in practice support is only slowly rolling out."

Would be nice "to roll this out".

Comment 1

[dr0i]

Just tried mutt, amd there is no such problem. They state to be RFC 6532 compatible since 2015 (https://fossies.org/linux/mutt/ChangeLog).

Comment 3

[Thomas D. (:thomas8)]

In bug 1799112, I've just added testcase messages and confirmed that this is still failing on 102.5.0 (64-bit), Win10. Broken threads are odd, and not honoring the RFCs looks like a bug.

Geoff, any idea how hard this might be?

Comment 4

[Magnus Melin [:mkmelin]]

Probably mostly about the strings using the old string types. https://searchfox.org/comm-central/rev/968fe9d536296413aeba7eb2d9c5dfdbbec6b118/mailnews/base/public/nsIMsgHdr.idl#87,89 (should switch to AUTF8String)

Comment 5

[Geoff Lankow (:darktrojan)]

This really isn't my territory. Magnus has a better idea about this stuff than I do.

Comment 6

[Magnus Melin [:mkmelin]]

Bug 1799112 is about qencoded reference header, and is invalid.
I actually don't see the problem if really UTF-8 is used. It may be two wrongs makes a right, and comment 4 would fix those.

Comment 7

[Magnus Melin [:mkmelin]]

Attachment: Bug 1596377 - Allow UTF-8 References header to work. r=benc

Comment 8

[Paul Menzel]

Comment on attachment 9304372 [details]
Bug 1596377 - Allow UTF-8 References header to work. r=benc

Nit: Typo in summary/title: referernces → references

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/4096fead717d
Allow UTF-8 References header to work. r=benc

Comment 10

[dr0i]

Thx :)