1954355 - Variation selectors stripped from subject breaking emojis and other Unicode characters

Status: NEW

(Thunderbird :: Message Compose Window, defect)

Description

[Teal Dulcet [:tdulcet] (TB Council)]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:136.0) Gecko/20100101 Firefox/136.0

Steps to reproduce:

• Create an e-mail with a subject that has the red heart emoji ❤️ (U+2764), including the required Unicode variation selector (U+FE0F).
• Confirm that the heart is red in the subject line, then save the message and open the drafts folder.

Actual results:

In the message list and in the message header, the heart is black and white, as if the variation selector was not included.

Expected results:

The heart should be red in all places. -> 🟥 This works as expected in Firefox.

I tested with both the latest Thunderbird 128 ESR and Daily 138. The bug seems to affect any emoji that requires a variation selector, as for example the information source emoji ℹ️ also does not display correctly.

Comment 1

[Teal Dulcet [:tdulcet] (TB Council)]

Attachment: image.png

See the attached screenshot for an example. Note how the hearts at both ends are in black and white, while the rest of the emojis are in color as expected.

Comment 2

[Alfred Peters [:infofrommozilla]]

Yes, the character (U+FE0F) is removed by the cleanToken() function.

https://searchfox.org/comm-central/source/mailnews/mime/jsmime/jsmime.mjs#335

    token = token.replace(
      /[\u034F\u17B4\u17B5\u180B-\u180D\uFE00-\uFE0F]/g,
      ""
    );

Here is the stacktrace from the Composer:

cleanToken (resource:///modules/jsmime/jsmime.mjs#335)
decodeRFC2047Words (resource:///modules/jsmime/jsmime.mjs#842)
parseUnstructured (resource:///modules/jsmime/jsmime.mjs#212)
parseStructuredHeader (resource:///modules/jsmime/jsmime.mjs#1570)
setRawHeader (resource:///modules/MimeJSComponents.sys.mjs#212)
GetComposeDetails (chrome://messenger/content/messengercompose/MsgComposeCommands.js#6026)
GenericSendMessage (chrome://messenger/content/messengercompose/MsgComposeCommands.js#6104)
SaveAsDraft (chrome://messenger/content/messengercompose/MsgComposeCommands.js#7067)
Save (chrome://messenger/content/messengercompose/MsgComposeCommands.js#7045)

Comment 3

[Alfred Peters [:infofrommozilla]]

Bug 1900708 is about a similar problem only with the zero-width joiner.

Comment 4

[Teal Dulcet [:tdulcet] (TB Council)]

Thanks for the information. Thunderbird should not be blindly striping any of the 16 variation selector characters (or the zero-width joiner), as they all have important uses in Unicode and so removing them changes the meaning of the resulting text. For example, of the 3,781 emojis in Unicode 16.0, 1,160 require a variation selector and 1,468 require at least 1 zero-width joiner, so this is breaking 1,687 emojis total or around 45% of them. Of course, the variation selectors have many uses outside of emojis, which this is also breaking. I agree with bug 1900708 comment 10.