1469335 - Thunderbird ignores character set setting for outgoing mails, ISO-8859-15 no longer supported as outgoing charset

Status: RESOLVED WONTFIX

(Thunderbird :: Untriaged, defect)

Description

[Lothar Paltins]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180516032328

Steps to reproduce:

I'm using Thunderbird under OpenSuse Linux in a locale with the ISO-8859-15 character encoding. Until recently, my outgoing mails had the correct encoding line "Content-Type: text/plain; charset=iso-8859-15; format=flowed".


Actual results:

I've now seen, that my outgoing mails have "charset=windows-1252" instead of "charset=iso-8859-15" regardless of any preference and any appropriate about:config setting. This must have changed with one of the last updates, I'm now using TB-52.8.0.


Expected results:

I want to have "charset=iso-8859-15" or at least "charset=iso-8859-1" in my outgoing mails.

Comment 1

[Jorg K (CEST = GMT+2)]

To my knowledge, the decoding is *not* picked up from the OS locale. It is instead configured here:

Tools > Options > Display, Formatting, Advanced button (maybe different on Linux).
It can also be set for an individual message via:
Options > Text Encoding.

I'm not aware of any change to this behaviour during the TB 52.x cycle.

Comment 2

[Lothar Paltins]

You are describing how it SHOULD work. I've set the coding for outgoing and incoming messages to Western (ISO-8859-1), but this doesn't have any effect. And for an individual message I can only choose between UTF-8, Western and some other encodings, but ISO-8859-1 and ISO-8859-15 aren't in the list and the "Western" encoding results in windows-1252 and not ISO-8859-1.

And as I've said, in the past my outgoing messages had "charset=iso-8859-15" although that's not at all selectable in the preferences. It must have been extracted from my environment.

Comment 3

[Jorg K (CEST = GMT+2)]

(In reply to lptmp10 from comment #2)
> You are describing how it SHOULD work. I've set the coding for outgoing and
> incoming messages to Western (ISO-8859-1), but this doesn't have any effect.
I cannot reproduce this. I set ISO-8859-1 in "Tools > Options > Display, Formatting, Advanced button" and the e-mail gets send as windows-1252 which is "Western". Note support for ISO-8859-1 has been removed from mozilla56 (bug 1261841) and windows-1252 is now a synomym for all things "Western", including ISO-8859-1.

> And for an individual message I can only choose between UTF-8, Western and
> some other encodings, but ISO-8859-1 and ISO-8859-15 aren't in the list and
> the "Western" encoding results in windows-1252 and not ISO-8859-1.
Yes, see above. 

> And as I've said, in the past my outgoing messages had "charset=iso-8859-15"
> although that's not at all selectable in the preferences. It must have been
> extracted from my environment.
We don't support ISO-8859-15 any more, not sure we ever did, the trend is certainly going towards UTF-8. Messages received in that encoding should still be displayed correctly.

Comment 4

[Lothar Paltins]

Did TB really give the official character set names of the International Standardization Organization for the proprietary names of one commercial software company? Very strange.

But it doesn't matter. It's only possible to set the default charsets to ISO-8859-1 in the preferences window and TB replaces it by windows-1252. But if I set the default_charset entries in prefs.js directly (or in the about:config window) to ISO-8859-15, then it's not replaced and the correct value appears in the outgoing mails.

Comment 5

[Jorg K (CEST = GMT+2)]

(In reply to lptmp10 from comment #4)
> ..., then it's not replaced and the correct value appears in the outgoing mails.
That might change in TB 60.

Henri, can you comment a bit on ISO-8859-15 and correct/confirm my comment #3. Thanks in advance.

Comment 6

[Henri Sivonen (:hsivonen)]

 * Gecko doesn't look at the glibc locale for character encodings. Local file system paths are treated as UTF-8. Non-UTF-8 glibc local are unsupported, but if the file system paths that Gecko needs to access are ASCII, stuff works.

 * The set of encodings that Gecko supports are those defined in the Encoding Standard (https://encoding.spec.whatwg.org/). The names and labels defined there are treated as authoritative. Thunderbird additionally supports receiving UTF-7 email.

 * ISO-8859-1 is a label (alias) of windows-1252 in the Encoding Standard. The name (preferred label) of the encoding is used when generating an outgoing label, which is why you see windows-1252 and not ISO-8859-1.

 * ISO-8859-15 is a distinct encoding, whose name is ISO-8859-15.

 * Firefox doesn't provide UI for manually overriding the encoding of a Web page to ISO-8859-15.

 * My unverified recollection is that Thunderbird synced its UI with Firefox and removed the UI for setting prefs to ISO-8859-15 as a side effect.

 * There is no practical need to send email as ISO-8859-15. UTF-8 and windows-1252 are older encodings and can represent everything that ISO-8859-15 can. (ISO-8859-15 exists for political reasons and not to address a practical need.)

 * Any system that can deal with receiving email sent by Gmail or Apple Mail has to be able to deal with receiving UTF-8 email, which is a pretty strong indication that sending UTF-8 email is fine.

 * I recommend removing the ability to send non-UTF-8 email. Bug 862292.

 * I think resolving this report as WONTFIX makes sense.

Comment 7

[Henri Sivonen (:hsivonen)]

(In reply to Henri Sivonen (:hsivonen) from comment #6)
>  * My unverified recollection is that Thunderbird synced its UI with Firefox
> and removed the UI for setting prefs to ISO-8859-15 as a side effect.

Correction: Thunderbird shortened the list even further compared to Firefox:
https://searchfox.org/comm-central/source/mailnews/base/content/charsetList.xml#19