494865 - [Central-European languages] Sent read receipts are incorrectly encoded if Thunderbird interface is set to non-English

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect)

Description

[Răzvan Sandu]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; ro; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10
Build Identifier: version 2.0.0.21 (20090409)

I am running a stock Ubuntu 9.04 machine, with stock Thunderbird 2.0.0.21.

The system is set to Romanian language.

UTF-8 is set in Thunderbird -> Preferences -> Display -> Fonts -> Character
Encoding (both drop lists; "Use default encoding in replies" is checked).

When sending read receipts for messages I've received, I get the following error from the mailserver (which runs stock amavisd-new, spamassassin and clamav on Fedora 10):

-------------------------------------------------------------------------------
The message WAS NOT relayed to:
  <[e-mail]>:
   554 5.6.3 Reject, id=10500-01 - BAD_HEADER: Non-encoded 8-bit data (char C8 hex): Subject: Confirmare de primire (afi\310\231at) - :...

This nondelivery report was generated by the program amavisd-new at host
mail3.mobexpert.ro. Our internal reference code for your message is
10500-01/RlpEsMKH2nTO

INVALID HEADER: INVALID 8-BIT CHARACTERS IN HEADER

  Non-encoded 8-bit data (char C8 hex): Subject: Confirmare de primire
    (afi\310\231at) - :))\n

Return-Path: <[e-mail]>
Message-ID: <[e-mail]>
Subject: Confirmare de primire (afișat) - :))

WHAT IS AN INVALID CHARACTER IN MAIL HEADER?

  The RFC 2822 standard specifies rules for forming internet messages.
  It does not allow the use of characters with codes above 127 to be
  used directly (non-encoded) in mail header.

  If such characters (e.g. with diacritics) from ISO Latin or other
  alphabets need to be included in a header, these characters need
  to be properly encoded according to RFC 2047. Such encoding is often
  done transparently by mail reader (MUA), but if automatic encoding is
  not available (e.g. by some older MUA) it is a user's responsibility
  to avoid using such characters in mail header, or to encode them
  manually. Typically the offending header fields in this category are
  'Subject', 'Organization', and comment fields in e-mail addresses
  of 'From', 'To' or 'Cc' header fields.

  Sometimes such invalid header fields are inserted automatically
  by some MUA, MTA, content checker, or other mail handling service.
  If this is the case, that service needs to be fixed or properly
  configured. Typically the offending header fields in this category
  are 'Date', 'Received', 'X-Mailer', 'X-Priority', 'X-Scanned', etc.

  If you don't know how to fix or avoid the problem, please report it
  to _your_ postmaster or system manager.

-----------------------------------------------------------------------------




Setting the Ubuntu system in English (form the System menu - Language support)
solves the problem, so I guess is a matter of encoding/localization.

I guess the bug is not specific to Romanian language.



Reproducible: Always

Steps to Reproduce:
1. Configure an Ubuntu 9.04 machine as described above.
2. Allow sending read receipts for incoming messages that contain non-english characters in header.
3. Analyze proper encoding of read receipts or filter them through amavis+clamav+spamassassin
Actual Results:  
Read receipts get rejected and returned to sender.

Expected Results:  
Read receipts should be encoded normally and pass through the mail filters.

The following Unicode codes are widely used in Romanian language (not
exclusively):

0218, 0219, 021A, 021B

System-wide support for Romanian language is correctly implemented in current
Ubuntu, Fedora, Debian and probably others (still not in CentOS 5.3).

Comment 1

[Ludovic Hirlimann [:Usul]]

Razvan is this still true with 3.0b2 ?

Comment 2

[Alexandru Szasz]

Maybe related with bug 484525 ?

Comment 3

[Răzvan Sandu]

@Ludovic

Hello,

I'm experimenting this on a stock Ubuntu 9.04 which is set in Romanian. I'm unable to install Thunderbird 3.0b2 here, so I neither infirm nor confirm.

As far as I know, I've never experienced this bug on Fedora 11 (rawhide), which I use on a different machine.

Regards,
Răzvan

Comment 4

[Alexandru Szasz]

You can get Thunderbird 3b2 in Romanian from here: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-central-l10n/thunderbird-3.0b3pre.ro.linux-i686.tar.bz2

Comment 5

[Răzvan Sandu]

@Ludovic

Hello,

I'm experimenting this on a stock Ubuntu 9.04 which is set in Romanian. I'm unable to install Thunderbird 3.0b2 here, so I neither infirm nor confirm.

As far as I know, I've never experienced this bug on Fedora 11 (rawhide), which I use on a different machine.

Regards,
Răzvan

Comment 6

[Răzvan Sandu]

I may bet that this bug has the same cause as bug no. 392815 in Ubuntu's Launchpad: the absence of that specific non-ASCII locale in Thunderbird 2.x:

https://bugs.launchpad.net/ubuntu/+source/thunderbird/+bug/392815

Romanian language is affected; can someone please verify these with other non-ASCII languages that have no official implementation of Thunderbird 2.x ?

It is *easy* to verify/confirm these bugs, even for developers that don't have oane of the affected languages as their mother tongue.

My two cents...


Răzvan

Comment 7

[Wayne Mery (:wsmwk)]

ovidiu, can you assist?

(In reply to comment #6)
> I may bet that this bug has the same cause as bug no. 392815 in Ubuntu's
> Launchpad: the absence of that specific non-ASCII locale in Thunderbird 2.x:
> 
> https://bugs.launchpad.net/ubuntu/+source/thunderbird/+bug/392815
> 
> Romanian language is affected; can someone please verify these with other
> non-ASCII languages that have no official implementation of Thunderbird 2.x ?
> 
> It is *easy* to verify/confirm these bugs, even for developers that don't have
> oane of the affected languages as their mother tongue.
> 
> My two cents...
> 
> 
> Răzvan

Comment 8

[Nikolay Shopik]

I'm pretty sure I have filled such bug long time ago, but can't find it.

Comment 9

[Nikolay Shopik]

Here is Bug 445033, which appears to be fixed for me since beta 2 of tb3.

Comment 10

[Ludovic Hirlimann [:Usul]]

Răzvan Sandu  is the issue still present in 3.1.7 / 3.1.8 ?

Comment 11

[Răzvan Sandu]

Thank you !

As far as I can test (not extensively), this is not present in Thunderbird 3.1.7 - I get the receipts correctly. I am using stock Ubuntu 10.10 now, set for Romanian language.

However, as a related matter:

If one composes a mail in plain-text mode and puts a word inside stars, to make it appear bold  at the recipient ( *bold* ), this DOES NOT work if the word contains some non-English characters (such as Unicode 0218, 0219, 021A, 021B).


Best regards,
Răzvan

Comment 12

[Wayne Mery (:wsmwk)]

Răzvan, thanks for the update.
please file a new bug for any remaining issues you may see when using version 6 or newer.