484525 - Save As Text doesn't save Romanian diacritics!!!

Status: RESOLVED DUPLICATE of bug 181456

(Thunderbird :: Mail Window Front End, defect)

Description

[((Gabriel)) Tudor]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.1b4pre) Gecko/20090318 Shredder/3.0b3pre

The commas below the Romanian characters S and T are put next to those letters, which is totally wrong! Romanian language has these diacritics: Ă Î Â Ș Ț, where Ș and  Ț are S and T with comma below (NOT CEDILLA BELOW!), so they must be saved as such. At the moment, Thunderbird doesn't save characters with comma below! It puts the comma after the character. 

Reproducible: Always

Steps to Reproduce:
1. Write this text in Thunderbird or copy it:     Bună ziua!

    Aplicația de pe Net care afișează datele meteo dă o eroare la selectarea tabelului în ordinea temperaturilor.
    Mesajul este acesta: Input array is longer than the number of columns in this table.
    Browser-ul folosit este Mozilla Firefox 3.0.6 (deși nu cred să aibă vreo legătură cu eroarea). 
    Sper că puteți remedia cât mai repede această problemă.

    O zi bună! 
2. File - Save As - Text 
3. Open the saved TXT file. 
Actual Results:  
You will get this after the Save As action: 

    Bună ziua!

    Aplicat,ia de pe Net care afis,ează datele meteo dă o eroare la selectarea tabelului în ordinea temperaturilor.
    Mesajul este acesta: Input array is longer than the number of columns in this table.
    Browser-ul folosit este Mozilla Firefox 3.0.6 (des,i nu cred să aibă vreo legătură cu eroarea). 
    Sper că putet,i remedia cât mai repede această problemă.

    O zi bună!

Expected Results:  
As you can see, the "s" and "t" characters have lost their comma below which has beep put right next to them. 

The correct display of the TXT file should be exactly the same as the initial text: 

    Bună ziua!

    Aplicația de pe Net care afișează datele meteo dă o eroare la selectarea tabelului în ordinea temperaturilor.
    Mesajul este acesta: Input array is longer than the number of columns in this table.
    Browser-ul folosit este Mozilla Firefox 3.0.6 (deși nu cred să aibă vreo legătură cu eroarea). 
    Sper că puteți remedia cât mai repede această problemă.

    O zi bună!

Comment 1

[Alexandru Szasz]

I can't reproduce this with the upcoming version 3, so I assume the version discussed is 2.

Comment 2

[((Gabriel)) Tudor]

It happens in the latest Thunderbird 3 nightly. This is actually old issue, as it happened since 3b1.

Comment 3

[Alexandru Szasz]

I tried on a Linux machine, could you give more details about your configuration ? Operating system, version, os language, regional settings, thunderbird's language, what program are you opening the file with after it was saved ?

Comment 4

[((Gabriel)) Tudor]

Windows XP SP3, English OS, Romanian settings (Regional and Language Options-Control Panel), Thunderbird language: Romanian (although this has nothing to do with the problem). I open the file with Notepad, regarding the fact that it's saved as TXT.

Comment 5

[Alexandru Szasz]

I assume you have the European Union Expansion Font Update ( http://www.microsoft.com/downloads/details.aspx?familyid=0ec6f335-c3de-44c5-a13d-a1e7cea5ddea&displaylang=en ) installed, right ?

For the record, the update contains fonts that have the characters s and t with comma bellow.

Maybe it would be useful to attach here the saved file.

Comment 6

[((Gabriel)) Tudor]

Yes, I have that EUupdate.exe installed. Without it, I coudn't see the Romanian characters. 

I attached the file saved as TXT from Thunderbird.

Comment 7

[((Gabriel)) Tudor]

Attachment: Message with lost RO diacritics

Comment 8

[((Gabriel)) Tudor]

Comment on attachment 368864 [details]
Message with lost RO diacritics

>Subiect:
>Eroare în programul cu datele meteo
>de la:
>"((Gabriel)) Tudor" <[e-mail]>
>Datã:
>Sun, 01 Mar 2009 14:18:33 +0200
>Cãtre:
>[e-mail]
>
>    Bunã ziua!
>
>    Aplicat,ia de pe Net care afis,eazã datele meteo dã o eroare la selectarea tabelului în ordinea temperaturilor.
>    Mesajul este acesta: Input array is longer than the number of columns in this table.
>    Browser-ul folosit este Mozilla Firefox 3.0.6 (des,i nu cred sã aibã vreo legãturã cu eroarea). 
>    Sper cã putet,i remedia cât mai repede aceastã problemã.
>
>    O zi bunã!
>-- 
>((Gabriel))
>editor-prezentator RT
>    Acest mesaj este confident,ial s,i adresat doar
>destinatarului, precum s,i altor persoane autorizate
>sã-l primeascã.
>    Dezvãluirea, copierea, distribuirea sau init,ierea unor
>act,iuni pe baza lui, fãrã acordul meu scris, sunt strict
>interzise s,i atrag rãspunderea juridicã.
>    Dacã at,i primit acest mesaj din gres,ealã, vã rog sã
>mã anunt,at,i imediat, s,i sã-l s,terget,i apoi din sistemul
>dumneavoastrã.
>    Subliniez cã orice comentarii pe care le fac reprezintã
>punctul meu de vedere, s,i nu cel oficial al Patriarhiei Române.

Comment 9

[Alexandru Szasz]

The files seems to be encoded in ISO-8859-2 (Romanian Obsolete) or ISO-8859-16 (Romanian). For the record ș and ț (s and t with comma under) are only in ISO-8859-16 and UTF-8.

Could you have a look at the message source and see what encoding does it have ?

In the View source window change the Encoding from the View (Vizualizare) -> Encoding (Codare caractere) until you see the s and t with comma correctly (șț).

Comment 10

[((Gabriel)) Tudor]

This is the message source: 

From - Thu Mar 05 13:29:38 2009
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Message-ID: <[e-mail]>
Disposition-Notification-To: "((Gabriel)) Tudor" <[e-mail]>
Date: Sun, 01 Mar 2009 14:18:33 +0200
From: "((Gabriel)) Tudor" <[e-mail]>
Organization: Radio Trinitas
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.1b3pre) Gecko/20090223 Thunderbird/3.0b2
MIME-Version: 1.0
To: [e-mail]
Subject: Eroare =?UTF-8?B?w65uIHByb2dyYW11bCBjdSBkYXRlbGUgbWV0ZW8=?=
Content-Type: multipart/alternative;
 boundary="------------000802070603010108040600"

This is a multi-part message in MIME format.
--------------000802070603010108040600
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable

     Bun=C4=83 ziua!

     Aplica=C8=9Bia de pe Net care afi=C8=99eaz=C4=83 datele meteo d=C4=83=
 o eroare la=20
selectarea tabelului =C3=AEn ordinea temperaturilor.
     Mesajul este acesta: Input array is longer than the number of=20
columns in this table.
     Browser-ul folosit este Mozilla Firefox 3.0.6 (de=C8=99i nu cred s=C4=
=83 aib=C4=83=20
vreo leg=C4=83tur=C4=83 cu eroarea).
     Sper c=C4=83 pute=C8=9Bi remedia c=C3=A2t mai repede aceast=C4=83 pr=
oblem=C4=83.

     O zi bun=C4=83!
--=20
*((Gabriel))
**editor-prezentator RT***
     Acest mesaj este confiden=C8=9Bial =C8=99i adresat doar
destinatarului, precum =C8=99i altor persoane autorizate
s=C4=83-l primeasc=C4=83.
     Dezv=C4=83luirea, copierea, distribuirea sau ini=C8=9Bierea unor
ac=C8=9Biuni pe baza lui, f=C4=83r=C4=83 acordul meu scris, sunt strict
interzise =C8=99i atrag r=C4=83spunderea juridic=C4=83.
     Dac=C4=83 a=C8=9Bi primit acest mesaj din gre=C8=99eal=C4=83, v=C4=83=
 rog s=C4=83
m=C4=83 anun=C8=9Ba=C8=9Bi imediat, =C8=99i s=C4=83-l =C8=99terge=C8=9Bi =
apoi din sistemul
dumneavoastr=C4=83.
     Subliniez c=C4=83 orice comentarii pe care le fac reprezint=C4=83
punctul meu de vedere, =C8=99i nu cel oficial al Patriarhiei Rom=C3=A2ne.=


--------------000802070603010108040600
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>

<meta http-equiv=3D"content-type" content=3D"text/html; charset=3DUTF-8">=

</head>
<body bgcolor=3D"#ffffff" text=3D"#3366ff">
<font face=3D"Tahoma">=C2=A0=C2=A0=C2=A0 Bun=C4=83 ziua!<br>
<br>
=C2=A0=C2=A0=C2=A0 Aplica=C8=9Bia de pe Net care afi=C8=99eaz=C4=83 datel=
e meteo d=C4=83 o eroare la
selectarea tabelului =C3=AEn ordinea temperaturilor.<br>
=C2=A0=C2=A0=C2=A0 Mesajul este acesta: <font color=3D"#ff0000">Input arr=
ay is longer
than the number of columns in this table.</font><br>
=C2=A0=C2=A0=C2=A0 Browser-ul folosit este Mozilla Firefox 3.0.6 (de=C8=99=
i nu cred s=C4=83 aib=C4=83
vreo leg=C4=83tur=C4=83 cu eroarea).=C2=A0 <br>
=C2=A0=C2=A0=C2=A0 Sper c=C4=83 pute=C8=9Bi remedia c=C3=A2t mai repede a=
ceast=C4=83 problem=C4=83. <br>

<br>
=C2=A0=C2=A0=C2=A0 O zi bun=C4=83! <br>
</font>
<div class=3D"moz-signature">-- <br>
<meta content=3D"text/html; charset=3DUTF-8" http-equiv=3D"Content-Type">=

<title></title>
<div class=3D"moz-signature">
<div class=3D"moz-signature">
<div class=3D"moz-signature">
<div class=3D"moz-signature">
<div class=3D"moz-signature">
<div class=3D"moz-signature">
<div><b><span
 style=3D"font-size: 10pt; color: rgb(51, 153, 255); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';"><font
 color=3D"#3366ff">((Gabriel))</font><br>

</span></b><b><span
 style=3D"font-size: 8pt; color: red; line-height: 115%; font-family: 'Ta=
homa','sans-serif';"><u4:p></u4:p>editor-prezentator
RT</span></b><b><span
 style=3D"font-size: 7pt; color: rgb(217, 217, 217); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';"></span></b><span
 style=3D"font-size: 7pt; color: rgb(226, 226, 226); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';"></span><span
 style=3D"font-size: 7pt; color: rgb(226, 226, 226); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';"><br>
=C2=A0=C2=A0=C2=A0 Acest
mesaj este confiden=C8=9Bial =C8=99i adresat doar <br>
destinatarului, precum </span><span
 style=3D"font-size: 7pt; color: rgb(226, 226, 226); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';">=C8=99i</span><span
 style=3D"font-size: 7pt; color: rgb(226, 226, 226); line-height: 115%; f=
ont-family: 'Tahoma','sans-serif';">
altor persoane autorizate <br>
s=C4=83-l primeasc=C4=83.<br>
=C2=A0=C2=A0=C2=A0 Dezv=C4=83luirea, copierea, distribuirea sau ini=C8=9B=
ierea
unor <br>
ac=C8=9Biuni pe baza lui, f=C4=83r=C4=83 acordul meu scris, sunt strict <=
br>
interzise =C8=99i atrag r=C4=83spunderea juridic=C4=83. <br>
=C2=A0=C2=A0=C2=A0 Dac=C4=83 a=C8=9Bi primit acest mesaj din gre=C8=99eal=
=C4=83, v=C4=83 rog s=C4=83 <br>
m=C4=83 anun=C8=9Ba=C8=9Bi imediat, =C8=99i s=C4=83-l =C8=99terge=C8=9Bi =
apoi din sistemul <br>
dumneavoastr=C4=83. <br>
=C2=A0=C2=A0=C2=A0 Subliniez c=C4=83 orice comentarii pe care le fac
reprezint=C4=83<br>
punctul meu de vedere, =C8=99i nu cel oficial al Patriarhiei Rom=C3=A2ne.=
 </span></div>

</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>

--------------000802070603010108040600--

    So it is UTF-8. I always send and receive as UTF-8.

Comment 11

[Alexandru Szasz]

Alright, I think the problem is that mailnews.send_default_charset was set in ro/mail/chrome/messenger/messenger.properties to ISO-8859-2. So it *may* be only a problem caused by the Romanian translation.

But I still find it odd that the value in messenger.properties would be used to save a message.

I changed it to UTF-8. I'm waiting for the builds to test again the windows build. I've tested locally with Thunderbird under wine and the language pack changed and the file was saved correctly this time.

What version do you have exactly ? e.g. 3b1, 3b3pre

Comment 12

[((Gabriel)) Tudor]

3b3pre. Please post a message here when this problem is fixed.

Comment 13

[Alexandru Szasz]

Alright, please try to reproduce the bug with this version: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-central-l10n/thunderbird-3.0b3pre.ro.win32.installer.exe

Comment 14

[((Gabriel)) Tudor]

It's still happening.

Comment 15

[Alexandru Szasz]

For the record, I tried reproducing the bug by creating a new message, pasting the text and saving before sending as you said in the first message. 

It looks like we need a thunderbird developer here anyway.

Comment 16

[((Gabriel)) Tudor]

Yes, and somebody should solve this issue fast, as it's something important.

Comment 17

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to comment #15)
> For the record, I tried reproducing the bug by creating a new message, pasting
> the text and saving before sending as you said in the first message. 

And what happened?

Comment 18

[Alexandru Szasz]

Sorry, I think I was on the wrong track.

I can reproduce the bug this way:
- compose an email with the text from comment #1
- send it
- view it either in Sent, or in Inbox if I sent it to myself
- choose Save as and select under the file name Text file

Any other format saves correctly, it seems that just the saving as text file
messes the file up.

Comment 19

[Dan Mosedale (:dmosedale, :dmose)]

Setting to NEW, based on Alexandru's confirmation.  It sounds like this might be a regression; adding qawanted in the hopes of finding that out.  Assigning to philor, in the hopes that he'd be interested in a taking a run at it (I'm hoping it's reproducible on English versions of Windows).

Comment 20

[Magnus Melin [:mkmelin]]

Isn't this just (a variant of) bug 181456?

Comment 21

[Alexandru Szasz]

Yes, and it seems more detailed there.

Comment 22

[Magnus Melin [:mkmelin]]

Duping then.