23324 - int. chars in fields of vCard seems to render Japaneese chars.

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect, P3)

Description

[Henrik Gemal]

I just looked at the new vCard stuff and everything seems to work just fine with
int. chars except the "address" field. which seems to render japaneese chars or
something like that.
My adr vcard line:
adr;quoted-printable:;;Fruebjergvej
3=0D=0A1=0D=0A2=0D=0A3=0D=0A4=0D=0A=E6;Copenhagenזרו;DKזרו;2100זרו;Denmarkזרו
note;quoted-printable:1=0D=0A2=0D=0A3=0D=0A4=0D=0A5=0D=0A6=0D=0A

Renders fine on Netscape 4.7:
Fruebjergvej 3 1 2 3 4 ז

But on Mozilla gives me:
Fruebjergvej 3 1 2 3 4 (Some WEIRD CHAR) DT>

Using build 2000010608

Comment 1

[Henrik Gemal]

I've changed the subject since it happens in more than the address fields.
It also happens in:
- Nickname
- Pager
- Home
- Notes

Comment 2

[nhottanscp]

21743 is for vCard Japanese view. Can this be dup of that?

Comment 3

[Henrik Gemal]

Attachment: vCard with Danish chars in it which renders into japanesse chars!

Comment 4

[nhottanscp]

Okay, I can see "זרו" in some fields.
21743 is a viewing problem. This bug seems to be a compose/send problem.
Was the vCard generated by 5.0?

Comment 5

[Henrik Gemal]

The vCard was created with 4.7!
Forgot to include the content type, etc...:

Content-Type: text/x-vcard; charset=us-ascii;
 name="gemal.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for Henrik Gemal
Content-Disposition: attachment;
 filename="gemal.vcf"

Ok. the whole problem is in the difference between the encoding of these two
fields:
tel;quoted-printable;home:=E6=F8=E5
tel;cell:([phone]זרו

Comment 6

[nhottanscp]

Cound you send a mail to [e-mail] with the vCard?
To make sure the problem is not 4.x compose problem.

Comment 7

[nhottanscp]

The vCard generated by 4.7 has a problem. The part's content type says it's
us-ascii and trasfer encoding is 7 bit while the actual data contains 8 bit
Latin1. This is the same problem as 21743.
Since 4.7 can view this kind of vCard, 5.0 also needs to understand it.

Comment 8

[nhottanscp]

After I received the vCard from Henrik, I saw it does not contain 8 bit Latin1.
Those were QP encoded so no inconsistency between the charset label and the
actual data. And I saw the string turned to Japanese.
Looks like the string "=E6=F8=E5" is treated as UTF-8 (one Japanese character)
instead of three Latin1 characters. Something is wrong in the vCard handling.
Adding rhp to cc.

Comment 9

[rhp (gone)]

Something got changed with the data being fed to the vCard handler. In the
past, it was never converted to UTF-8, but something changed so that for much
of the data, I am getting UTF-8. I stopped doing converstion in the vCard
converter because of this.

This is probably where the problem lies.

- rhp

Comment 10

[nhottanscp]

When I apply a charset conversion in OutputBasicVcard, I was able to see some of
the fields. What I did was convert the result string of fakeCString from
"ISO-8859-1" to "UTF-8" using INTL_ConvertCharset(). But the charset name should
be taked from the content type instead of hard coded. It may be cleaner to
change fakeCString to take charset and generate UTF-8 string.
Rich, could you take a look at this bug?

Comment 11

[rhp (gone)]

Are you sure there wasn't a charset on the MIME part for this attached vCard?
Maybe we are doing charset converstion from the wrong charset?

- rhp

Comment 12

[Katsuhiko Momoi]

Do we know how to deal with incorrect MIME charset info
on vCards? In addition to us-ascii for QP'ed ISO-8859-1 data,
we had also in the past had a problem of JPN data incorrectly
labeled as "iso-2022-jp" though the data was in Shift_JIS.
Lately 4.71 does seem to encode this correctly but some earlier
Communicator versions did not. Can we possibly run JPN vCard
through an auto-detector rather than trusting MIME charset info.

Comment 13

[nhottanscp]

Reply to Rich's question, yes it has a charset (may be not correct as momoi
mentioned). I was just lazy so used hard coded string because I was just trying
to check my assumption (missing charset conversion) was correct.

Comment 14

[rhp (gone)]

Ok, I have a fix for this in my tree...I just have to see if I like it or not
and if it is the best way to approach this. Basically, we shouldn't be
converting quoted-printable text in libmime to UTF-8, then pass it through to
the vcard content type handler and trying to decode the quoted printable in
there. I just have to see which approach we should be taking.

- rhp

Comment 15

[rhp (gone)]

Should be fixed now.

- rhp

Comment 16

[Katsuhiko Momoi]

** Checked with 1/24/00 Win32 build **

Latin 1 vCard and in particular the one gemal provided is now displaying OK.
We have problems with Japanese vCard. 
Should we file a new bug or try to deal with that here?

Comment 17

[rhp (gone)]

Can you send me an example of a problem vCard before you file a new bug.

- rhp

Comment 18

[nhottanscp]

There is a bug filed for the Japanese vCard problem, 21743.

Comment 19

[Katsuhiko Momoi]

We'll deal with the Japanese problem in Bug 21743, then.
Marking this particular fix verified given the above results.