69170 - Splitting MIME-encoded words does not include meaningful SP

Status: NEW

(MailNews Core :: MIME, defect)

Description

[Katsuhiko Momoi]

** Observed with 1/16/2001 Mtrunk Build **

Under conditions that have to do with the length of 
the first encoded word in the To: header field, the
second encoded may be split into 2. When there is
an original sequence that contains a meaningful Space
character, that character needs to be encoded since
spaces between encoded words are not meaningful. 
But we don't encode such SP character. Instead,
we produce a sequence of the following kind:

-----
To: =?ISO-2022-JP?B?GyRCRVo6NBsoQiAbJEI4JBsoQg==?= 
	<[e-mail]>,
	=?ISO-2022-JP?B?GyRCPSlFRBsoQg==?=
	 
	=?ISO-2022-JP?B?GyRCOCQbKEI=?= <[e-mail]>
-----

where between the split encoded words, we have 

0x0D 0x0A 0x09 0x20 0x0D 0x0A 0x09

There seem to be a couple of things wrong with this
sequence:

1. The meaningful SP (0x20) is not encoded and a program can 
    thus ignore it.
2. It is inserting a blank line (CRLF) with a TAB at the end
    then followed by the SP and CRLF and a TAB.

I'm not sure what should be the correct way to split
such encoded words is in case, the line breaking
forces a split.
Please investigate what if any we can do about this.

Comment 1

[Katsuhiko Momoi]

Assigning this to nhotta initially.

Comment 2

[Katsuhiko Momoi]

To re-create this problem, you can send a message
using the 2 strings I attach in an HTML file as
the Address line data for outgoing mail. These
contain bogus addresses and so may get an error msg
but you can save a copy in a local folder and examine
the source of such a msg.

Comment 3

[Katsuhiko Momoi]

Attachment: Data file in HTML containing 2 Jpn address strings. The data was used originally for a NS internal bug.

Comment 4

[Katsuhiko Momoi]

CC;ing a bunch of iqa people.

Comment 5

[Katsuhiko Momoi]

The relevant RFC is 2047. See in particular the following
section:

5. Use of encoded-words in message headers

It seems clear that if the SP is meaningful, then it
must be part of one of the encoded words in this case.

In the attached test strings, the Japanese names (LAST <SP> FIRST)
are separated by a single SP character.

Comment 6

[John G. Myers]

The endcoder appears to be using intlmime_encode_next8bitword() in order to 
ensure spaces get encoded when necessary.  It is not clear why this isn't 
working correctly.

Comment 7

[Katsuhiko Momoi]

I've confirmed that neither 

=?ISO-2022-JP?B?GyRCPSlFRBsoQg==?=

nor 

=?ISO-2022-JP?B?GyRCOCQbKEI=?=

contains the original SP character which
separated the Last name from First name. In the
Message envelope pane, the orignla "AB C" in Chinese
is displayed as "ABC" when this problem occurs.

Comment 8

[Katsuhiko Momoi]

Just a wild guess but when there is a sequence of the
following type:

AB C <[e-mail]>

where A, B, C are Japanese words and need to be MIME-encoded.

When A, B, C are encoded, somehow, the header line folding occurs 
in such a way as to cause a split right where the "AB" sequence
ends, somwhow the following SP is not encoded but used as a separator
between the 2 encoded words. 
Note the the first address-to line contains nearly identical 

XY Z  <[e-mail]>

But this does not get split and contains the SP in the
encoded portion.

Comment 9

[Frank Tang]

both naoki and I are off mozilla for about 20 monthes. If these bugs are still
here, the real status is 'wont fix'. If you want to reopen it, please find a
better owner who really looking at the bug database now. 

Comment 10

[Travis Chase]

Mass Reassign Please excuse the spam

Comment 11

[Travis Chase]

Mass Re-opening Bugs Frank Tang Closed on Wensday March 02 for no reason, all
the spam is his fault feel free to tar and feather him

Comment 12

[Travis Chase]

Reassigning Franks old bugs to Jungshik Shin for triage - Sorry for spam