This is blocker bug for JP testers. ISO-2022-JP encoder may be broken on trunk. If a line ends non-ASCII character, the state should be changed to ASCII. However, the current trunk doesn't append the state change to the end of the line. Therefore, Japanese testers cannot use trunk Tb. We should fix this bug before Tb3b1 release. Simon, can you take this?
Ah, I meant that we don't send Japanese email when the *message body* has non-ASCII characters and one or more lines end with non-ASCII character.
OS: Windows Vista → All
Hardware: PC → All
Created attachment 341129 [details] testcase Steps to reproduce: 1. Open the testcase with Fx. 2. Save the testcase as text. Expexted: Saved as ISO-2022-JP Actual: Saved file is broken. regression range: 20080604-20080605 Maybe Bug 399257 caused this.
adding some mail developers to cc list. If this bug is not fixed on Tb3b1, Japanese testers cannot test it. I hope that this bug blocks the b1 release, if Simon or someone can fix this bug soon.
(In reply to comment #3) > adding some mail developers to cc list. > > If this bug is not fixed on Tb3b1, Japanese testers cannot test it. I hope that > this bug blocks the b1 release, if Simon or someone can fix this bug soon. If it helps, Tb3b1 is now going to be TB 3 Alpha 3. Not sure if there is anyone we can chase about this. I don't know enough about how character sets work to take this on.
Saved data of test case by "Save Page As/Test Files" on MS Win-XP SP3 : (A) Saved ISO-2022-JP data(in hexa-decimal) by : [App] Name=Firefox, Version=3.1a1pre, BuildID=2008060403 [Gecko] MinVersion=1.9.1a1pre, MaxVersion=1.9.1a1pre ( [ESC]$B : Escape sequence to double-byte ) ( [ESC](B : Escape sequence to single-byte ) ( [CRLF] : 0x0d0a ) > [ESC]$B [ESC](B [CRLF] [ESC]$B [ESC](B [CRLF] [CRLF] > 1B2442 2422 2424 2426 2428 242A 1B2842 0D0A 1B2442 242B 242D 242F 2431 2433 1B2842 0D0A 0D0A (B) Saved ISO-2022-JP data(in hexa-decimal) by : [App] Name=Firefox, Version=3.1a1pre, BuildID=2008060504 [Gecko] MinVersion=1.9.1a1pre, MaxVersion=1.9.1a1pre > 1B2442 2422 2424 2426 2428 242A 000D 000A 242B 242D 242F 2431 2433 000D 000A 000D 000A As seen in above, following phenomena are observed. (1) "Escape sequence to single-byte" is lost, when no 7bit-ascii at line end (2) [CRLF](0x0d0a) is changed to 0x000d + 0x000a (3) Due to (1), double-byte data in next line is put without escape seq to double-byte This causes cut-off(due to illegal data) of mail body text lines in ISO-2022-JP at "lack of esc to single-byte", upon all Thunderbird trunk builds after upgrade to Gecko 1.9.1a2pre. >(Regression range of Tb trunk) > (no problem) > Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:220.127.116.11pre) > Gecko/2008072303 Shredder/3.0a2pre > (problem started to occur) > Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a2pre) > Gecko/2008072500 Shredder/3.0a2pre The "cut-off" means not only "loss of mail body text data" but also "loss of following / last [CRLF]". So it can produce "never ending mail send". - Tb sends data before first 000D/000A in above corrupted mail body lines, and sends final ".[CRLF]" without preceding [CRLF], and waits for OK from SMTP server forever. - Because of no "[CRLF].[CRLF]" from Tb, SMTP sever waits for additional data and final "[CRLF].[CRLF]" forever. Fault of converter from Unicode to ISO-2022-JP? Or fault of caller of converter?
Created attachment 341280 [details] [diff] [review] patch [Checkin: Comment 14] Bug 399257 fix is valid only for 1-byte charsets. With this patch, I could save attachment 341129 [details] as text. This patch also contains automated test case.
Assignee: smontagu → VYV03354
Status: NEW → ASSIGNED
Attachment #341280 - Flags: review?(smontagu)
Thunderbird would really like to have this for our a3 build, but we would like to cut the final build in the next day or so, so anything people can do to drive this fix in, assuming it's good, as soon as possible, would be much appreciated.
jshin, if you're reading this, and could review the patch before Simon gets to it, that would be very helpful. The patch looks pretty simple...
I should be able to review this by tomorrow. I just want to confirm that there are no regressions in other encodings.
I'll be offline for a while. Feel free to modify the patch if you find any problems and want to meet the deadline. FYI, this patch passed all xpcshell tests.
Comment on attachment 341280 [details] [diff] [review] patch [Checkin: Comment 14] r+moa=smontagu
Attachment #341280 - Flags: review?(smontagu) → review+
Comment on attachment 341280 [details] [diff] [review] patch [Checkin: Comment 14] this looks reasonable to me.
Attachment #341280 - Flags: superreview+
Comment on attachment 341280 [details] [diff] [review] patch [Checkin: Comment 14] http://hg.mozilla.org/mozilla-central/rev/d2f88b94140d
Attachment #341280 - Attachment description: patch → patch [Checkin: Comment 14]
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.1b2
You need to log in before you can comment on or make changes to this bug.