Cannot send ISO-2022-JP email on Tb trunk

RESOLVED FIXED in mozilla1.9.1b2

Status

()

P2
blocker
RESOLVED FIXED
10 years ago
10 years ago

People

(Reporter: masayuki, Assigned: emk)

Tracking

({fixed1.9.1, jp-critical, regression})

Trunk
mozilla1.9.1b2
fixed1.9.1, jp-critical, regression
Points:
---
Bug Flags:
blocking1.9.1 +
in-testsuite +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

This is blocker bug for JP testers.

ISO-2022-JP encoder may be broken on trunk. If a line ends non-ASCII character, the state should be changed to ASCII. However, the current trunk doesn't append the state change to the end of the line. Therefore, Japanese testers cannot use trunk Tb. We should fix this bug before Tb3b1 release.

Simon, can you take this?
Flags: blocking1.9.1?
Ah, I meant that we don't send Japanese email when the *message body* has non-ASCII characters and one or more lines end with non-ASCII character.
(Reporter)

Updated

10 years ago
Keywords: jp-critical
OS: Windows Vista → All
Hardware: PC → All

Comment 2

10 years ago
Created attachment 341129 [details]
testcase

Steps to reproduce:
1. Open the testcase with Fx.
2. Save the testcase as text.

Expexted: Saved as ISO-2022-JP
Actual: Saved file is broken.

regression range: 20080604-20080605
Maybe Bug 399257 caused this.
adding some mail developers to cc list.

If this bug is not fixed on Tb3b1, Japanese testers cannot test it. I hope that this bug blocks the b1 release, if Simon or someone can fix this bug soon.
(In reply to comment #3)
> adding some mail developers to cc list.
> 
> If this bug is not fixed on Tb3b1, Japanese testers cannot test it. I hope that
> this bug blocks the b1 release, if Simon or someone can fix this bug soon.

If it helps, Tb3b1 is now going to be TB 3 Alpha 3. Not sure if there is anyone we can chase about this. I don't know enough about how character sets work to take this on.
Saved data of test case by "Save Page As/Test Files" on MS Win-XP SP3 :
(A) Saved ISO-2022-JP data(in hexa-decimal) by :
    [App]   Name=Firefox, Version=3.1a1pre, BuildID=2008060403
    [Gecko] MinVersion=1.9.1a1pre, MaxVersion=1.9.1a1pre
    ( [ESC]$B : Escape sequence to double-byte )
    ( [ESC](B : Escape sequence to single-byte )
    ( [CRLF]  : 0x0d0a )
> [ESC]$B                         [ESC](B [CRLF] [ESC]$B                         [ESC](B [CRLF] [CRLF]
> 1B2442 2422 2424 2426 2428 242A 1B2842  0D0A   1B2442 242B 242D 242F 2431 2433 1B2842  0D0A   0D0A

(B) Saved ISO-2022-JP data(in hexa-decimal) by :
    [App]   Name=Firefox, Version=3.1a1pre, BuildID=2008060504
    [Gecko] MinVersion=1.9.1a1pre, MaxVersion=1.9.1a1pre
> 1B2442 2422 2424 2426 2428 242A 000D 000A             242B 242D 242F 2431 2433 000D 000A 000D 000A

As seen in above, following phenomena are observed.
 (1) "Escape sequence to single-byte" is lost, when no 7bit-ascii at line end
 (2) [CRLF](0x0d0a) is changed to 0x000d + 0x000a
 (3) Due to (1), double-byte data in next line is put without escape seq
     to double-byte

This causes cut-off(due to illegal data) of mail body text lines in ISO-2022-JP at "lack of esc to single-byte", upon all Thunderbird trunk builds after upgrade to Gecko 1.9.1a2pre.
>(Regression range of Tb trunk)
> (no problem)
>   Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.2pre)
>   Gecko/2008072303 Shredder/3.0a2pre
> (problem started to occur)
>   Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a2pre)
>   Gecko/2008072500 Shredder/3.0a2pre
The "cut-off" means not only "loss of mail body text data" but also "loss of following / last [CRLF]". So it can produce "never ending mail send".
  - Tb sends data before first 000D/000A in above corrupted mail body lines,
    and sends final ".[CRLF]" without preceding [CRLF],
    and waits for OK from SMTP server forever.
  - Because of no "[CRLF].[CRLF]" from Tb, SMTP sever waits for additional data
    and final "[CRLF].[CRLF]" forever.

Fault of converter from Unicode to ISO-2022-JP? Or fault of caller of converter?

Comment 6

10 years ago
Backing out of Bug 399257 fixes this.
Blocks: 399257
(Assignee)

Comment 7

10 years ago
Created attachment 341280 [details] [diff] [review]
patch
[Checkin: Comment 14]

Bug 399257 fix is valid only for 1-byte charsets.
With this patch, I could save attachment 341129 [details] as text.
This patch also contains automated test case.
Assignee: smontagu → VYV03354
Status: NEW → ASSIGNED
Attachment #341280 - Flags: review?(smontagu)

Updated

10 years ago
Assignee: VYV03354 → smontagu
Status: ASSIGNED → NEW

Updated

10 years ago
Assignee: smontagu → VYV03354
Status: NEW → ASSIGNED

Comment 8

10 years ago
Thunderbird would really like to have this for our a3 build, but we would like to cut the final build in the next day or so, so anything people can do to drive this fix in, assuming it's good, as soon as possible, would be much appreciated.

Comment 9

10 years ago
jshin, if you're reading this, and could review the patch before Simon gets to it, that would be very helpful. The patch looks pretty simple...
I should be able to review this by tomorrow. I just want to confirm that there are no regressions in other encodings.
(Assignee)

Comment 11

10 years ago
I'll be offline for a while.
Feel free to modify the patch if you find any problems and want to meet the deadline.
FYI, this patch passed all xpcshell tests.
Whiteboard: [tb3needs]
Comment on attachment 341280 [details] [diff] [review]
patch
[Checkin: Comment 14]

r+moa=smontagu
Attachment #341280 - Flags: review?(smontagu) → review+

Comment 13

10 years ago
Comment on attachment 341280 [details] [diff] [review]
patch
[Checkin: Comment 14]

this looks reasonable to me.
Attachment #341280 - Flags: superreview+
(Assignee)

Updated

10 years ago
Keywords: checkin-needed
Comment on attachment 341280 [details] [diff] [review]
patch
[Checkin: Comment 14]

http://hg.mozilla.org/mozilla-central/rev/d2f88b94140d
Attachment #341280 - Attachment description: patch → patch [Checkin: Comment 14]
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Flags: in-testsuite+
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.1b2

Updated

10 years ago
Flags: blocking1.9.1? → blocking1.9.1+
Priority: -- → P2
Whiteboard: [tb3needs]
(Assignee)

Updated

10 years ago
Keywords: fixed1.9.1
You need to log in before you can comment on or make changes to this bug.