Fixed the append by copying the the python patch. Fixed in Bug 1571672. (In reply to Alessandro Vesely from comment #17) > Created attachment 9156903 [details] > Daily 79.0a1 (2020-06-14) worsened the appearance of UTF-8 fields > > See bug 1571672 comment 34 Now back to this. On reply, forward, edit-as-new, edit draft etc, any reflected address that contains non-ascii utf-8 bytes (between 0x7f and 0xff) are seen as invalid and appears at a ? inside a triangle. If the email is sent, these bytes are send as the 3 byte "replacement character" sequence 0xef, 0xbf, 0xbd (U+FFFD). But if the at least one of the characters has a code-point greater than U+00FF, such as U+01FF or ǿ, the bytes are processed as char16's instead of bytes. With my fix for imap UTF8=ACCEPT in mozilla code, the "lost" upper bytes that were discarded are retained and the reflected address displays correctly and are sent correctly as UTF8. Without my imap change, having code points greater then U+00FF in the address doesn't help (still get replacement chars) because the upper bytes are discarded in the mozilla code. This fixes address chars with with at least 1 code point greater than U+00FF. It doesn't help when all code points are non-ascii less than U+01FF: attachment 9166880 [details] [diff] [review] Anyhow, I'm kind of stuck here and don't know where these "reflected" address are written. Even if I did, not sure that would help.
Bug 1563891 Comment 23 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Fixed the append by copying the the python patch. Fixed in Bug 1571672. (In reply to Alessandro Vesely from comment #17) > Created attachment 9156903 [details] > Daily 79.0a1 (2020-06-14) worsened the appearance of UTF-8 fields > > See bug 1571672 comment 34 Now back to this. On reply, forward, edit-as-new, edit draft etc, any reflected address that contains non-ascii utf-8 bytes (between 0x7f and 0xff) are seen as invalid and appears at a ? inside a triangle. If the email is sent, these bytes are send as the 3 byte "replacement character" sequence 0xef, 0xbf, 0xbd (U+FFFD). But if at least one of the characters has a code-point greater than U+00FF, such as U+01FF or ǿ, the bytes are processed as char16's instead of bytes. With my fix for imap UTF8=ACCEPT in mozilla code, the "lost" upper bytes that were discarded are retained and the reflected address displays correctly and are sent correctly as UTF8. Without my imap change, having code points greater then U+00FF in the address doesn't help (still get replacement chars) because the upper bytes are discarded in the mozilla code. This fixes address chars with with at least 1 code point greater than U+00FF. It doesn't help when all code points are non-ascii less than U+01FF: attachment 9166880 [details] [diff] [review]. The proposed changes to mozilla code are described here: bug 1571672 comment 51. Anyhow, I'm kind of stuck here and don't know where these "reflected" address are written. Even if I did, not sure that would help.
Fixed the append by copying the the python patch. Fixed in Bug 1571672. (In reply to Alessandro Vesely from comment #17) > Created attachment 9156903 [details] > Daily 79.0a1 (2020-06-14) worsened the appearance of UTF-8 fields > > See bug 1571672 comment 34 Now back to this. On reply, forward, edit-as-new, edit draft etc, any reflected address that contains non-ascii utf-8 bytes (between 0x7f and 0xff) are seen as invalid and appears at a ? inside a triangle. If the email is sent, these bytes are send as the 3 byte "replacement character" sequence 0xef, 0xbf, 0xbd (U+FFFD). But if at least one of the characters has a code-point greater than U+00FF, such as U+01FF or ǿ, the bytes are processed as char16's instead of bytes. With my fix for imap UTF8=ACCEPT (bug 1571672) in mozilla code, the "lost" upper bytes that were discarded are retained and the reflected address displays correctly and are sent correctly as UTF8. Without the UTF8=ACCEPT change in mozilla code, having code points greater then U+00FF in the address fails (reflects replacement chars or the wrong char) because the upper byte of any UTF-16 char are discarded in the mozilla code. This fixes address chars with with at least 1 code point greater than U+00FF. It doesn't help when all code points are non-ascii less than U+01FF: attachment 9166880 [details] [diff] [review]. The proposed changes to mozilla code are described here: bug 1571672 comment 51. Anyhow, I'm kind of stuck here and don't know where these "reflected" address are written. Even if I did, not sure that would help.