622057 - Structured plain text (*/_foo_/*) is not displayed as formatted (bold, italic, underline) when special character (even accents) begins or ends the string

Status: RESOLVED DUPLICATE of bug 106028

(Thunderbird :: Message Reader UI, defect)

Description

[dot.cdrc]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13
Build Identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b9pre) Gecko/20101229 Thunderbird/3.3a2pre

Happens in each charset :
latin1
latin9
utf-8

Reproducible: Always

Steps to Reproduce:
1.write a text and format it with asterisks, slashes, underscores
2.save it in your drafts or send them to yourself
3.read and see
Actual Results:  
not formatted

Expected Results:  
formatted

Sometimes slashes (italic format) don't work.
I have to delete & rewrite the slashed word with slashes to make it formatted.

Comment 1

[Ludovic Hirlimann [:Usul]]

Are you sending plain text , html or in mixed mode ?

Comment 2

[WADA:World Anti-bad-Duping Agency]

Ludo, "format it with asterisks, slashes, underscores" is "structured text" for text mail display. (mail.display_struct=true)

Is format=flowed used? (save as draft, and check mail source of draft mail)
If yes, are words you want to format kept in single line in mail source?

What is your usual charset for mail composition?
If DBCS charset, format=flowed is currently disabled in mail composition internally for such charset, because DelSp=yes is not supported yet in mail composition by Tb. But format=flowed is not automatically disabled for Unicode charset like UTF-8. 

What is special character used in the words which is not formatted as expected? 

Slash("/", italic) only problem? No problem with "*" or "_"?

Trunk only issue? No problem if Tb 3.1.x?

What character set do you call by "charset of latin1/latin9"?
Latin1/Latin9 in Unicode definitions?

Comment 3

[Natalia]

The same problem occurs with Roman and Cyrillic alphabets (UTF-8 and Windows-1251).
The words are not kept in single line - I've tried each format (bold, italic, underline) in new line and no one of them represents correct in received and sent letter.

Comment 4

[Natalia]

The same problem occurs with Roman and Cyrillic alphabets (UTF-8 and Windows-1251).
The words are not kept in single line - I've tried each format (bold, italic, underline) in new line and no one of them represents correct in received and sent letter.

Comment 5

[Natalia]

Attachment: Incorrect representation of the fonts

Comment 6

[Natalia]

In "View" menu "Message Body as Original HTML" is chosen. In this case Actual Result represents as Expected Result plus special characters "*", "/", "_" in the beginning and ending of the word.
If "Message Body as Plain Text" is chosen than Actual Result represents as plain text with special characters as well.
I use Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> Incorrect representation of the fonts

Which part of display is the "Incorrect representation of the fonts"?
Can you save the mail in .eml file and attach the .eml file to this bug?

Comment 8

[Natalia]

Attachment: Incorrect representation of the font style

Comment 9

[Natalia]

Attachment: Incorrect representation of the font styles

Comment 10

[Natalia]

Attachment: Incorrect representation of the font styles

Comment 11

[Natalia]

(In reply to comment #7)
> (In reply to comment #5)
> > Incorrect representation of the fonts
> 
> Which part of display is the "Incorrect representation of the fonts"?
> Can you save the mail in .eml file and attach the .eml file to this bug?

I beg your pardon, I misled you in some way - saying "font" I meant font styles: bold, italic, underline. In that attachment they're in the red circle.

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
> saying "font" I meant font styles: bold, italic, underline.
> In that attachment they're in the red circle.

What's wrong in screen shot of "Incorrect representation of the fonts" attached to comment #5?
*Bold* is shown in bold and /Italic/ is shown in italic, according to spec of "Structured Text", as expected.
Please never confuse structured text in text/plain with <B>bold</B> and <I>Italic</I> in text/html.

As you said next in comment #0, I assumed text mode composition and you typed *Bold* /Italic/ at text mode composition window. 
> Steps to Reproduce:
> 1.write a text and format it with asterisks, slashes, underscores
Did you compose in HTML mode and change attribute to Bold or Italic and send without change Option/Format to "Rich Text(HTML) Only" or "Plain and Rich Text(HTML)"?

Comment 13

[Natalia]

(In reply to comment #12)

> What's wrong in screen shot of "Incorrect representation of the fonts" attached

The words themselves are represented correct in every font styles except slaches, asterixes and underlines which can confuse user. Are they really should be represented? Aren't they superfluous?  

> Did you compose in HTML mode and change attribute to Bold or Italic and send
> without change Option/Format to "Rich Text(HTML) Only" or "Plain and Rich
> Text(HTML)"?

I composed in HTML mode using ctrl+b (or ctrl+i) to change to Bold or Italic and send with Option/Format=Auto-Detect.
Everything works fine with Option/Format="Rich Text(HTML) Only" and "Plain and Rich Text(HTML)".

Comment 14

[dot.cdrc]

(In reply to comment #1)
> Are you sending plain text , html or in mixed mode ?

Plain Text.

The problem that Nathalia reported seems to be a differnt problem than the one I reported...

If you see the image in the URL : http://www.cedricmenassa.be/share/bug_thunderbird_plain_text.png you'll see that the first sequence is correct and other ones are not. Begining or ending a sequence with special characters including accents doesn't format the text.

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #13)
> > Did you compose in HTML mode and change attribute to Bold or Italic and send
> > without change Option/Format to "Rich Text(HTML) Only" or "Plain and Rich
> > Text(HTML)"?
> I composed in HTML mode using ctrl+b (or ctrl+i) to change to Bold or Italic
> and send with Option/Format=Auto-Detect.
> Everything works fine with Option/Format="Rich Text(HTML) Only" and "Plain and
> Rich Text(HTML)".

Next in comment #0 is for original problem of this bug's bug summary; "plain text is not formatted (bold, italic, underline) when special character (even accents) begins or ends the string". 
> Steps to Reproduce:
> 1.write a text and format it with asterisks, slashes, underscores
Original problem of comment #0 and bug summary is for "composition in text mode using structured text", as [e-mail](bug opener) says in comment #14.

<B>Bold</B> and <I>Italic</I> in text/html part is converted to *Bold* and /Italic/ in text/plain part by Tb when composed in HTML mode.
Because Tb has problem of bug 414299/bug 603474, mail is sent as text/plain even when you composed mail in HTML mode if text only mail, unless you select Option/Format="Rich Text(HTML) Only" or "Plain and Rich Text(HTML)".
(See also bug 533680)
This is reason why Ludovic Hirlimann asked question of Comment #1.

Natalia, please never confuse phenomenon with next.
(a) text/plain mail composed in text mode, using structured text by you.
(b) text/html mail or part composed in HTML mode.
(c) text/plain mail or part composed in HTML mode, which is converted from
    data in text/html part by Tb using structured text. 
Please rule out phenomenon due to bug 414299/bug 603474, which is essentially irrelevant to this bug's structured text related problem which occurs in text mode commposition.

Comment 16

[WADA:World Anti-bad-Duping Agency]

[e-mail](bug opener), can you attach mail data to this bug, in order to avoid confusion on this bug's problem?

Comment 17

[dot.cdrc]

Attachment: tested on Shredder/3.3a3pre

not correct when beginning or ending with special characters including accents which are very common in other languages than English.

Comment 18

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #17)
> Created attachment 509738 [details]

For next cases which produces this bug.
> *%test*
> *$test*
> *testé*

If "A" is inserted between starting "*" and special char, or between special char and ending "*", string between "*" was shown in bold as expected by Tb 3.1.7 on Win.   
> *A%test*
> *A$test*
> *testéA*

Confirming per attached test case.

Comment 19

[WADA:World Anti-bad-Duping Agency]

Some additional tests for ascii only case.

                       Bold by *   Underscore by _   Italic by /
***te st*               X           N/A               N/A           
*A**te st*              O           N/A               N/A

*_/te st*               X           N/A               N/A
*A_/te st*              O           N/A               N/A              

*_/te st/_*             X           X                 O 
*A_/te st/_A*           O           X                 O
*A_B/te st/B_A*         O           X                 X
*A _B /te st/ B_ A*     O           O                 O
   shown like <B>*A <UNDERLINE>_B <I>/te st/</I> B_</UNDERLINE> A*</B>

(i) Special character after *(_ and /) doesn't start formatting.
(ii) To close formatting, "non usual character in a word" after *(_ and /) looks required. "non usual character in a word" in this case is space, new line character, some non-alpha-numeric characters.
It looks spec of structured word.

Comment 20

[WADA:World Anti-bad-Duping Agency]

FYI.
Tb 3.1.7 generated next in text/html part of HTML mail for bold, underlined, italic.
> <b>**te st</b><br>
> <u>__te st</u><br>
> <i>//te st</i><br>
And, Tb 3.1.7 generated next in text/plain part by converter.html2txt.structs=true. html2txt converter of Tb looks simply adding *, _, /, for bold, underline, italic, respectively. 
> ***te st*
> ___te st_
> ///te st/
As you know, these are currently not shown in bold, underlined, italic by mail.display_struct=true, if View/Message Body As/Plain Text.

Comment 21

[dot.cdrc]

(In reply to comment #18)
> (In reply to comment #17)
> If "A" is inserted between starting "*" and special char, or between special
> char and ending "*", string between "*" was shown in bold as expected by Tb
> 3.1.7 on Win.   
> > *A%test*
> > *A$test*
> > *testéA*

Okay... Will be intended to "correct" this in the next release ?
I'm not comfortable to put an "A" before or after any special characters. As a French speaker, I use accents.

What I see now, numbers are also affected by this behavior.

In purpose to dispel any misunderstandings, can you tell me if that's a correct email client's behavior or that is specific to TB. I'm not an email guru, so what I reported as a bug, can be a normal behavior.

Comment 22

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #21)
> I'm not comfortable to put an "A" before or after any special characters. As a
> French speaker, I use accents.

"Structured word" or "formatted text" looks for 7bit ascii word only.
"*" for Japanese text like next doesn't show it in bold. (日本語 is Japanese Kanji for Japanese word coresponds to English word of Japanese.)
> *日本語* * 日本語 * * 日 AA 本 AA 語 *

> In purpose to dispel any misunderstandings, can you tell me if that's a
> correct email client's behavior or that is specific to TB.

I can say nothing about correct behaviour of mail.display_struct=true, because I know nothing about spec of mail.display_struct=true.
As for HTML2Text conversion of Tb, I believe that developer who wrote HTML2Text converter of Tb probably thought "any text between * is shown as bold by mail.display_struct=true", as you and me thought.

> I'm not comfortable to put an "A" before or after any special characters. As a
> French speaker, I use accents.

Workaround is;
  1. converter.html2txt.structs=false
  2. Compose mail in HTML mode, and request bold/underlined/italic of text
     if you want.
  3. Upon send, change Options/Format to "Rich Text(HTML) Only" or "Plain and
     Rich Text(HTML)" from default of "Auto Detect"(bypass of bug 414299),
     if you want to send text/html part even text only mail.
  Many mailer including Tb shows text/html part by default.
  Even if recipient loves text mode display, there is no problem because
  "structured word" or "formatted text" is not generated in text/plain part
  by converter.html2txt.structs=false of mail sender side Tb.
I believe mail sender shouldn't force mail recipient to use "structured word" or "formatted text" in text mail viewing.

Comment 23

[Răzvan Sandu]

Hello,

My feeling tells me it's the same bug - please analyze this:

When writing a text mode message using Romanian diacritics, especially these (but not only):

- Unicode 0218 - capital S with comma below
- Unicode 0219 - small s with comma below
- Unicode 021A - capital T with comma below
- Unicode 021B - small t with comma below

and putting a word containing diacritics between stars, to bold it ( like *thisțthis* ), the word don't appear bolded.


Best regards,
Răzvan

Comment 25

[Thomas D. (:thomas8)]

In spite of the useful research done here, I think it's better to consolidate in original bug 106028 where we already collected 15 duplicates because of a too technical summary which users can't find when they report this.