1210244 - Please give us option to send mail which I composed in HTML mode as HTML mail, even when Delivery Format=Auto-Detect is used or forced, even when gMsgCompose.bodyConvertible() says Convertible (Please approve patch proposed in this bug or in Bug 1204379)

Status: VERIFIED WONTFIX

(MailNews Core :: Composition, enhancement, P5)

Description

[WADA:World Anti-bad-Duping Agency]

This bug is spin-off of Bug 1204379 Comment #107.

Please give us option to send mail which I composed in HTML mode as HTML mail, 
  even when Delivery Format=Auto-Detect is used or forced,
  even when gMsgCompose.bodyConvertible() says Convertible,
  even when we forgot to change Unknown to HTML for all contacts in all Address Book,
  even when we mixed HTML recipients and Text recipients and Unknown recipients.

Plese see "User Story" of Bug 1204379 for current Tb's behavior upon send, and possible one-liner patch.
Note: An issue in this patch.
  If the patch is applied, and if user chooses HTML in Send Options,
  when news post is composed in HTML mode,
  (note: default mode is HTML in Tb unless user change account settings)
  it's posted as HTML even when gMsgCompose.bodyConvertible() kindly says convertible.
  Then, many news users say "Thunderbird posted HTML news posting!!!",
  instead of saying to the USER "HTML posting is not so good manner, please stop it".

Comment 1

[WADA:World Anti-bad-Duping Agency]

Note:
Bug report which starts with "I composed HTML mail, but sent as text/plain..." looks closed as dup of bug 414299 which is about "<tt> and friends" in many cases, even when "text only in Body Text or PRE only" case.

Comment 2

[WADA:World Anti-bad-Duping Agency]

FYI.
Well known War in News :
(1) Text post vs. HTML post War (still Text is preferred by many news users)
(2) (a) "quote, then body text" vs. (b) "body text, then quote(or attach)" War
    In text mail or news posting, users usually prefer (a) for ease of understanding conversation history,
    with accepting bad readability of message body due to long/deep quoting.
    However, in busines mail, "send referred mail as attachment" is frequently used(Lotus Notes did so).
    This is : most important statements is message body text, so write it first concisely.
    So, (b) is used in many cases even when text only mail(not multipart/mixed).
    Then, War happened, and perhaps still continues(endless war...)

Comment 3

[WADA:World Anti-bad-Duping Agency]

Note:
There already is simple way to force "send as HTML", if user want it when user composed mail in HTML mode.
1. Delivery Formaat=HTML
2. Use Hn, P, Inser Image/Link/Table etc., background color, text color, Bold/Italic etc. 
   which are ctuaally exists in Tb's HTML maaail composition window,
   because creating "HTML Mail".
3. Change "Unknown" in Address Book to "HTML" or "HTML Domain".
4. At Send Options, HTML(or Both) or Ask with replying HTML or Both at dialog, instead of Text.

However, when Delivery Format=Auto-Detect is used, there is small window of "mail composed in HTML is sent as text/plain with UNKNOWN reason".
Because of UNKNOWN reason for user, it's mystery for user. If text only in Body Text/PRE only, this mystery can be observed always with "Unknown recipient type" or with "mixed recipient type".
And, the mystery for user produced not-so-small numbee of bug opens at bugzilla.mozilla.org around Auto-Detect feature.

Comment 4

[WADA:World Anti-bad-Duping Agency]

About News posting.

Auto-complete at To:, Newsgroup: etc., behaviour in Send, is currently as follows.
(a) When Composer is invoked using Identity for Email account.
    Show contact in Address Book.
    If email address and Newsgrop: is mixed, following error message is shown.
       This message is for email recipients only.
(b) When Composer is invoked using Identity for News account.
    Show newsgroup under news account where the used Identity is defined.
    If HTML mode, and if email address and Newsgrop: is mixed, Dialog by Send Options is shown(Ask is set).

Is allHTML, allPlain, Unknown etc. applied to News posting? (Newsgroup:, Followup-To:)
If yes, how it's obtained from where?

I believe Tb user almost always invokes Composer via Identity for News account, and user usually set "default composition mode=Text" in news account setting if user wants Text posting at Newsgroup of the news server, and user usually set "default composition mode=HTML" in news account setting if user wants HTML posting at Newsgroup of the news server.

Comment 5

[WADA:World Anti-bad-Duping Agency]

If NewsGroup is contained in recipients, and if it's composed in HTML mode, send format is Asked.
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#4879
> 4879 nsMsgCompose::DetermineHTMLAction(int32_t aConvertible, int32_t *result)
> 4886   // Right now, we don't have logic for newsgroups for intelligent send
> 4887   // preferences. Therefore, bail out early and save us a lot of work if there
> 4888   // are newsgroups.
> 4889   if (!newsgroups.IsEmpty())
> 4890   {
> 4891     *result = nsIMsgCompSendFormat::AskUser;
> 4892     return NS_OK;
> 4893   }

Unless user force Thunderbird to send HTML post, Thunderbird won't send HTML post to news server.
If Newsgroup/Followup-To is contained in recipients of mail with Identity of email account, Thunderbird rejects to send it.
Nice! There is no need to care News case.

Comment 6

[WADA:World Anti-bad-Duping Agency]

Look into acman's patch attached to bug 1204379 again.
One-liner patch :
> -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
> +  if (allPlain && (aConvertible == nsIMsgCompConvertible::Plain))
This is applicable to Email only, because always asked if "HTML news posting" is involved.

Table in user Story of bug 1204379("changed by the one-liner patch" only)
> ------  ----------------  --------------------------------------  ----------------------------------------
> CaseID  Action by Tb      <------ Corresponding Condition ----->
>           Should be                        aConvertible                             (* = chnged by patch)
>             user option?  allHtml allPlain ==Convertible::Plain   Current Action  Action proposed by patch
> ------  ----------------  ------- -------- --------------------   --------------  ------------------------
> (B-2)    ???              false   true     false                  SF::PlainText   *UseSendOptions     
> ------  ----------------  ------- -------- --------------------   --------------  ------------------------
> (C-1)    ???              false   false    true                   SF::PlainText   *UseSendOptions     
> ------  ----------------  ------- -------- --------------------   --------------  ------------------------
> 
> Current Action in (C-1) is cause of user's confusion,
> cause of following funny phenomenon == mystery for user.
>   Even though user actually composed mail in HTML mode,
>   the mail was sent as plain/text by Tb silently, with unknown reason.

When gMsgCompose.bodyConvertible() dosn't say "Covertible to Text" && all recipients is Text Domain recipients,
  Before patch : send as text/plain silently
  After  patch : use Send Options(sked if Asked set == delivery format is determined by user)
When gMsgCompose.bodyConvertible() says "Covertible to Text" && mixed recipients
     (combination of two or more in "HTML, Text, Unknown)", never "Unknown only" case)
  Before patch : send as text/plain silently  <= This is the mystery. Sent as text/plain by UNKNOWN reason.
  After  patch : use Send Options(sked if Asked set == delivery format is determined by user)

What's wrong in the one-liner patch?

Comment 7

[WADA:World Anti-bad-Duping Agency]

Note:
Issue when the patch is applied.
- Because Send Options has only one set of user options,
  same option setting is applied to all of (B-2) and (C-1) and other cases which uses Send Options.
  So, if user chooses Both, multipart/alternative is sent regardless of gMsgCompose.bodyConvertible() value.
- When user chooses Ask, because user can not know reason why asked,
  and because there is no way to know "which case in Table of User Story of bug 1204379",
  delivery format selection is slightly hard for user when asked for send format.

Comment 8

[WADA:World Anti-bad-Duping Agency]

My thoughts:
- RFC says "multipart/alternaative is for mailer who can't handle text/html part".
  But, if mailer can understand multipart/alternative, I believe such mailer usualy can process HTML well.
  And, I belive that many mailers can show HTML mail as Text by converting HTML to Text by himself
  to protext user of the mailer from Spam/Scam.
  Why "send Both HTML and Text" in Send Options" is mandatory?
  Delivery Formt=Both HTML and Text in menu" is sufficient, isn't it?
  Objective of multipart/alternative is for simply "order of preference of sender", isn't it?
  Is such "order of preference" actually needed in HTML mail sent from Thunderbird? 
- If gMsgCompose.bodyConvertible() doesn't say Convertible, HTML is not so simple.
  i.e. text/plain part is not so usefull for both mail sender and many of mail recipients.
  Why "text converted from HTML by Thunderbird" shoud be sent in addition to text/html part?
  If text/plqin part is sufficient for mail recipients, 
  what is reason to compsee mail in HTML mode with many formatting by HTML Tag and/or many Styles?
  This is Email for easy/quick communication between human beings, isn7t it?
- If gMsgCompose.bodyConvertible() says Convertible, HTML is pretty simple.
  So, there is no big difference between text/html and text/plain,
  for both *MAIL RECIPIENTS" and mailer who can handle HTML.
  Why text/plain should be forced by Tb merely by gMsgCompose.bodyConvertible() says Convertible?
- I believe that "When an mailer(and user) *SHOULD* send HTML mail as text/plain" is following case only.
  - Mailer of recipients can't understand HTML mail.
  - Recipient never accept HTML mail or rejects HTML mail in order to protect from Spam/Scam etc.
  I believe that "HTML vs Text War in Email world" already ended.

Comment 9

[WADA:World Anti-bad-Duping Agency]

My thoughts-2:
- I think "Unknown" is as follows, although different from some other peoples.
  - Text Domain was for next initially.
    - Mailer of recipients can't understand HTML mail.
    And following was added.
    - Recipient never accept HTML mail or rejects HTML mail in order to protect from Spam/Scam etc.
    I believe that "if (allPlain), always send as text/plain" part of current code is for this "Text Domain".
  - HTML Domain was "Can surely accept HTML. ie. Can accept Text too.
  - Unknown in this case == Not in Text Domain nor HTML Domain == "HTML can be accepted or not" is Unknown.
                         == It may be "Text Domain" which can't handle HTML or rejects HTML.
- However, after "message preference in Address Book" was added,
  following were generated in user's interpretation of "HTML/Text Domain" and "Address Book Settings"
  due to term of "Prefers" in Address Book.
  (a) Text : (a-1) Can handle Text only, 
             (a-2) Prefers Text(i.e HTML can be received too),
  (b) HTML : (b-1) Can handle HTML(can receive Text too), but prefers HTML.
  (c) Unknown in this case == (a-1) or (a-2) or (b-1) is Unknown.

Comment 10

[Magnus Melin [:mkmelin]]

(In reply to WADA from comment #6)
> -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
> +  if (allPlain && (aConvertible == nsIMsgCompConvertible::Plain))
> What's wrong in the one-liner patch?

Because allPlain is usually false (unknown user preference), even when it's convertible to plain we would not do that after the patch.

Having looked at this some more, I guess the logic should really be
  
  if (aConvertible == nsIMsgCompConvertible::Plain)
    return nsIMsgCompSendFormat::PlainText;
  if (allPlain && aConvertible == nsIMsgCompConvertible::No)
    return nsIMsgCompSendFormat::AskUser; 

... so that we don't silently convert to plain if we can't (but the user prefers plain)

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #10)
> Having looked at this some more, I guess the logic should really be
>   
>   if (aConvertible == nsIMsgCompConvertible::Plain)
>     return nsIMsgCompSendFormat::PlainText;

If it's for mixed recipients case(allHTML==false&&allPlain==false), this is (C-1).
   Note: mixed case = Unknown Only, HTML+Unknown, Text+Unknown, HTML+Text*Unkown, HTML+Text
         Proposal of "User's default choice for Unknown in Send Options"
         is to change mixed case to "HTML+Text only".
Current action=nsIMsgCompSendFormat::PlainText == same as your proposal.
Patch propses "use Send Options". In current Send Options, user can choose Ask, Both, HTML Only, Text Only.
My request is:
  Current action is cause of mystery of "silently sent as text/plain by UNKNOWN reason".
  This mystery troduced bugs such as:
    "we need other default than Delivery Format=Auto-Detect",
    "Give us option to kill Auto-Detect",
    "<tt> and some friends should be kept",
  and produced useless discussion like "lossless or lossy".
  If user sets "HTML in Send Options" and if Tb uses "Send Opyions" in this case,
  mystery won't occur.
  If user sets "Text in Send Options", "by UNKNOWN reason" is changed to "by Understandable reasion"
  because user requets it in "Send Options". So myster disappears.
  Thus, unwanted bug open, excess commants to bugs, will be avoided.

Why "Send Options" should not be used in this case?

>   if (allPlain && aConvertible == nsIMsgCompConvertible::No)
>     return nsIMsgCompSendFormat::AskUser; 

This is (B-2). Current Action=nsIMsgCompSendFormat::PlainText. After patch, changed to useSendOption.
In current Send Options, user can choose Ask, Both, HTML Only, Text Only.

Why Ask should be forced in this case?

Comment 12

[WADA:World Anti-bad-Duping Agency]

Bug repoert for phenomenon of "I composed mail in HTML mode but it's silently sent as text/plain by unknown reason..." is periodically opened although number of such bugs is not so large. However, number of such bug reports is not so small.
Magnus, do you want to increase number of bugs in dependency tree for meta bug 889315?
Do you want to increase number of comments in bugs listed in dependency tree for meta bug 889315?

As I wrote in comment #9, meaning of "Text Domain" seems changed by "Prefers in Address Book".
What do you think?

Comment 13

[WADA:World Anti-bad-Duping Agency]

Code for allPlain case is identicaal to following
>         Current Code                                       Your proposal  
> (B-1) if      (allPlain && aConvertible == nsIMsgCompConvertible::Plain)
>         return nsIMsgCompSendFormat::PlainText;            Same as curreny 
> (B-2) else if (allPlain && aConvertible == nsIMsgCompConvertible::No)
>         return nsIMsgCompSendFormat::PlainText;            return nsIMsgCompSendFormat::AskUser; 

Why current "return nsIMsgCompSendFormat::PlainText;" in (B-2) is bad?

I think why "return nsIMsgCompSendFormat::PlainText when allPlain regardless of aConvertible value" is currently done is "Text Domain == recipient who can receive Text mail only".
So, if allPlain, Tb is better to send as text/plain as done currently.
Was meaning of "Text Domain" changed by "Prefers in Address Book"?
Please note that I'm not opssite to nsIMsgCompSendFormat::AskUser or "use Send Options" in (B-2) because user can choose send format in this case.
I'm merely asking about "there is any reason to change in allPlain case, or not".

Comment 14

[WADA:World Anti-bad-Duping Agency]

Magnus, why following is bad or wrong?
  If allHtml,       send as text/html,  (note: same as current)
  Else if allPlain, send as text/plain, (note: same as current)
  Else (== mixed case, including Unknown only) use Send Options which has option of Ask, Both, HTML, Text.
      (note: in aConvertible == nsIMsgCompConvertible::No case, same as current)
      (      changed in aConvertible == nsIMsgCompConvertible::Plain case only )
aConvertible value is not used any more?
If aConvertible value is not used, there is no need to call gMsgCompose.bodyConvertible() any more?

I believe aConvertible value can be utilized in Send Options panel, in Composer Window(Convertible or not indictor), in Ask diaog etc. It'll help user's decision on send format at Ask dialog etc.

Comment 15

[WADA:World Anti-bad-Duping Agency]

Attachment: (already rejected by Ben, don't comment on it, please) {Auto-downgrade only for allPlain} - Pseudo one-liner patch created by altering aceman's patch attached to Bug 1204379

Comment 16

[WADA:World Anti-bad-Duping Agency]

FYI.
"if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)"	line was generated by great sorting out work of bug 970118 for mail sending logic.
And "Otherwise check preference part == Use Send Options part" looks to have been introduced by that bug, because "otherwise ask" like comment is seen in comment of source before bug 970118.
> http://hg.mozilla.org/comm-central/log/258f0a389e63/mailnews/compose/src/nsMsgCompose.cpp
> http://hg.mozilla.org/comm-central/rev/642cfedbc637#l5.560
Before the sorting out, logic is too complicated for me...

Comment 17

[WADA:World Anti-bad-Duping Agency]

Joshua Cranmer(opener/assignee of bug 970118), is next change acceptable?
-  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
+  if (allPlain)
   {
     *result = nsIMsgCompSendFormat::PlainText;
     return NS_OK;
   }
   // Otherwise, check the preference to see what action we should default to.

Comment 18

[Magnus Melin [:mkmelin]]

(In reply to WADA from comment #14)
> Magnus, why following is bad or wrong?
>   If allHtml,       send as text/html,  (note: same as current)
>   Else if allPlain, send as text/plain, (note: same as current)

This is a dataloss case. It should be asked if allPlain and we can't really convert.

>   Else (== mixed case, including Unknown only) use Send Options which has
> option of Ask, Both, HTML, Text.

That means you'd per current prefs just silently send both even if there's no formatting at all.

Comment 19

[Magnus Melin [:mkmelin]]

Comment on attachment 8669272 [details]
(already rejected by Ben, don't comment on it, please) {Auto-downgrade only for allPlain} - Pseudo one-liner patch created by altering aceman's patch attached to Bug 1204379

In addition to not fixing the dataloss case, you're basically making plaintext a no-op even if there's no formatting at all.

Comment 20

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #18)
> >   Else (== mixed case, including Unknown only) use Send Options which has
> > option of Ask, Both, HTML, Text.
> That means you'd per current prefs just silently send both even if there's
> no formatting at all.

"Silently" part is same, but "by UNKNOWN reason" part is changed to "by Understandable reason" because user sets it in "Semd Options" panel. So, mystery for Tb user disappears.
What's wrong in "User want it, so Tb simply sends as user wants"? :-)

Comment 21

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #19)
> Comment on attachment 8669272 [details]
> Pseudo patch created by altering aceman's patch attached to Bug 1204379
> 
> In addition to not fixing the dataloss case,

But it's same as current. Current logic is identicl to "if(allHtml) send as text/html, else if(allPlain), send as text/plain, else if(mixed && Convertible), send as text/plain, else if(mixed), use Send Option, else {;}".
I don't think current logic when allText should be modified.
Definition of "Text Domain" was changed?
I though that "Text Domin == recipient can't understand HTML, so Tb kindly sends mail as text/plain when user forgot his Text Domain setting" :-)

In this case(allPlain && !Convertible), "Asked or use Send Options" is better nowadays?
(note: this is proposal by aaceman)

> you're basically making plaintext a no-op even if there's no formatting at all.

You are correct. My proposal is "Stop referring to aConvertible value before passing control to Send Options" and "Utilize aConvertible value under control of Send Options".

Anyway, I can't understand reason why you rejected aceman's patch proposed to Bug 1204379...
It sounds foe me "your comments in this bug" == "what acemen does in tht patch"...

Comment 22

[Magnus Melin [:mkmelin]]

(In reply to WADA from comment #21)
> But it's same as current. 

Like I said, current design is dataloss for allPlain. That's in practice a very rare case so I think it would make sense to actually ask (not just follow mail.default_html_action).

> Definition of "Text Domain" was changed?

No but it's always been *prefers* plaintext.

Comment 23

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #22)
> > Definition of "Text Domain" was changed?
> No but it's always been *prefers* plaintext.

If so, why "if(allPlain), send as text/plain" part in "if(Convertible||allPlain), send as text/plain" was used in Thunderbird for so long time and is currently used?
Please note that change by bug 970118 done in 2014 is simply a sorting out work. (see comment #16, please)

Please note that I'm never opposite to "Ask or use Send Options in such case" which is proposed by aceman and is recommended by you.
I simply want to know "Why".

Comment 24

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #22)
> (In reply to WADA from comment #21)
> > But it's same as current. 
> 
> Like I said, current design is dataloss for allPlain. That's in practice a　very rare case
> so I think it would make sense to actually ask (not just follow mail.default_html_action).

Please see Table in "User Story".
Modifiable case by simple change is (B-2) and (C-1) only.
I prefer your proposal.
  My proposal produces "Data Loss" in (B-2).
  aceman's proposal may force statement changes in Send Options panel,
  becusee curent Send Options is for "mixed case".
  If your proposal, change in Send Option panel is not required, becuse of "Simply Ask" as done in News case.

For (C-1), "UseSendOptions" is mandatory for this bug to resolve "myster for user" problem.

Comment 25

[Joshua Cranmer [:jcranmer]]

(In reply to WADA from comment #17)
> Joshua Cranmer(opener/assignee of bug 970118), is next change acceptable?

Bug 970118 was an attempt to replace a really crappy do-everything API with more sensible, finer-grained API calls and did not attempt to change any logic (although I did try to simplify the logic to make it easy to figure out the basic decision without hurting your brain).

> -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
> +  if (allPlain)

As I've said in another bug (and I'm not going to try to find the reference because there are already far too many comments on this topic and having to revisit them to find just one message is too depressing), the problem that you're essentially trying to dance around is that the convertible parameter isn't reliable. The solution then is to fix the preparation of that parameter to be accurate.

I know that some people feel that's untenable, but here's the basic situation:
* Most people don't change the prefer mail preferences. These are basically remnants of a by-gone era when some clients literally couldn't handle HTML email and sending HTML email would be problematic. So effectively, these usually provide absolutely 0 signal.

* By specific intention, the code is designed to avoid asking the user whether or not HTML email should be down-converted to plain text.

* Plain text is generally preferred to HTML when the same information can be conveyed losslessly.

I consider it a minimum, non-negotiable requirement than any change in this arena preserve the property that, in a default install, the user can send an email to a previously unknown recipient (and reply to a plaintext email) that was composed in the HTML editor that used no extra formatting, and we will send that email as a plain text email without prompting the user.

Comment 26

[Thomas D. (:thomas8)]

(In reply to Joshua Cranmer [:jcranmer] from comment #25)
> (In reply to WADA from comment #17)
> > Joshua Cranmer(opener/assignee of bug 970118), is next change acceptable?
> 
> Bug 970118 was an attempt to replace a really crappy do-everything API with
> more sensible, finer-grained API calls and did not attempt to change any
> logic (although I did try to simplify the logic to make it easy to figure
> out the basic decision without hurting your brain).

Joshua, thanks for that plain-text reply (pun intended ;P)...

> > -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
> > +  if (allPlain)
> 
> As I've said in another bug (and I'm not going to try to find the reference
> because there are already far too many comments on this topic and having to
> revisit them to find just one message is too depressing), the problem that
> you're essentially trying to dance around is that the convertible parameter
> isn't reliable. The solution then is to fix the preparation of that
> parameter to be accurate.

Yes, it's necessary to improve the isConvertible algorithm, which currently has a massive potential of non-lossless conversion although it claims convertible::plain (which requires lossless conversion).

However, improving the isConvertible algorithm isn't sufficient for all types of users and use cases.

1) No matter how much we finetune the algorithm, converting from HTML to plaintext is NEVER entirely lossless conversion. E.g. sending HTML vs. plain usually affects variable with vs. fixed width font settings for receiver. Yes, we can't entirely control them, but sending HTML has *more chances* of getting the same rendering between sender and receiver. E.g. dropping even a simple <p> and replacing that with \n\n involves a loss of logical paragraph definition *and* a slight change/loss of visual appearance details, which are pretty predictable with HTML because it's strongly standardized.

2) I think we can agree that plaintext vs. HTML war is basically over. HTML email format is now definitely at least equally acceptable as plaintext for the vast majority of scenarios, clients etc. Short of saying HTML won (as it's the universal format which drives the whole WWW), there's good reason to assume that HTML format is de facto used for the overall majority of messages sent.

3) In view of 1) and 2), there's no reason why our default settings should *force* users to accept that their messages, composed in HTML mode, even to HTML recipients if mixed with "unknown" recipients, should get downgraded to plaintext without ways of opting out. Instead, we should let the user define (regardless of the amount of formatting used), if he prefers to send messages to "unknown" recipients in plaintext, OR in HTML, OR both. I'd suggest a simple dropdown in send options to do just that.
 
> I know that some people feel that's untenable, but here's the basic
> situation:
> * Most people don't change the prefer mail preferences. These are basically
> remnants of a by-gone era when some clients literally couldn't handle HTML
> email and sending HTML email would be problematic. So effectively, these
> usually provide absolutely 0 signal.

+1

> * By specific intention, the code is designed to avoid asking the user
> whether or not HTML email should be down-converted to plain text.

Problem being that code was designed when "Ask" was still default; but now we default to "Send both".
Again this boils down to giving the user a choice IF he generally prefers plain text at all; users who prefer HTML send format should NOT be affected by workarounds/tweaks which we implemented for the downgrade-to-plaintext cases at a time where we still had a massive bias for plaintext.

> * Plain text is generally preferred to HTML when the same information can be
> conveyed losslessly.

Lossless conversion is currently a BIG lie; after improving the algorithm, as shown above, it will still NEVER be entirely lossless, so it's still a small lie. Which is acceptable IF we provide simple way of opt-in/opt-out for so-called "lossless conversion".

> I consider it a minimum, non-negotiable requirement than any change in this
> arena preserve the property that, in a default install, the user can send an
> email to a previously unknown recipient (and reply to a plaintext email)
> that was composed in the HTML editor that used no extra formatting, and we
> will send that email as a plain text email without prompting the user.

Hmmm. I'm not exactly sure how to read that statement; there seems to be a semantic conflict between "non-negotiable" and "can" (which implies opt-in/opt-out).

It's OK if the user CAN send messages that way if he so wishes; it's definitely NOT OK to *force* everyone into that downgrading behaviour without ways of opt-in/opt-out (as we do now), and without giving any indication about downgrading in the UI.

And again, it's not clear who defines what is "extra formatting". I don't know such animal. I only know formatting, and ANY HTML TAG is formatting; virtually all tags effect *visual* formatting; at least *logical* formatting which is one of the main points of using HTML (so even when wysiwyg might not strictly apply, there's still a loss of logical information when we drop even simple things like <p>, <div>, <pre>, <tt> etc.).

So in a nutshell, I consider it non-negotiable that any delivery format decision that has the slightest potential of being non-lossless (i.e. every downconversion from HTML to plaintext-only delivery format) must be OPTIONAL by way of general user-definable settings.

Finally, assuming that you're arguing for default settings which include auto-downgrading if so-called "lossless conversion" is deemed possible: WHY should that be the *default* behaviour for HTML composition? Iow, why do you think that for cases of little formatting (never no formatting), sending plaintext is superiour over sending HTML?

Comment 27

[Thomas D. (:thomas8)]

(In reply to Magnus Melin from comment #18)
> (In reply to WADA from comment #14)
> > Magnus, why following is bad or wrong?
> >   If allHtml,       send as text/html,  (note: same as current)
> >   Else if allPlain, send as text/plain, (note: same as current)
> 
> This is a dataloss case. It should be asked if allPlain and we can't really
> convert.

***** Proposal ***********************
+1. Aceman, please note that Magnus (like me) is in favor of preventing the currently uncontrolled dataloss here. Not sure why WADA disagrees. We could limit current send options to "allPlain" case so the user can decide if we wants to accept silent dataloss or not. Which imo will work very well if we just add a simple dropdown where user can define preferred format for "prefers-unknown" between plain|html|both, and then we can apply auto-detect decisions accordingly.
**************************************

> >   Else (== mixed case, including Unknown only) use Send Options which has
> > option of Ask, Both, HTML, Text.
> 
> That means you'd per current prefs just silently send both even if there's
> no formatting at all.

Current send options and behaviour are nonsense because they treat "unknown" case same as "prefers-plain" case, plus get it wrong and datalossy for mixed cases, so that even explicit "prefers-html" will be ignored if mixed. "unknown" should never be treated same as "plain", because "unknown" can be "prefers-html" (*OR* prefers-plain). As shown above in my comment 26, "no formatting at all" is a myth for any HTML formatted composition, so we need to give users a choice at least if they are willing to accept automatic downgrading or not. Sending "both plain+html" is a legitimate scenario even for "unknown" + "hardly any formatting".

Predictability is another factor. Many modern users who are not aware of plaintext vs. HTML war will find it surprising that their HTML composition gets sent as plaintext under certain circumstances, but not others. Imho auto-downgrading should no longer be the default nowadays, even for very little formatting after we improve the isConvertible algorithm.

Comment 28

[Thomas D. (:thomas8)]

And/or we can consider adding a simple checkbox for disabling auto-downgrade to plaintext when little formatting:

Settings for delivery format auto-detect        [that scope headline is currently missing!!!]

[ ] When there's virtually no HTML formatting, and no recipient prefers HTML, send message as plain text instead.

Comment 29

[Thomas D. (:thomas8)]

WADA has quite rigidly advised me to stop mentioning "dataloss of formatting", probably because it's subject to contestation and personal preference for cases of little / less consequential formatting (where formatting really means *any* HTML tags).

Indeed, regardless of everyone's personal stance on "lossless conversion", I think the main point is that we are in no position to decide over the user's head which delivery format should be used for a given message when using auto-detect with "unknown" recipients, or formatting with allPlain recipients, etc.

By analogy:

How would you like it if your favorite Word processor had an inbuilt auto-downgrade algorithm that saves your Word.doc or OpenOffice.odt documents as plaintext.txt documents when there's "no formatting", and would NOT give you a default choice over that downgrading behaviour? OR:

How would you like if all your Word.doc or OpenOffice.odt documents were automatically "losslessly converted" to .pdf files instead upon saving, without giving you a default choice over that "conversion" behaviour?

So if we would never want that for our documents, why do we force such behaviour onto our users when composing HTML messages with HTML composition mode?
That's exactly the same kind of presumptuous behaviour which many of our users are no longer willing to accept. In fact, there's little difference between a simple word processor and our HTML composer.

Comment 30

[WADA:World Anti-bad-Duping Agency]

With skipping some comments.

(In reply to Joshua Cranmer [:jcranmer] from comment #25)

My question was: (see table in User Story of this bug, please)
  Is "change from send as text/plain to UseSendOptions in (C-1)" acceptable?
  (this is mandtory for this bug)
  If it's acceptable, is "keeping current logic in (B-2)" acceptable?
  Or action in (B-2) should be changed from "silently send as text/plain" to "Ask or UseSendOptions"?
  Note: Magnus says "it should be asked to user when (B-2)",
        and says "no need to use Send Options in this case".
        I agree with him.

In order to avoid useless discussion on "aCovertible value is correct or wrong", "lossless or lossy" and so on, This bug is limited to "text only in Body Text and/or PRE" case only, so "surely always Convertible" case.
What I need in this bug is proper solution to resolve this bug's issue in this bug's case.
So, "what is proper action in other case(not Convertible case)" is also needed.

Comment 31

[WADA:World Anti-bad-Duping Agency]

(In reply to Joshua Cranmer [:jcranmer] from comment #25)
> the problem that you're essentially trying to dance around is that
> the convertible parameter isn't reliable.
> The solution then is to fix the preparation of that parameter to be accurate.

This bug is for "text only in Body Text and/or PRE only" case only. In this case, "Convertible" is surely returned. Therefore, "convertible parameter isn't reliable, or is reliable" is absolutely irrelevant to this bug :-)

A Questio: (no need of answer)
  What is definition of "absolutely lossless conversion of HTML mail to Text mail"?

> * Most people don't change the prefer mail preferences. These are basically
> remnants of a by-gone era when some clients literally couldn't handle HTML
> email and sending HTML email would be problematic. So effectively, these
> usually provide absolutely 0 signal.

A possible solution is "user's default choice for Unknown at Send Options panel".
Because Unknown falls into one of HTML or Text, Unknown disappers. So, "mixed recipients" case === "HTML reciients + Text recipients" case only.
If "mix of HTML and Text", "Alwaays Ask, not use Send Options" is a reasonaable action, as currently done when HTML news posting case.
However, it's different issue or enhancement from this bug.

> * By specific intention, the code is designed to avoid asking the user
> whether or not HTML email should be down-converted to plain text.

I believe "avoid HTML mail vs. Text mail War" was also a design objective.
I think "use Send Options" is also a method to avoid asking user, because option of Text, HTML, Both are already provided at "Send Options" panel in addition to Ask.

> * Plain text is generally preferred to HTML when the same information can be conveyed losslessly.

But Tb users, who deeply love HTML mail and who never accept "HTML mail is sent as text/plain by Tb when Unknown", surely exist, as seen in bugs listed in dependency tree for meta bug 889315. Such users usually never accept or permit existence of exception in edge cases or rare cases.
I believe that "if user wants, Tb sends as user wants" is best practice for Thunderbird.
Even if "HTML mail vs. Text mail War" happend by it, even if network trafic is increased by it, even if disk space is wasted by it, even if "Advanced Search/Quick Search" doesn't work well by it, responsible person is never Tb. It's always user :-)

How about next?
  Drop down menu of Compose button etc. (similar to View/Messaage Body As)
    Text mode
    Simple HTML mail mode (same as current HTML mail composition)
    Force HTML mode (for heavy HTML user. Always send HTML mail. Generaates HTML similar to one by MS)
Similar to/same as "other default than Auto-Detect", "option to kill Auto-Detect" etcc.?  :-)

Comment 32

[:aceman]

I'm lost in all the reasoning here. Just please decide on some compromise and I implement it.
All I can say is I have also proposed "an option to send HTML mail based on the composing more, ignoring AutoDetect" (as in the title here) in bug 136502.

Comment 33

[WADA:World Anti-bad-Duping Agency]

To Magnus.
Please see Table in User Story of this bug.
As seen the table, relevant/affected cases are (B-2) = allPlain case and (C-1) = mixed case.
So, issue in (B--a2) and issue in (C1) is independent, and change in (B-2) won't affect (C-1) vice versa.
For (B-2), "format loss in allPlain && Not-Convertible", it doesn't look that final solution is found.
In contrast to it, final solution looks already found for (C-1).
And patch for (C-1) only is sufficient for this bug.
Number of affected users by (B-2) is not so large because allPlain case.
However, number of affected user by (C-1) is large, because of mixed case, and because mixed case is usually Unknown and user usually keeps Unknown in Address Book which is default.

Is next approach possible?
  step-1 : Apply patch for (C-1) only. Don't touch code for (B-2). Keep current logic for (B-2).
  step-2 : open separate bug for (B-2) = "format loss in allPlain && Not-Convertible".
  step-3 : Find final solution in the new bug for (B-2).
  step-4 : Apply patch for (B-2).

Or both (B-2) and (C-1), which are independent issues, should be resolved at once by single patch and single patch landing because both are code in single function in single feature?

Or sharing single "Send Options" panel between (C-1) and (C-2) will produce critical problem, because one is "mixed recipients && Convertible case" and another is "mixed recipents && Not-Convertible case"?

Or there is no need to apply patch for (C-1) because impact by problem of this bug is pretty low and is negligible, and because problem in (C-1) is negligible?

Comment 34

[WADA:World Anti-bad-Duping Agency]

To Thomas D.
Could you you attach actual mail which produces problem of this bug so frequently and produces so critical your "format loss" or your "data loss", please.
I usually test with "text only in Body Text and/or PRE" in order to surely force Convertible. So actual mail is needed to test patch.

Comment 35

[WADA:World Anti-bad-Duping Agency]

FYI.

(In addition to comment #11)
>  Proposal of "User's default choice for Unknown in Send Options"
>  is to change mixed case to "HTML+Text only".

"Capability to set default of Prefers in Address Book" was requested by Bug 56834 on 2000-10-16.
I didn't know that Prefers in Address Book was so old feature.
I surely couldn't be aware of importance of it, unless great sorting out of Send Logic by bug 970118, which was done in 2014, was implemented. "Default for Unknown by user" is mandatory to end useless discusstion on Unknown.

Comment 36

[WADA:World Anti-bad-Duping Agency]

Please look into pretty simple/clear logic after sorting out by bug 970118 in nsMsgCompose::DetermineHTMLAction(line number is as of today)
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#4879
> 4879 nsMsgCompose::DetermineHTMLAction(int32_t aConvertible, int32_t *result)
> 
> 4969   // If everyone supports HTML, then return HTML.
> 4970   if (allHtml)
> 4971   {
> 4972     *result = nsIMsgCompSendFormat::HTML;
> 4973     return NS_OK;
> 4974   }
> 4975 
> 4976   // If we can guarantee that converting to plaintext is not lossy, send the
> 4977   // email as plaintext. Also send it if everyone prefers plaintext.
> 4978   if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
> 4979   {
> 4980     *result = nsIMsgCompSendFormat::PlainText;
> 4981     return NS_OK;
> 4982   }
> 4983 
> 4984   // Otherwise, check the preference to see what action we should default to.
> 4985   nsCOMPtr<nsIPrefBranch> prefService(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv));

Comment 37

[WADA:World Anti-bad-Duping Agency]

To Thomas D.

Which HTML Type in [HTML mail types] in "User Story" of this bug do you call "format loss" or "data loss", in this bug, in your Bug 1202227 and Bug 1202276, and some other bugs listed in dependency tree for meta bug 889315?
How frequently does your "format loss" or "data loss" occur in your environment?
On how many mails does your "format loss" or "dataloss" occur?
What is Severity of your "format loss" or "data loss"?
  Blocker? Critical? Major? Normal? Minor? Trivial? Or Enhaancement?

Comment 38

[WADA:World Anti-bad-Duping Agency]

FYI.
Bug 389417 and Bug 766860 in Blocks: field of this bug is report for same phenomenon as (C-1) in "User Story" of this bug, which was fortunately wasn't closed as bug 414299, which are listed in dependency tree for meta bug 889315.

Comment 39

[WADA:World Anti-bad-Duping Agency]

Sorry, typo.
  fortunately wasn't closed as bug 414299 => fortunately wasn't closed as dup bug 414299

Comment 40

[WADA:World Anti-bad-Duping Agency]

FYI.
Typical simple HTML mail without formatting by HTML tag:
  (a) Body Text only
  (b) PRE only
  (c) P only
  (d) <a> with Link Text != Link URL
  (e) <a> with Link Text == Link URL
  (f) combination of above.
And, (d) was not member of "<tt> and friends" but (e) was member of "<tt> and friends".

Brief history of bug summaary of bug 414299. 
> 1. Initial bug summary(opened on 2008-01-27)
>    Messages composed with html email composer get sent as plain text
> 2. Bug summary set by [e-mail] on 2012-06-14.
>    Messages composed as HTML with <tt> or simple links get sent as text/plain format=flowed
> 3. Bug summary set by me on 2012-06-15.
>    Messages composed as HTML with <tt> or simple links get sent as text/plain,
>    because defaulted Options->Format->Auto-Detect is always applied
>    regardless of user's want/expectation, and sadly,
>    there is no way to change the default action...
> 4. Bug summary set by [e-mail] on 2012-06-15,
>    Messages composed as HTML with <tt> or simple links get sent as plain text
> 5. Current bug summary(set on 2012-06-21).
>    Format | Auto-Detect: Messages composed as HTML
>    with <tt> for "fixed width" sections
>    or with simple links get sent as plain text 

Bug summary on 2012-06-15 by me is similar to claim of this bug. 

I was not interested in that bug, because that bug is only for discussion on "what tag is convertible or not".
So, I wasn't aware of actual meaning of *simple* in "simple link".
I merely thought that <a> tag is considerd Convertible, because Tb(and many mailers too) has Auto-Linkify in text mail display.
And, when I checked with Insert Link, I couldn't observe problem always.
  I'm good HTML user, so I put appropriate Link Text always :-)

I guess that majority of simple HTML mail creators usually uses Copy&Paste for Link creation in HTML mail instead of "Insert Link".
I guess "(e) <a> with Link Text == Link URL" is a big reason why bug for "created HTML mail, sent as text/plain by Tb" is periodically opened without shouting "Format/Data Loss!!!".
In many cases, when bug opener knows simple/easy workaround in his bug, bug opener won't add comment any more, because his problem has been resolved. They usually don't complain repeatedly.
If "<tt> for fixed width sections or simple link" is major cause of phenomenon of "created HTML mail, sent as text/plain by Tb",
- I can't understand reason why comment sounds like "Format Loss!!!" or "Data Loss!!!" is seen 
  in some comments in some bugs which is relevant to Auto-Detect.
- I can't understand reason why problem is called "critical",
  even though a phenomenon, which can be called "a data loss, a format loss",
  is surely involved in phenomenon/issue.

Comment 41

[WADA:World Anti-bad-Duping Agency]

FYI.

According to Bug 414299 Comment #25 on 2010-06-26, following case seems design.
  link that is equal to the link text
Term of "simple link" was introduced to Bug Summary of that bug on 2012-06-14.

It looks for me.
1. Bug for change of Convertibility check logic exists.
2. Even after many improvements in Convertibility check logic, some cases remained.
3. So, "other default than Auto-Detct in Delivery Format(current term)" was requested.
5. And, "option to kill Auto-Detect" was requested.
6. Phenomenon of "created HTML mail, silently sent as plain/text by Tb with unknown reason"
   will disappears by one of 1. 3. 4.
   So, any bug report for next phenoenon falled into one of 1. 3. 4.
       "created HTML mail, silently sent as plain/text by Tb with unknown reason"

There are other possible/effective solutions.
(a) User choice of default of Unknow. (already requeed by Bug 56834 on 2000-10-16)
    By this, Unknown falls into one of HTML or Text => Unknown disappears.
(b) Improve send logic/send options.
    I couldn't find bug for this type of request. So I opened this bug.
    Found change to resolve problem was "one-liner patch".
    i believe that this was possible because sorting out of send logic by bug 970118 was implemented in 2014.

Comment 42

[WADA:World Anti-bad-Duping Agency]

FYI.

Because test case is needed to check patch if patch is accepted, I tried to find outstanding cases in bug 414299.
And, I found following.

According to History of this bug, it look for me;
 1. Bug was opened(2008-01-27).
 2. Closed as WONTFIX(2010-06-26), because of current design.
 3. As <tt> for Fixed Width case remained, re-opened(2010-06-26), and accepted(because not closed again).
 4. After it, term of "<tt>" and term of "simple link" were added to Bug Summary"(2012-06-14)
 5. Many comments, many attachmens were added to this bug.
 6. Recently, patch for "<tt>" and "style attribute of tags" were attached to this bug(latest is on 2015-09-17).

I think I could understand about "what had occurred on some bugs in dependency tree for meta bug 889315".
Many comments were perhaaps produced by following:
- if <tt> only(format loss!!!), or if simple link only(data loss!!!), sent as text/plain.
- if <style> in <head> + minimum tag for text in <body>,
  or if minimum tag in <body> + style atribute of the minimum tag,
  sent as text/plain(format loss!!!).

Comment 43

[WADA:World Anti-bad-Duping Agency]

Attachment: zyx

Comment 44

[WADA:World Anti-bad-Duping Agency]

Attachment: wvu

Comment 45

[WADA:World Anti-bad-Duping Agency]

Attachment: tsr

bug 414299 : <tt> for Fixed Width Font, <a> with link text==link url
bug 584313 : <style> in <head>, style attribute of tag

Comment 46

[WADA:World Anti-bad-Duping Agency]

FYI.

As easily known by Table in "User Story" of this bug, "Where problem is generated" is (C-1) in the Table.

There are 4 kinds of possible solution.

(1) Because of (allHtml==false && allPlain==false) case,
if Unknown falls into one of HTML or Text by "user's choice of default of Unknown",
(allHtml==false && allPlain==false) case is changed to "HTML Domain+Text Domain only".
Because of "HTML Domain + Text Domain only" && IsConvertible case, "silently send as text/plain" is a reasonble/acceptable action.

(2) Because Auto-Detect is relevant, if Auto-Detectt is killed, problem can't occur.

(3) Because of IsConvertible case, if it's changed to IsNotConvertible case, falls into (A-2)/(B-2)/(C-2). So, problem won't occur.

(4) Because problem is generated by action of SendAsText in (C-1), if action is changed to UseSendOptions, problem can be bypassed.

So, in addition to following,
  - Official method-1 : Delivery Format=Text only/HTML only/Both HTML and Text,
  - Official method-2 : Correctly set HTML Domain/Text Domain, Prefers=HTML/Text in Address Book
  - Known simple/easy workarounds, for example, msgcompose.background_color=#FEFEFE
there are 4 kinds of bug which can resolve problem.
> (1) Bug   56834 : Opened on 2000-10-16
>                   If user can choose default for Prefers=Unknown,
>                   Prefers=Unknown falls into one of HTML Domain or Text Domain.
>                   So Unknown disappears.
>                   Then problem due to Unknown will never occur.
> (2) Bug  136502 : Opened on 2002-04-09
>                   If Auto-Detect feature is not used or killed by option,
>                   or if other default action than Auto-Detect of Delivery Format menu can bet set by option,
>                   Auto-Detect is not used, then problem won't occur.
> (3) If HTML created by user is considered Not-Convertible-To-Text, problem won't occur.
>     Bug  414299 : Opened on 2008-01-27
>                   <tt> for Fixed Witdth Font, <a> with Link Text=Link URL 
>     Bug  584313 : Opened on 2010-08-04
>                   <style> in <head>, style attribute of tag
> (4) Bug 1210244 : Opened on 2015-09-30
>                   If send logic is modified,
>                   and if Send Options is referred before doing Auto-Downgrade-To-Text,
>                   user can send HTML mail as text/html by setting HTML in Send Options.
>                   (Surprizingly, one-liner patch was suffcient.)

This bug is proposal of the 4-th solution in above.

Comment 47

[WADA:World Anti-bad-Duping Agency]

To Magnus, I can accept "close this bug as WONTFIX or FUTURE by following reason".
  Send Options panel is currently designed/implemented for (C-2) only.
  "Mixing both (C-1) and (C-2) in current Send Options panel" may produce unwanted problem/bad side effect.
  So, without sufficient enhancement of "Send Options feature" and "Send Options panel",
  "Mixing both (C-1) and (C-2) in current single Send Options panel",
  or "Sharing current Send Options panel by (C-1) and (C-2)" is currently not acceptable.

If improvement in (C-1) only is acceptable, I'll open separate bug for 4-th possible solution(one-linet patch in send logic), and try to find best solution.
For (B-2), silently send as text/plain when (allPlain && Not-Convertible) case, if you　believe it's a "dataloss", please open separate bug for the "dataloss" problem in (B-2).

Comment 48

[WADA:World Anti-bad-Duping Agency]

To thomas D., please asnwer to comment #34 and coomment #37.
Because you posted many comemnts in Bug 1204379, and because you are bug opener of Bug 1202227 and Bug 1202276, and because you posted many cooments in your two bugs, I believe that you have actual mail data which produces critical/severe "data loss" and "format loss" pretty frequently in your environment.

Comment 49

[WADA:World Anti-bad-Duping Agency]

There are several types of HTML mail.

(1) Truly plain text HTML for Tb :
    No formatting, Text only.
    Note: Some users expect "Asked" in (1-3).
          They want "silently sent as text/plain by Auto-Detect" when PRE is not used,
          but they want text/html when PRE is used,
          because they want display by Fixed Width Font for PRE text.
(1-1) No tag
   <body bgcolor="#FFFFFF" text="#000000">
    TextWithVariableWidthFont-2
   </body>
(1-2) <P> is used
   <body bgcolor="#FFFFFF" text="#000000">
    TextWithVariableWidthFont-1
    <P>TextWithVariableWidthFont-2</P>
   </body>
(1-3) <PRE> is used
   <body bgcolor="#FFFFFF" text="#000000">
    TextWithVariableWidthFont-1
    <P>TextWithVariableWidthFont-2</P>
    <PRE>TextWithFixedWidthFont-1
TextWithFixedWidthFont-2</PRE>
   </body>

(2) Almost plain text HTML-A for Tb :
    No formatting, Text only.
    If someone complains on <tt>, why no complaint on <pre> by the someone?
    Note: <tt> doesn't exist in HTML 5.
(2-1) <tt> is used
   <body bgcolor="#FFFFFF" text="#000000">
    TextWithVariableWidthFont <tt>TextWithFixedWidthFont</tt> 
   </body>
(2-2) <a> with Link Text==Link URL is used
   <body bgcolor="#FFFFFF" text="#000000">
    <a href="http://a.b.c/x/y.z">http://a.b.c/x/y.z</a> 
   </body>

(3) Almost plain text HTML-B for Tb :
    Text only, CSS Style is used.
(3-1) CSS Style is used
   <head><style>
    BODY {border: solid 5px #FF0000; padding: 5px;}
    .box {border: solid 2px #0000FF; padding: 2px;}
   </style></head>
   <body bgcolor="#FFFFFF" text="#000000">
    <p class="box">TextWithVariableWidthFont</p> 
   </body>

(4) Never plain text HTML and fully formatted HTML for Tb :
(4-1) background color is changed from default, or tag is utilized
   <body bgcolor="#FEFEFE" text="#000000">
     TextWithVariableWidthFont
   </body>

A big cause of problem is :
  mismatch between
        (X) convertible check logic of Tb for (2) and (3)
    and (Y) user's expectation on (2) and (3)

Even if Tb's convertible check logic will be changed,
I believe that mismatch between (X) and (Y) will exist forever,
as seen in example of "user's expectation on PRE".
How about calling msgcompose.background_color=#FEFEFE "Tb's option to skip Auto-Downgrade"?
If "Tb's option to skip Auto-Downgrade" is described in Release Notes,
I believe majority of Tb users won't complain on Auto-Detect feature which was made for a decent purpose。

Comment 50

[Ben Bucksch (:BenB)]

Comment on attachment 8669272 [details]
(already rejected by Ben, don't comment on it, please) {Auto-downgrade only for allPlain} - Pseudo one-liner patch created by altering aceman's patch attached to Bug 1204379

This disables the whole code that tries to detect whether we can send as plaintext or not, based on whether the content of the email is inherently plain text or not.

Comment 51

[Ben Bucksch (:BenB)]

This whole HTML vs. plain text has always had people shouting from both sides: either for plaintext or for HTML or for whatever in-between. No matter what we do, there are people shouting.

The code we currently have is exactly there to strike a fair balance between the camps:
* Enhance the plaintext with wrapping text, some formatting etc.pp.
* Send plaintext when we can, and send HTML when the user uses formatting that we need HTML for.

You're trying to disable that code that tries to strike that balance. Thus, WONTFIX.

If somebody wants to add a pref to say "always send HTML+plain, even if the text is just plain text and we can down convert to plaintext", go ahead. That's the only possible concession still possible.

Comment 52

[WADA:World Anti-bad-Duping Agency]

(In reply to Ben Bucksch (:BenB) from comment #51)
> You're trying to disable that code that tries to strike that balance.

Yes, if the one-liner patch will be lnded, it disables it.
So, (a) Send Options for "mixed recipients && Convertible" and (b) Send Options for "mixed recipients && Not-Convertible"(this is current Send Options) are needed.
This is an issue in my one-liner patch which I wrote in Table of "User Story" of this bug.

Why one-liner patch + Send Options for "mixed recipients && Convertible" is needed.
  For some users, following are unforgiven case.
     - sent as text/plain when <tt> is used
     - <a> with class="moz-txt-link-freetext" and Link Text==Link URL
     - some other cases.
  So, many comments will be posted bug such as bug 414299.
  If such user set HTML in new (a) Send Options for "mixed recipients && Convertible"
  and current (b) Send Options for "mixed recipients && Not-Convertible",
  mail composed in HTML mode is always sent as tet/html,
  so many comments for complaints won't be posted.
  So, I proposed 4-th solution.
For such users, "code that tries to strike that balance, or not" is irrelevant to them.

> Thus, WONTFIX.

I didn't know what cases are currently/actually processed by bug 414299. Attached file to this bug is cases with which I could observe "silently sent as text/plain".
Thanks for clear decision of WONTFiX.
If this bug was closed as WONTFIX, some other useless bugs can be closed.
By it, I can say "it's design decision with code that tries to strike that balance" in any report for "composed HTML mail, silently sent as text/plain". 

I've found a good bug report for CSS Style case which was closed as dup of bug 414299.
I hope CSS Style case will be improved.
This will be perhaps needed if Composer will start to utilixe CSS Style in HTML mail composition.
And, I hope Bug 56834(Opened on 2000-10-16) will be fixed.

By the way, I thought why "always send as text/plain" in "allPlain && Not-Convertible" case(B-2 in my Table) was done for long time, is as follows.
  Text Domain was for "mail recipent can not receive/understand or rejects HTML mail".
Is it rigt?
After Prefers=HTML/Text in Address Book, it seems for me that Text Domain is changed to "recipient prefers Text, although recipient can accept HTML".

Comment 53

[Thomas D. (:thomas8)]

Ben, please just keep out of this. You've blocked progress on this issue for long enough, and we won't tolerate that any longer. We're very close to fixing the random dataloss algorithms which you coded, and we're very close to striking a balance that is really balanced. Iow, instead of having the hard-coded and unpredictable bias for plaintext which can't be lastingly altered by user no matter how hard he tries, we'll finally give the user a choice over which send format he prefers, especially for cases of "prefers-unknown" with "auto-detect".
Rest assured, your favorite use pattern of downgrading to plaintext in spite of composing in HTML will still remain as an option.

There's a constructive discussion going on here, fuelled by users in countless bugs who are confused by the intransparent mechanisms which we have today. Fwiw, I have seen very few people shouting in favor of plaintext, and mostly relating to edge cases like newsgroups. Some like Magnus still want their message to get sent as plaintext if there's really NO formatting whatsoever - fair enough, we'll preserve that use case but make it optional.

Comment 54

[Thomas D. (:thomas8)]

(In reply to Ben Bucksch (:BenB) from comment #51)
> This whole HTML vs. plain text has always had people shouting from both
> sides: either for plaintext or for HTML or for whatever in-between. No
> matter what we do, there are people shouting.

A classic case for giving the user more options, so that everyone can have their way.
People shouting for plaintext have become a tiny minority; times have changed because HTML is no longer a problem, but the default choice for the whole internet.

> The code we currently have is exactly there to strike a fair balance between
> the camps:
> * Enhance the plaintext with wrapping text, some formatting etc.pp.
> * Send plaintext when we can, and send HTML when the user uses formatting
> that we need HTML for.

"Send plaintext when we can" is not balanced; it's prejudicial in favor of plaintext.

> You're trying to disable that code that tries to strike that balance. Thus,
> WONTFIX.

No, we will just enable users to make their own choices, instead of trying to force them into choices they don't want, for whichever reason.

> If somebody wants to add a pref to say "always send HTML+plain, even if the
> text is just plain text and we can down convert to plaintext", go ahead.
> That's the only possible concession still possible.

That's exactly what this bug is about, to add ways / an *option* of sending HTML from HTML composer even when there's very little or "no" formatting. Thanks for "conceding" that users have a right to choose freely in which delivery format they want to send.

Comment 55

[Thomas D. (:thomas8)]

Ben, again for your peace of mind, the current plan is to keep 'your' downgrade-html-to-plaintext auto-detection as the default.

Comment 56

[Thomas D. (:thomas8)]

Ben, the main problem is that for a number of use cases, the current set of options around delivery format does NOT do what it says on the box, and forces delivery format decisions on the user that should be optional even if we keep them as default, to avoid elements of surprise and unpredictable / uncontrollable results including silent removal of inline images, styles, etc.

* We provide send options: "When sending messages in HTML format and one or more recipients are not listed as being able to receive html: e.g. Send HTML+Plaintext". However, sending a fully formatted message to recipients who all prefer plaintext (i.e., they are "not listed as being able to recieve HTML"), will nevertheless always send plaintext without even warning about the massive dataloss involved (inline images, styles, and all). And user has no default option for a safer behaviour, nor knowing in advance from the status of the UI that this silent dataloss is going to happen.

* A message sent to one prefers-plain and one prefers-html recipient will currently just get sent out as plaintext when there's little or 'no' html. We ignore/disrespect prefers-html, and prefer plain, without giving the user a choice.

* What is the meaning of "prefers-unknown"? We provide a per-contact setting of prefers-plain | unknown | prefers-html. So clearly, by the inherent meaning of the word "unknown", we *cannot know* exactly which delivery format is preferred by the recipient. Maybe recipient prefers-plaintext. Maybe recipient prefers-html. In fact, most recipients these days are able to receive html and probably wouldn't find anything wrong or problematic with that. So if anything, we are supposed to err in favor of html. Otherwise, we must give the user a free choice if "prefers-unknown" should be considered as "prefers-text" or "prefers-html". A simple dropdown in options will do. Not more, not less. Then auto-detect can take a truly informed decision based on users personal preferences, and with the full range of send-options for the cases which actually involve opt-in downgrading.

* What is the meaning of "prefers-plain"? I believe there are no serious email clients out there who will actually still choke on multipart/alternative or multipart/mixed. So "prefers-plain" should mean just that: *Prefers* to receive a plaintext variant of the message. But if same message also gets sent out to another recipient who actually *prefers-html*, we must send multipart/alternative (html+plain) as a compromise so that everyone gets what they prefer, and we err in favor of preserving the original html composition. Which still leaves plenty of avenues for those who really want their html composition to be delivered as plaintext.

Comment 57

[:aceman]

(In reply to Ben Bucksch (:BenB) from comment #51)
> If somebody wants to add a pref to say "always send HTML+plain, even if the
> text is just plain text and we can down convert to plaintext", go ahead.
> That's the only possible concession still possible.

This is exactly what I want to implement (and also proposed specific implementation in bug 136502).
I just wait until the dust settle and all parties actually agree on this (or any other outcome).

Comment 58

[WADA:World Anti-bad-Duping Agency]

Attachment: qpo

If default, same as current behavior.
And, user can choose any action in any case aat any time.

Objection?

Comment 59

[WADA:World Anti-bad-Duping Agency]

Attachment: oml

Comment 60

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #57)
> (In reply to Ben Bucksch (:BenB) from comment #51)
> > If somebody wants to add a pref to say "always send HTML+plain, even if the
> > text is just plain text and we can down convert to plaintext", go ahead.
> > That's the only possible concession still possible.
> This is exactly what I want to implement (and also proposed specific implementation in bug 136502).

A possible simple pseudo code for it.
  -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
  +  var skip_auto_downgrade = prefs setting value of mailnews.compose.skip_auto_downgrade ;
  +  var force_auto_downgrade_if_Convertible =
  +      !skip_auto_downgrade  &&  aConvertible==nsIMsgCompConvertible::Plain ;
  +  if ( allPlain || force_auto_downgrade_if_Convertible )
     {
       *result = nsIMsgCompSendFormat::PlainText;
       return NS_OK;
     }
When user sets mailnews.compose.skip_auto_downgrade=false(default),
     user can gets absolutely same begavior as current.
When user sets mailnews.compose.skip_auto_downgrade=true,
     when user sets Text at Send Options panel, user can get absolutely same begavior as current,
     when user sets HTML at Send Options panel, user can get text/html even if Convertibe.
Problem is:
  If user sets mailnews.compose.skip_auto_downgrade=true,
  and If user sets HTML at Send Options panel,
  user can not get text/plain when created HTML is Convertibe HTML.
  This is because single Send Options panel is shared between (C-1) and (C-2).
  This kind of ussue is a reason why WONTFIX of my one-liner patch is prerry reasonable and acceptable,
"A full solution" I attached is a way to resolve this kind of issues in current Send Options panel.

Big difference between "mailnews.compose.skip_auto_downgrade=true/false" and "msgcompose.background_color=#FEFEFE in mailnews.js".
  mailnews.compose.skip_auto_downgrade=true/false :
     excess code change and new preference is needed.
  msgcompose.background_color=#FEFEFE in mailnews.js as "official option to skip auto-downgrade" :
     one-liner patch in mailnews.js is sufficient.
     This merely forces aConvertible==nsIMsgCompConvertible::No, without hurting anything.
     What's bad in this? Converible check logic is so designed and is so coded.
     Problem is :
       This kind of action usually generates war among developers and among users,
       and produces useless/endless discussion on something,    
       as seen in Bug 1204379 :-)

Comment 61

[WADA:World Anti-bad-Duping Agency]

To aceman, please note following.
(1) Read bug reports on phenomenon of "composed HTML mail, silently sent as text/plain", please.
    How many peoples shouted/are shouting "format loss or "data" in such bugs in Bugzilla?
    Please note that official method(Delivery Format=HTML) and simple/easy workaroud is already known.
(2) See change by bug 970118 in 2014, please.
    Before sorting out by bug 970118, I believe that touching around auto-downgrade was pretty hard.
    Possible solution had been/was perhaps improvement in Convertible check logic only for long time.
    Fortunately, when I opened Bug 1204379, the sorting out by bug 970118 was already implemented in 2014.
    Fortunately, you joined Bug 1204379, and looked into send logic,
                 and you succesfully found pretty simple one-liner patch.
    Without bug 970118 and your proposal of one-line patch,
    I couldn't be aware of importance of Bug 56834(Opened on 2000-10-16).
(3) "User's default choice for Unknown" is one of best solution.
    Because problem occurs only in "mixed recipents && Covertible" case,
    If Unknown is changed to HTML or Text by user,
    problem occurs only when "recipents is HTML + Text && Covertible".
    "A recipient is member of Text Domain or HTML Domain" can occur only by:
       Text Domain/HTML Domain setting by user
       Prefers=HTML or Text in Address Book by user
       Default setting of HTML or Text for Unknown by user
    Becaause Unknown case disappers, and because "Prefers=HTML or Text" is done by user,
    following is pretty easy-to-understand phenomenon for user.
      sent as text/plain by Auto-downgrade when Text Domain recipient is contained in recipients.
    And, action by user is usually "set Prefers=HTML for the recipient".
    I believe that action by user is never shouting "Format Loss" or "Data Loss" at bugzilla.mozilla.org.
(4) Advantage of "User's default choice for Unknown".
    This merely helps user to set Prefers=HTML or Prefers=Text for Prefers=Unknown Contact.
    This doesn't hurt anything.
    This never generates war among peoples and never produces useless/endless discussion on something.

Comment 62

[WADA:World Anti-bad-Duping Agency]

By the way, to aceman, a suggestion in Bug 1204379.
In mailnews.js,
   msgcompose.background_color = #FFFFFF
   msgcompose.text_color       = #FFFFFF <= what a nice solution! I like default like this :-)
User shouts "Text is lo--st!!!", then exausted by the shouting, so doesn't shout about this bug :-)

Comment 63

[WADA:World Anti-bad-Duping Agency]

To aceman.

Following comment is seen in Bug 1204379.
> 4976   // If we can guarantee that converting to plaintext is not lossy, send the
> 
> Note the wrong claim in the comment: IF WE CAN GUARANTEE that converting to plaintext is NOT LOSSY...
> What a brazen lie...

Difference of language may be involved.
                                         +---
                                         | width>0 in Japaneze usually, especially in daily life
                                         | width=0 in English always, perhaaps
  <- pretty good ->                      V                                        <- pretty bad ->
  <-------------- Good --------------><------ Grey Zone ------><-------------- Bad --------------> 

                          In Japanese 
                          <- not so good -> "may be slightly bad or bad" is involved in many cases
                     In English 
                     <- not so good -> "bad" is perhaps never involved

  GUARANTEE in Japanese, in daily life :
    We believe that our product is pretty good.
    If flaw is found, we repair with no charge.
  GUARANTEE in other language :
    There is no flaw in our product on the Earth.
    If you thought you found flaw in our product, it's merely your mistake or prejudice.
    For user : Exception is unforgiven.

  NOT LOSSY in Japanese       : identical to "Not so lossy" in many cases, in daily life.
  NOT LOSSY in some languages : identical to LOSSLESS.

Words in "source COMMENT" may be better changed.
  GUARANTEE => consider
  NOT LOSSY => not so lossy, not so heavily lossy, not so badly lossy,...

Comment 64

[WADA:World Anti-bad-Duping Agency]

Ben, see question at bottom of commment #52, please.
Was meaning of "Text Domain" changed?
If not changed, I think what occurs is :
   In start of 21-th century, "treat Unknown as Text" was far safer and was pretty reasonable.
   But nowadays, "treat Unknown as HTML" is appropriate for majority of recent Tb users.
   (This is my feel which was added to bug summary of bug 584363 on 2013-05-28)

Comment 65

[WADA:World Anti-bad-Duping Agency]

Sorry, wrong linkify.  See bottom of comment #52, please.

Comment 66

[WADA:World Anti-bad-Duping Agency]

Another question to Ben.

(In reply to Ben Bucksch (:BenB) from comment #51)
> If somebody wants to add a pref to say "always send HTML+plain, even if the
> text is just plain text and we can down convert to plaintext", go ahead.
> That's the only possible concession still possible.

Ben, does it mean that you fully agree on change like comment #60?
(that is a possible change for bug 136502 which could be found by looking Table in "User Story" of this bug and by checking affect by my one-liner patch in this bug)

I dislike such prefs setting, so I prefer "user's default choice for Prefers=Unknown" which was requested in 2000, at Autum of last year of 20-th century.

Comment 67

[WADA:World Anti-bad-Duping Agency]

Ben, please see Comment #46. 
My Comment #46 is overview/quick summary of requests/issues around Auto-Detect.
After I knew current send logic by aceman's great help, and after proposing my one-liner patch at this bug which was borrowed from aceman, I checked some bug reports around Auto-Detect/Auto-Downgrade, especially bugs which was closed as dup of bug 414299 in the past.
I could find such bug relevant to Auto-Detect/Auto-Downgrade easily, because meta bug 889315 was kept useful by great effort of Thomas D. since 2013.

It looked for me :
  When phenomenon of "silently sent as text/plain by Auto-Detect" is reported at bugzilla,
  it's blindly considered "culprit is Convertible check logic" by many peoples,
  except several fortunate bug report for the phenomenon such as Bug 389417,
  except bug for request of "Kill Auto-Detect, Kill Auto-Downgrade-To-Text".

As seen in Bug 389417(opened 2007-07-24), old user surely accepted Auto-Downgrade and surely needed/utilized it.
  User's want in this case was:
    If Body Text+P only, sent as text/plain silently by Auto-Detect/Auto-Downgrade.
    If PRE is used, always Asked, as asked when many tags are used.
    He needed different font use for Body Text/P(Proportional Font) and for PRE(Fixed Width Font),
    because he used PRE for attached code sample etc.(Fixed Width Font is preferable for program source.)
I think environment is already different and users are already different from 20-th century.
There are surely 2 kinds of user :
  User who needs and utilizes Auto-Downgrade.
  User who hates Auto-Downgrade and can't permit behavior of Auto-Detect.
And, I think proper solution is not new prefs option for "Kill Auto-Detect, Kill Auto-Downgrade-To-Text", because proper user's action of "Add to HTML Domain(Prefers=HTML)" or "Add to Text Domain(Prefers=Text)" surely existed since year 2000.

Comment 68

[Thomas D. (:thomas8)]

(In reply to WADA from comment #63)
> Difference of language may be involved.
>   GUARANTEE in Japanese, in daily life :
>     We believe that our product is pretty good.
>     If flaw is found, we repair with no charge.
>   GUARANTEE in other language :
>     There is no flaw in our product on the Earth.
>     If you thought you found flaw in our product, it's merely your mistake
> or prejudice.
>     For user : Exception is unforgiven.
> 
>   NOT LOSSY in Japanese       : identical to "Not so lossy" in many cases,
> in daily life.
>   NOT LOSSY in some languages : identical to LOSSLESS.
> 
> Words in "source COMMENT" may be better changed.
>   GUARANTEE => consider
>   NOT LOSSY => not so lossy, not so heavily lossy, not so badly lossy,...

lol... Thanks for that intercultural insight :)

Yes, parts of this exercise are in the range of perfecting the pretty good.
But other parts are required to prevent gross violations of commonly-accepted ux-expectations (I won't use the evil data.... word while WADA is around...).

Comment 69

[Thomas D. (:thomas8)]

Attachment: Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

I think WADA's proposal of attachment 8672017 [details] is already pretty good and going in the right direction of giving more send options control to the user. However, the total matrix exposed in WADA's proposal (P-W2) is pretty large and many possible settings would not actually make sense. For ux-error-prevention, e.g. we must not allow users to get themselves into massive dataloss scenarios when they set "send plaintext-only" when all recipients are "prefers-HTML".

Maybe we can get away with less options.

The underlying considerations, key changes and resulting default behaviour of these two proposals are largely the same or similar.

Some differences between the two proposals (WADA's P-W2 and my P-T1):

a) As Magnus said, we must never silently downgrade HTML compositions to plaintext when there's significant formatting (esp. convertible::No). So I changed those scenarios of ux-error-prevention.

b) The biggest problems come from downgrading HTML to plaintext. The most significant usecase of downgrading is when all recipients prefer plain (allPlain). So for that case, dedicated send options are most needed, as reflected in my proposal.

c) For most mixed cases where some recipients of the same message prefer plain while others explicitly prefer HTML, imo that's the classic usecase for sending multipart/alternative (both plaintext+HTML) without asking, which is also the current default. While in theory that default could be changed by user in current send options, in practice all options except "both" have significant shortcomings in the current design. So in reality, "both" is the only useful default for such cases, which is why I didn't provide so many choices for some mixed cases involving prefers-HTML or unknown-prefers-both.

d) For recipients of type prefers-unknown, WADA and I agree we need disambiguation, i.e. allow the user to globally translate "prefers-unknown" into "prefers-plain" or "prefers-html". Which is very helpful for more predictable and controllable auto-detection. But "prefers-unknown" is also the classic case for sending both "plaintext+HTML" because, well, we don't know their preferred format, so I added that missing option, which also allows mimicking the current behaviour for those who want it. The default for "unknown" is "prefers-html" in my design, but there are separate options for auto-downgrade which allow overriding that (see next point).

e) One of the main bones of contention in many bugs was the "surprise effect" of auto-downgrading to plaintext when there's ostensibly little or no HTML formatting (we'll fix the dataloss of that broken algorithm elsewhere, but the inherent problems of that behaviour and forcing it onto users will still remain). Many users just don't like such surprises.
Even Ben Bucksch conceded that we could make "auto-downgrade" optional.
I provide a centralized option for that (like WADA), but I shrinked WADA's complete matrix to cover only the most important sets of recipients and cases. In fact, auto-downgrade is now optionally permitted for any type of recipients, so I'm actually expanding the possible target range for those who like the feature. I'm not a big fan, but Auto-Downgrade for convertible::plain is even still enabled by default (so that Magnus and Ben can be happy).

f) I also included full solution for bug 136502, allowing users to set a message-centric default delivery format instead of using auto-detect, which is recipient-centric. As most recipients these days are capable of receiving HTML, the complexity of recipient-based auto-detection of the best send format compromise is much less relevant/useful than in former times.

Note: For the actual UI, dropdown boxes will be used instead of bullets, so the real options layout will be much cleaner, more intuitive and condensed than the functional UI sketch. If somebody could provide an XUL mockup of my proposal, that would be great.

Comment 70

[Thomas D. (:thomas8)]

Correction:

> SendOptions [only for allPlain]
> (default_html_action pref):
> - Ask
> - *Try converting to plaintext, otherwise ask* if (!convertible::plain && !convertible::altering)
I forgot !convertible::yes.
I guess that could be rewritten as: ...otherwise ask if (convertible::no).
> - HTML
> - Both

Comment 71

[:aceman]

Sorry, but what is this bug for? To me the description is the same as in bug 136502 comment 0.
Then in bug 136502 comment 80 you WADA claim that the reporter there actually does not want what he wrote (but we can solve it in another way). Then you file a new bug being exactly the same as bug 136502 (at least to me).

So what is the difference here?

Comment 72

[Thomas D. (:thomas8)]

(In reply to WADA from comment #67)
> Ben, please see Comment #46. 
> [...]
>     If PRE is used, always Asked, as asked when many tags are used.
>     He needed different font use for Body Text/P(Proportional Font) and for
> PRE(Fixed Width Font),
>     because he used PRE for attached code sample etc.(Fixed Width Font is
> preferable for program source.)

Dumping <pre> is a classic design failure of auto-downgrade, thanks for raising that problem.

> I think environment is already different and users are already different
> from 20-th century.

Indeed.

> There are surely 2 kinds of user :
>   User who needs and utilizes Auto-Downgrade.
>   User who hates Auto-Downgrade and can't permit behavior of Auto-Detect.

Oh yes, boy...

> And, I think proper solution is not new prefs option for "Kill Auto-Detect,
> Kill Auto-Downgrade-To-Text", because proper user's action of "Add to HTML
> Domain(Prefers=HTML)" or "Add to Text Domain(Prefers=Text)" surely existed
> since year 2000.

Hmmm. I'm failing to see how you arrive at that conclusion. Is it a test if we're paying attention or if we are fit in general logic?

From WADA's pretty excellent analysis (though not always easy to understand), i find it quite clear that we DO need new options which allow users to generally opt out of Auto-Downgrade as well as Auto-Detect.
Providing such options is the only way to end this everlasting saga and close many bugs, unless we want to rip out those features completely and move them into addon space.
Even Ben has agreed that providing such options is possible.
It's not about "killing Auto-Downgrade" or "Auto-Detect", it's about returning control, transparency and predictability and to our users and respecting/preserving/sending their messages exactly as composed if they so wish, without forcing them into per-message decisions or other extra hassles.

On top of that, the bugs and inherent design failures of current "Auto-Downgrade" and "Auto-Detect" must be fixed, e.g. unwarranted dumping of styles and silent real dataloss scenarios for allPlain, as Magnus pointed out.

My proposal provides most of that, with a pretty minimal UI (remember, all the bullets get shrinked into tiny dropdowns, we really need to look at this in XUL). And for those users who don't like the whole complexity currently underground, now tamed and slightly more exposed in my proposal, the good news is that you can just opt-out.

TB should never force users against their will. Users must be in control of their UX. That's a key part of the Mozilla creed. The wish to actually and reliably send HTML compositions in HTML format *always* without worrying about {domains, workarounds, particular formattings or other internal randomness/finesse} is neither insane nor asking too much, but obviously reasonable.

Comment 73

[WADA:World Anti-bad-Duping Agency]

Attachment: Pseudo code for user's choice of Default for Unknow with new pref of mailnews.compose.defaultForUnknownIsText

mailnews.compose.defaultForUnknownIsText = true(default)/false
     false means defaultForUnknownIsHTML.
This is new name for mailnews.compose.skip_or_force_autodowngrade_if_Convertible :-)

Unknown falls into Text Domain or HTML Domain by user's choice.
Unknown disappers, and mixed case is changed to "Text Domain recipients + HTML Domain recipients".
There is no need to chenge Prefers of all Contacts of all Address Book.
Following is sufficient. And user can do it because who choosed defaultForUnknownIsText valuee is the user.
(a) If default=HTML, and "sent as text/html when mixed(&&Convertible)" is bad for user,
    change Prefers of recipient to Text. 
(b) If default=Text, and "sent as text/plain when mixed(&&Convertible)" is bad for user,
    change Prefers of recipient to HTML. 

There is no need to touch Address Book UX/UI.
Using in send logic, is sufficient.
UI for it is better provided in Send Options panel.

Comment 74

[WADA:World Anti-bad-Duping Agency]

Attachment: (v 2) Pseudo code for user's choice of Default for Unknow with new pref of mailnews.compose.defaultForUnknownIsText Reas

Sorry, typo in previous upload.

Comment 75

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #71)
> So what is the difference here?

I'm trying to find proper solution to reduce user's confusions for user such as bug opener of bug 1030288.
What do you think about new "user's default choice of Unknown"?

Comment 76

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #71)
> Sorry, but what is this bug for? To me the description is the same as in bug 136502 comment 0.

But request in that bug is change around Delivery Format menu(attached patch is trial of "change default(checked) to Both") and/or option to kill Auto-Detect. I'm trying to find a solution which can be first answer to that bug.

Comment 77

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #69)
> Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

I prefer HTML version to PDF version...
Please don't remove "user's default chioce for Unknown" part...

Comment 78

[:aceman]

(In reply to WADA from comment #75)
> I'm trying to find proper solution to reduce user's confusions for user such
> as bug opener of bug 1030288.
> What do you think about new "user's default choice of Unknown"?

Ah, another bug coming into play :(

(In reply to WADA from comment #0)
>   Then, many news users say "Thunderbird posted HTML news posting!!!",
>   instead of saying to the USER "HTML posting is not so good manner, please
> stop it".

Can we set plain-text composing as the default for News accounts?

Comment 79

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #78)
> Can we set plain-text composing as the default for News accounts?

As I wrote in comment #4 and comment #5, there was no need to worry about news posting. Developers know about "Text post vs. HTML post War in News" well.

Comment 80

[WADA:World Anti-bad-Duping Agency]

Attachment: (v 3) Pseudo code for user's choice of Default for Unknow with new pref of mailnews.compose.defaultForUnknownIsText=true(Text)/false(HTML)

Woops, forgot to include importaant part(source comment change). Sorry for frequent change.

Comment 81

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #71)
> Sorry, but what is this bug for? To me the description is the same as in bug
> 136502 comment 0.
> Then in bug 136502 comment 80 you WADA claim that the reporter there
> actually does not want what he wrote (but we can solve it in another way).
> Then you file a new bug being exactly the same as bug 136502 (at least to
> me).
> 
> So what is the difference here?

Hi Aceman,

Sorry for the confusion. Admittedly, I'm also sometimes failing to get all the details of what WADA is trying to say, which involves language/presentational issues as well. But by far the biggest confusion is inherent to the two distinct features of Auto-Downgrade (via message body analysis) and Auto-Detect (via recipient preferences). Apart from many bugs and design shortcomings (including UI misrepresentations/lies) one big confusion comes from the fact that they are bundled together. So from looking at the UI, the user has no way of telling the two features apart, so to them it just appears as ONE big mystery/surprise or hated misfeature called "Auto-Detect".

So WADA is very correct that there's a difference between this Bug 1210244 and Bug 136502, and that the reporter of Bug 136502 Comment 0 is not aware of the twin-feature and thus asking us for option to disable B (Auto-Detect), while the real cause of his problem is A (Auto-Downgrade). So it's like when you want to throw away the whole car (and use new car instead) because the steering wheel is blocked. Using the new car will avoid the steering problem; but unblocking the steering wheel of the current car would also avoid the steering problem.

Bug 136502: Auto-Detect unexpectedly downconverts my messages, please allow delivery format default setting which is not Auto-Detect.

This Bug 1210244:
(In reply to WADA from comment #0)
> This bug is spin-off of Bug 1204379 Comment #107.
> 
> Please give us option to send mail which I composed in HTML mode as HTML
> mail, 
>   even when Delivery Format=Auto-Detect is used or forced,
>   even when gMsgCompose.bodyConvertible() says Convertible,
>   even when we forgot to change Unknown to HTML for all contacts in all
> Address Book,
>   even when we mixed HTML recipients and Text recipients and Unknown
> recipients.

So, the problem description of this bug 1210244 is much more detailed as it correctly mirrors more involved factors. Consequently, it seems more open for *any* solution which tweaks/fixes or allows the user to control any of those factors so that the net result of the whole conglomerate actually matches the user's expectation, to just send HTML after composing HTML.

So, while bug 136502 wants to abandon the entire car (bypass delivery format called "Auto-Detect" in UI, i.e. bypass Auto-Downgrade AND Auto-Detect), solution for this bug 1210244 might be just unblocking the steering wheel (e.g. option to disable Auto-Downgrade), or anything else which could be unblocked to get the car running as it should (e.g. provide pref which allows user to define "prefers-unknown"=="prefers-html", so that auto-detect can use that to hopefully choose delivery format "HTML".

Apart from code complexity, design bugs etc. it's also about the human factor and strategic bug management. Ben has for a long time stubbornly and aggressively denied/downplayed/not acknowledged many of the obvious problems, especially by saying that the status quo was a ceasefire compromise from Plaintext vs. HTML war and thus sacrosanct/never to be discussed again/never to be changed. Decades later, the post-war generation no longer cares about that as we see the whole world wide web flourishing in HTML. So we're no longer willing to suffer from outdated compromises which choke our freedom of choice, and violate common sense ux principles.

Another human factor is the fact that many developers do NOT fully understand all the factors involved and their interaction, and even though I figure that I might have seen and known all of them, and perhaps understood the whole system for some light moments, it's all so twisted in logic, buggy and intertwined that it starts to swirl if you look at it just a little too long. Just too many factors. So it's very hard, even for myself after all this time, to fully understand the consequences of, say, unblocking the steering wheel and make Auto-Downgrade optional. Turns out if you drive backwards (all recipients prefer plain), the car will crash against the wall without braking (silent, massive dataloss of formatting including inline images and all). Driving backwards is also driving, but maybe driving backwards is another scenario. Or maybe not. Who knows?

Finally, due to the human factors and shortcomings which have so far prevented any progress on this matter, I also suspect strategic bug management. As long as all of our proposals for change get shot down for one reason or another (good or bad, benevolent or malevolent, lack of understanding, wrong approach of patch etc.), we have to attack this broken war-thing from all possible angles until something will budge, somewhere. Hopefully one day, we'll arrive at a new world order where more freedom of choice prevails, and less safe-guarding against threats/problems which are no longer existing.

I'm not entirely sure about the best approach in this unclear/muddled situation; I strongly suspect that we need a wholistic approach which looks at all the various problems in context, not in isolation.
Hence my full solution proposal of attachment 8672330 [details], which seeks to address as many issues of the misbehaving twin features as possible; except the inherent dataloss issues of the Auto-Downgrade algorithm which we can fix elsewhere. My main constraint is time; just wish I could do the XUL demo because it will be much more accessible and friendly than the murky theory of the sketchy draft...

Dear Aceman, I do hope that my attachment 8672330 [details] also explains my envisioned solution for bug 136502 which I think is a legitimate request in its own right and coming with its own benefits, namely this:
Indeed allow users (who so wish) to totally bypass both "Auto-Downgrade" and "Auto-Detect" and set a stable default delivery format of their choice (and I still think your minimalistic addition of delivery-format: use-composition-format wouldn't work well for the reasons I explained). Message-centric as an alternative to Recipient-centric, as I have suggested at least 2 years ago, e.g. in Bug 78794 Comment 41. The full panorama in meta bug 889315. A new post-war world order in the making (where HTML messages are no longer frowned upon, nor randomly oppressed and destroyed by Thunderbird...) ;)

Comment 82

[Thomas D. (:thomas8)]

(In reply to WADA from comment #77)
> (In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment
> #69)
> > Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf
> 
> I prefer HTML version to PDF version...
> Please don't remove "user's default chioce for Unknown" part...

I have user's default choice for "unknown== plain | html | both".
What kind of "user's default chioce for Unknown" part have I removed?

Comment 83

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #81)
> I'm not entirely sure about the best approach in this unclear/muddled
> situation; I strongly suspect that we need a wholistic approach which looks
> at all the various problems in context, not in isolation.
The problem is, nobody can propose a wholistic approach that will pass by all commenters and reviews.
Everybody proposes slightly different result requests or design proposals. It is so messy I already do not read half of the comments (sorry). So I do not think a one-size-fits-all approach is possible here.

I think I see the whole picture (code-wise) in this problem and can see which places affect the convertibility and final send format.
Therefore I propose to tackle the various places piecemeal and fix their deficiencies. Then everybody can set the system up to get resulting format he wants. We just need to decide how many possible results we want to support and that determines the complexity of the options we have to provide for that.

> Dear Aceman, I do hope that my attachment 8672330 [details] also explains my
> envisioned solution for bug 136502 which I think is a legitimate request in
> its own right and coming with its own benefits, namely this:
> Indeed allow users (who so wish) to totally bypass both "Auto-Downgrade" and
> "Auto-Detect" and set a stable default delivery format of their choice (and
Yes, I aim to allow to select this. But also keep the various auto-detect and downgrade options intact.

> I still think your minimalistic addition of delivery-format:
> use-composition-format wouldn't work well for the reasons I explained).
But I have not understood the reasons:( So I have uploaded my proposal in bug 136502. Consider what I wrote above. The patch in 136502 is not the full solution. But it is a step for those that want to force HTML format.

In your car analogy:
1) this bug (or 136502, I can't decide) is unblocking the steering wheel :) You can tell the car where to drive (HTML or plain).
2) But even if you now have free steering wheel, you are not allowed to drive over pedestrians as there is an "autopilot" built in that prevents you from doing so:) To turn off (or finetune) the "autopilot", there is bug 1204379 or some other bug where you guys want to split Send options for unknown and plain text recipients (I would support and implement that when you can decide on it).
3) But even with all that there is the bug that the "autopilot" wrongly thinks fireflies to also be pedestrians and drives you around them. So to fix its sensors, there is bug 414299 and 584313.

So I think we need all 3 places to exist so that anybody can mix the behaviour they want. Someone wants to risk everything and drive over anything (also pedestrians) and other user may want total autopilot (also the locked wheel) to drive avoid even fireflies for safety's sake. And then someone in the middle with free wheel but autopilot with working sensors prompting him if a pedestrian is to be hit.

Comment 84

[WADA:World Anti-bad-Duping Agency]

To aceman.

My "Full solution" is:
- To analyze what is needed in Send Options panel and Dialog by Send Options,
  if "if ( allHtml || allPlain || (!allHtml&&!allPlain) ) UseSendOptions" will be done.
  (All is done under control of Send Options. User can choose any action user wants, in any case)
  This can be perhaps called what Thomas D. wants.
- To analyze what kind of UI for some solutions will be needed in Send Options panel.
  Currently, following is considered.
     Default of dropdown of Delivery Format menu, order in the dropdown menu.
     Enable/Disable Auto-Downgrade,
     Default choice by user for Prefers=Unknown

For "user's choice of default for Prefers=Unknown".
Because initial proposal of next was rejected,
  Use current Send Options panel for both mixed&&Convertible case and mixed&&Not-Convertible case.
I tried to find next simple solution.
This is pseudo code I uploaded.
This is preparation of next step.

aceman, please review the psuedo code.

By the way, initial patch was a friend of bug 136502(can be called a subset of bug 136502). Psuedo code in comment #60 is a possible change for bug 136502 which could be obtained by analysis of initial one-liner patch by you and me. This bug sufficiently helped to look for proper solutions.
This bug is already in hard-to-read, hard-to-understand, hard-to-follow state, as Bug 1204379 was so.
If the psuedo code for "user's choice of default for Prefers=Unknown" is effective to resolve problem, I'll open bug for it. Please note that who re-opened this bug was NOT ME. The new bug will be member of bug 56834 family.

Comment 85

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #82)
> (In reply to WADA from comment #77)
> > Please don't remove "user's default chioce for Unknown" part...
> I have user's default choice for "unknown== plain | html | both".
> What kind of "user's default chioce for Unknown" part have I removed?

Oh, sorry, I thought it's for dropdown of Delivery Format menu, because "both" is contained.
In Address Book, Prefers is one of Unknown, HTML, Text only, and Domain is for HTML Domain and Text Domain only.

Comment 86

[WADA:World Anti-bad-Duping Agency]

Following is a part of opinion of Thomas D. on my "Full solution" attached to this bug, which was sent to me via. +PLAIN TEXT* Email.
> > [1] Change in send logic.
> >
> >    In any case, UseSendOptions.
> 
> Yeah, seems right for your approach... I think/hope...

As I wrote in comment #84, that "Full solution" is:
- To analyze what is needed in Send Options panel and Dialog by Send Options,
  if "if ( allHtml || allPlain || (!allHtml&&!allPlain) ) UseSendOptions" will be done.
  (All is done under control of Send Options. User can choose any action user wants, in any case)
i.e. Analysis of Send Options panel and Dialog by Send Options, if following change will be made.
User can control anything on "sent mail format", with utilizing setting in Send Options panel, with utilizing Prefers setting in Address Book, with respecting aCnvertible value, with utilizing new prefs settings, for example, compose.DefaultForPrefersInAddressBook=HTML or Text, compose.Always_Send_As_HTML_When_I_Composed_Mail_In_HTML_Mode_Because_I_Want_HTML_Mail_Is_Sent_As_HTML_Mail, etc.

"What Thomas D. wants" seems what I call "All is done under control of Send Options and user".

Comment 87

[:aceman]

Comment on attachment 8672330 [details]
Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

What are the a) b) c) items in the Table of results?
e.g. "3.all unknown", so how do you get into case a) or b) or c) ?

Another question: can we just send Both if the mix of recipients is "all unknown" or "some unknown & some HTML"? Only if there is at least one plain recipient, consult the Send options. Would that work? That is basically case 1. in the table, I just do not understand how to get into e.g. 3.a) where Send options come into play again.

Comment 88

[WADA:World Anti-bad-Duping Agency]

Comment on attachment 8672330 [details]
Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

> default choice for "unknown== plain | html | both".

In Address Book, Prefers is one of Unknown, HTML, Text only, and Domain is HTML Domain and Text Domain only.
"Both" shuldn exist in the setting.

Comment 89

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #72)
> > And, I think proper solution is not new prefs option for "Kill Auto-Detect,
> > Kill Auto-Downgrade-To-Text", because proper user's action of "Add to HTML
> > Domain(Prefers=HTML)" or "Add to Text Domain(Prefers=Text)" surely existed
> > since year 2000.
> Hmmm. I'm failing to see how you arrive at that conclusion.
> Is it a test if we're paying attention or if we are fit in general logic?

Yes.
It can be called "a kind of quesions to know following".
- Whether you distinguish "Auto-Detect" and "what I call Auto-Downgrade", or not.
- Whether you understand following, or not:
    Current mixed case(Unknown in many cases) is changed to "HTML + Text"(Unknown is not involved),
    if user sets HTML or Text, via. any of Address Book setting, "default for Unknown" setting.
- Whether you think "Kill Auto-Detect" or "Kill Auto-Downgrade" like option is adequate, or not.
- Whether you think "always HTML", "give me Not-Convertible always!", "send as HTML even when Convertible",
  like one is appropriate for option, prefs, feature, or not.
- Whether you think like next, or not.
    "default setting for Unknown by user" first,
    a way to ask Tb for "please skip Auto-Downgrade for me" or "give me Not-Convertible always!", second,
    is better as solution for general Tb users.

Comment 90

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #87)
> Comment on attachment 8672330 [details]
> Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf
> 
> What are the a) b) c) items in the Table of results?
> e.g. "3.all unknown", so how do you get into case a) or b) or c) ?

Sorry for not explaining that in the proposal, although I added red text color and bold to point out the solution ;)

WADA and I assume that it's helpful for more user-controllable/predictable auto-detection of best send format, if TB or user can pre-set the prefered send format for recipients whose preference is "unknown", iow to disambiguate "unknown" into "known".
In my proposal, the, choices are:
a) unknown==plain
b) unknown==html
c) unknown==both
So in the auto-detect results table, the different translations of unknown must reflect accordingly.

WADA is against option of unknown==both, but I think it's necessary and useful. It's actually current default format for mixed cases if !convertible::plain.

> Another question: can we just send Both if the mix of recipients is "all
> unknown" or "some unknown & some HTML"?

Basically yes, but "both" should not be sent if everyone prefers HTML, because it's just a waste of bandwidth and unnecessarily increasing message size.

"some unknown" & some HTML will be translated in my proposal to known preferences:
a) if unknown==plain: some plain & some HTML
--> send both (one prefers plain, one prefers html, so sending both is right, isn't it?)
b) if unknown==html: some html & some HTML == allHTML
--> send HTML only (this where we are better now; currently results in both from send options)
c) if unknwn==both: some both & some HTML
--> send both (some prefer both, others HTML).

> Only if there is at least one plain
> recipient, consult the Send options. Would that work?

No. As I tried to explain in the proposal, imo combined send options for plain + any other preference will never work. We need separate options which are only for allPlain case, because that case can be handled more radically than any mixed cases. You can never set a radical downgrading as default for mixed cases.

> That is basically case
> 1. in the table, I just do not understand how to get into e.g. 3.a) where
> Send options come into play again.

3a) is case of: all unknown==plain (by global user preference for unknown). The table is only for !convertible::plain, so we're talking about messages with relevant HTML formatting, but all recipients want plaintext. So we use send options to decide what to do. And my send options, like Magnus suggested, will always ask if there's loss of formatting, *at least* when it's convertible::no. (for convertible::yes and convertible::altering, silent downgrading might be acceptable if user agrees and everyone prefers plaintext).

Comment 91

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #90)
> WADA and I assume that it's helpful for more user-controllable/predictable
> auto-detection of best send format, if TB or user can pre-set the prefered
> send format for recipients whose preference is "unknown", iow to
> disambiguate "unknown" into "known".
> In my proposal, the, choices are:
> a) unknown==plain
> b) unknown==html
> c) unknown==both
> So in the auto-detect results table, the different translations of unknown
> must reflect accordingly.
Ah, so this is another group of settings. It would surely be no problem to add them. But I worry we are getting into too many options, which Magnus was against. Also, this feature we are talking about is decided in the mailnews code so this would affect Seamonkey. They would have to get the same options and UI (or at least hardcode the prefs to some default values). Of course, it could be easier to include the "kitchen sink" into SM than into TB (which pretends to be the simpler/cleaner one). But I think we haven't yet heard any opinion from them. We probably missed to CC them to the relevant bugs.

> > Another question: can we just send Both if the mix of recipients is "all
> > unknown" or "some unknown & some HTML"?
> 
> Basically yes, but "both" should not be sent if everyone prefers HTML,
> because it's just a waste of bandwidth and unnecessarily increasing message
> size.
Sure. I was not asking about "All HTML" case. Only about "Some unknown (more than 1), some HTML (0 or more), and 0 plain text" recipients.

> > Only if there is at least one plain
> > recipient, consult the Send options. Would that work?
> 
> No. As I tried to explain in the proposal, imo combined send options for
> plain + any other preference will never work. We need separate options which
> are only for allPlain case, because that case can be handled more radically
> than any mixed cases. You can never set a radical downgrading as default for
> mixed cases.

Why? If there is at least one plain text recipient, we must send the msg in the lowest common denominator.
For HTML recipients, it is HTML, for unknown it is Both (which I asked above). For plaintext recipients, we consult Send options (so the user decides whether it is Ask me, HTML, Both, Plain). So the Send options now decide the final result for all recipients. What is missing in that?

For allPlain we currently do not ask and just silently downconvert. In my proposal in the paragraph above, we would consult Send options. It seems to me to be the same as your next paragraph.

> 3a) is case of: all unknown==plain (by global user preference for unknown).
> The table is only for !convertible::plain, so we're talking about messages
> with relevant HTML formatting, but all recipients want plaintext. So we use
> send options to decide what to do. And my send options, like Magnus
> suggested, will always ask if there's loss of formatting, *at least* when
> it's convertible::no. (for convertible::yes and convertible::altering,
> silent downgrading might be acceptable if user agrees and everyone prefers
> plaintext).

Comment 92

[Thomas D. (:thomas8)]

OOps. I just noticed I forgot one case in the auto-detect send format results table of my proposal of attachment 8672330 [details].

Recipient preferences  Proposed      Current
6. plain & HTML        Both          --> Send Options [only for mixed]
                                         default: *both* (and any other settings are not feasible,
                                                          as I explained in benefits section of proposal)

My proposal forces "Both" for some mixed cases of auto-detect. I think that's reasonable and useful; it's also the current default for most mixed cases, so behaviour is similar to current, only better! :)

Of course we can add more preferences where user can explicitly choose prefered send format for mixed cases. E.g., we can add another 2 lines in options UI like following:

For messages in HTML format, if some recipients prefer plain text but others prefer HTML:
- Ask me what to do
- Try converting to plaintext, otherwise ask (if convertible::no)
- Send the message in HTML anyway
- Send the message in both plain text and HTML

It's possible to add user choice for mixed cases like that.
But by adding more options, we'll be on the way to offering WADA's full matrix, which is maybe too many options (and many possible combinations of the matrix are NOT useful as a default).

My proposal is also different because it presents the entire (Auto-Downgrade if convertible::plain) issue as an independent feature in options dialogue, because it really supersedes auto-detect. In my proposal, auto-downgrade can be simply switched on/off in general, and it comes with some options for defining on which scope of recipients auto-downgrade should be tried. WADA put auto-downgrade into the big matrix, which is certainly technically correct, but imo it's not advisable because it's really too much and too confusing, especially for users who'll just want the feature off but perhaps still need auto-detect.

Comment 93

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #90)
> WADA is against option of unknown==both, but I think it's necessary and useful.
> It's actually current default format for mixed cases if !convertible::plain.

I need option for all 6 cases.
  In any case, choosable actions should be same in all cases.
  We have right to send HTML mail which I composed in HTML mode in any format. 
  Even when allHtml, even if Tb says Convertible, I have right to silently send as text/plain. :-)
  (Please don't say to me "use Shift+Write, or dropdown of Write button")
  Even when allPlain, even if Tb says Not-Convertible,
  if I know recipient's smart phone can show HTML mail, I have right to send as text/hrml.
For all 6 cases, Send Options panel is too small, even when ordinal Notebook with not so small panel. 
So, options for Both should be removed to keep room for oimportant ptions in Send Options panel.

Comment 94

[WADA:World Anti-bad-Duping Agency]

Sorry, typo.
  Even when allHtml, even if Tb says Not-Convertible, I have right to silently send as text/plain. :-)

Comment 95

[:aceman]

(In reply to WADA from comment #94)
> Sorry, typo.
>   Even when allHtml, even if Tb says Not-Convertible, I have right to
> silently send as text/plain. :-)

Now this is a completely new requirement to me :( I always thought this is a clear case to send HTML (only) automatically and be done with it. If you really want to have this optional, then this will inflate the number of options. I always thought we want to make most cases automatic and really only bother Send Options or the user when necessary.

Comment 96

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #91)
> (In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment
> #90)
> > WADA and I assume that it's helpful for more user-controllable/predictable
> > auto-detection of best send format, if TB or user can pre-set the prefered
> > send format for recipients whose preference is "unknown", iow to
> > disambiguate "unknown" into "known".
> > In my proposal, the, choices are:
> > a) unknown==plain
> > b) unknown==html
> > c) unknown==both
> > So in the auto-detect results table, the different translations of unknown
> > must reflect accordingly.
> Ah, so this is another group of settings.

Please. It's not another "group of settings". It's a single setting (one line with single dropdownbox in the UI) which allows user to define the intended meaning of "Unknown". Which imo is VERY useful because it avoids all the Plain vs. HTML wars by giving the user a choice.

> It would surely be no problem to
> add them. But I worry we are getting into too many options, which Magnus was
> against.

The problem is that the current options try to pack all-in-one, but it fails miserably for many scenarios. In fact, some of the current options cannot really be used as a default, because they are mixing too many cases of prefers-plain and prefers-unknown. And we don't do what we promise in the options UI. At all. We just do what we want, Auto-Downgrade before auto-detect, even discarding full-fledged HTML messages with inline images without warning, when all recipients prefer-plain, in spite of promise in options "when not listed to receive HTML, send both | Ask | whatever".

I've added just *two* small options with dropdown boxes for feature of Auto-Detect. Please don't get confused by how it looks in the UI sketch of my proposal attachment 8672330 [details], the real UI is totally slim because it uses dropdown boxes for all the options, never full page as in the proposal.

One of the main painpoints of current recipient-centric auto-detect is that the most common recipient type, "prefers-unknown", is UNDEFINED. And auto-downgrade for alleged cases of convertible::plain comes with a forced bias for plaintext (while dumping loads of potential css styling in the process, because the algorithm is not sufficiently aware of styling). Undefined recipients are hard to handle in a recipient-centric model, worse when the same message goes out to recipients with different preferences (plain vs. HTML). So I think adding a one-line option with single dropdown for disambiguating "prefers-unknown" into any of {prefers: plain | html | both} is a very plausible and useful idea for making the outcome of recipient-centric auto-detect to be more controllable/predictable.

The other option I added (also just one line + dropdown), is a checkbox for enabling/disabling Auto-Downgrade. That feature is the reason for countless bugs on BMO, and 12000 users have downloaded addon to disable it. I think it's definitely needed that we allow users to send their HTML composed messages as HTML. It's so ridiculous to even talk about that. So if we want to keep the current default of recipient-centric auto-detect, we must disentangle that from message-centric auto-downgrade and give users a choice, short of adopting a message-centric model.

Problem of current send format design in Tb: We currently blend a recipient-centric model (auto-detect) with a message-centric model (auto-downgrade, and delivery format menu), but neither of them is fully developed and fully controllable by user. The net result is a big unpredictable mess which no one can understand, even developers are struggling. And the list of bugs filed by users is endless.

So to complete our message-centric use cases, I've added another *one* option to allow user to start composing with a default delivery format other than recipient-centric Auto-Detect. E.g., allow users to just have "send as HTML" as their default setting, without any of the auto-detect confusions/surprises. WYSIWYG. That's definitely needed if we agree that message-centric composition is the more likely scenario in 2015, where I cannot imagine many users manually setting any of their recipients to prefers-plain.

We could also think about ripping out the entire recipient-centric Auto-Detect thing, removing preferred send format from AB, and just keep Auto-Downgrade as an option. Recipient-centric auto-detection has certainly outlived its purpose. Auto-Detection is what makes everything so complicated, especially when it comes to downgrading to plaintext, which should never be uncontrolled.
Message-centric is much easier. For most messages composed in HTML, just send HTML only.
Allow user to default to Both if needed for his recipients. *Defaulting* to sending *plaintext* makes no sense for HTML editor (but we could still allow it for freaks); user per-message delivery format switch, or just use Plaintext editor instead. Then you'll notice that other big problem that we don't allow switching from plaintext editor to html editor (existing bugs with hundreds of votes).

> Also, this feature we are talking about is decided in the mailnews
> code so this would affect Seamonkey. They would have to get the same options
> and UI (or at least hardcode the prefs to some default values). Of course,
> it could be easier to include the "kitchen sink" into SM than into TB (which
> pretends to be the simpler/cleaner one). But I think we haven't yet heard
> any opinion from them. We probably missed to CC them to the relevant bugs.
> 
> > > Another question: can we just send Both if the mix of recipients is "all
> > > unknown" or "some unknown & some HTML"?
> > 
> > Basically yes, but "both" should not be sent if everyone prefers HTML,
> > because it's just a waste of bandwidth and unnecessarily increasing message
> > size.
> Sure. I was not asking about "All HTML" case. Only about "Some unknown (more
> than 1), some HTML (0 or more), and 0 plain text" recipients.

You misunderstand. You asked about case of "all unknown" or "some unknown & some HTML". In my proposal, these will automatically be translated according to the user's general translation for "unknown==prefers: plain | HTML | both. So "unknown" cases are no longer unknown, but known.
And one possible translation, if user sets "unknown==html" (which makes a lot of sense in 2015, and I havve that as a default in my proposal), is that "all-unknown"=="all-html", and "some-unknown & some html"=="all-html" also. Simple. Actually easier, as it reduces the number of cases auto-detect needs to handle.
 
> > > Only if there is at least one plain
> > > recipient, consult the Send options. Would that work?
> > 
> > No. As I tried to explain in the proposal, imo combined send options for
> > plain + any other preference will never work. We need separate options which
> > are only for allPlain case, because that case can be handled more radically
> > than any mixed cases. You can never set a radical downgrading as default for
> > mixed cases.
> 
> Why? If there is at least one plain text recipient, we must send the msg in
> the lowest common denominator.

NO. We MIGHT send the msg in lowest common denominator, if the user so wishes, and after warning the user about the dataloss of HTML formatting. What is "plain text recipient" in 2015? Even PINE text email client can handle MIME formatted messages, even basic HTML. Our UI has "prefers-plain", never "accept-nothing-but-plain". I think multipart/alternative (send msg as both plaintext and html) is exactly for such mixed cases of prefers-plain & prefers-html. Come on, if you are really composing to recipients that would choke on a multipart/mime message, please use plaintext editor instead.
Having said which, my proposal actually improves things for recipient-centric delivery to "prefers-plain", because I've reduced the scope of our traditional send options to "allPlain", so that users can choose more radical downgrade options for that type of recipients, without hurting themselves or others by silent, massive dataloss of styling, inline images etc.

> For HTML recipients, it is HTML, for unknown it is Both (which I asked
> above).

Well, I'd agree with WADA that sending "both" for unknown is not ideal, and not needed for most everyday scenarios, because everybody can read HTML-only. That's why I want to disambiguate "unknown" and let user decide what "unknown" should mean. I suggest defaulting to "unknown"=="prefers-html"; then we can avoid sending "both" for cases of "all-unknown", "some-unknown" etc. where sending both is often not needed. I think sending "both" is only needed when "prefers-plain" is involved, or if user wants "unknown"=="prefers-both", which is also reasonable if user suspects some "unknown" might not like or have problems with HTML, and wants to err on the safe side. And for conservative users who prefer plaintext, they can even make all their "unknown" friends to be considered as "prefers-plain". So everybody gets their will in a simple and transparent manner, instead of the current internal guesswork which makes many people unhappy. All that by the cost of one line in options with a single dropdown.

> For plaintext recipients, we consult Send options (so the user
> decides whether it is Ask me, HTML, Both, Plain).

You mean, we *will* consult send options, because currently we don't. For allPlain, we force sendformat::Plain *without* consulting the Send options (but Magnus has already agreed to fix it, so we'll do that soon). And because current send options are mixed-scope including "unknown" with unknown preference, we default send options to "silently send both" because any other option would be unsafe or nagging for unknown.

> So the Send options now
> decide the final result for all recipients. What is missing in that?

"Send options"== single current option "When sending messages in HTML format..."?

If yes, there's an inherent problem in current send options.
Text says "not listed as being able to receive HTML".
The correct language and current behaviour translation of that is the following scope:
prefers-plain and/or prefers-unknown
This was done under the assumption that prefers-unknown==prefers-plain.
But that assumption is wrong in 2015.
Worse, current send-options scope seems to cover all mixed cases, too.
So the net result is, that we follow send options (made for prefers-plaintext) even for cases of
prefers-unknown & prefers-html.

Mixed scope problems of current send options:

ASK is send option for prefers-plain; it's too intrusive for any other scope.
Plain is send option for prefers-plain; but it's too dangerous for any scope without per-message warning, full dataloss possible.
HTML is send option for unknown; it's never for prefers-plain. (I'll just ignore WADA's ideosyncracies on that).
BOTH is send option for mixed: prefers-plain & prefers HTML, prefers-plain & unknown, perhaps for prefers-unknown and prefers-html (but only if unknown==prefers-plain; what if unknown==prefers-html?); it's not ideal or cases of all-unknown==prefers-html; not ideal for cases of all-plain.

> For allPlain we currently do not ask and just silently downconvert. In my
> proposal in the paragraph above, we would consult Send options. It seems to
> me to be the same as your next paragraph.
> 
> > 3a) is case of: all unknown==plain (by global user preference for unknown).
> > The table is only for !convertible::plain, so we're talking about messages
> > with relevant HTML formatting, but all recipients want plaintext. So we use
> > send options to decide what to do.

> > And my send options, like Magnus
> > suggested, will always ask if there's loss of formatting, *at least* when
> > it's convertible::no. (for convertible::yes and convertible::altering,
> > silent downgrading might be acceptable if user agrees and everyone prefers
> > plaintext).

I have not seen a full proposal by you, can you point me to it? Maybe too much information on these bugs.

Comment 97

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #95)
> (In reply to WADA from comment #94)
> > Sorry, typo.
> >   Even when allHtml, even if Tb says Not-Convertible, I have right to
> > silently send as text/plain. :-)
> 
> Now this is a completely new requirement to me :( I always thought this is a
> clear case to send HTML (only) automatically and be done with it. If you
> really want to have this optional, then this will inflate the number of
> options. I always thought we want to make most cases automatic and really
> only bother Send Options or the user when necessary.

+1. If you want to make most cases automatic and match user's expectation, then I think you are in favor of making preference for "prefers-unknown" explicit for user...

Comment 98

[Thomas D. (:thomas8)]

(In reply to WADA from comment #94)
> Sorry, typo.
>   Even when allHtml, even if Tb says Not-Convertible, I have right to
> silently send as text/plain. :-)

Nonsense. Please stop it, WADA, you're creating even more confusion than we already have here. Sometimes I think you actually want some of your own bugs to fail and be closed as wontfix or whatever, in some sort of strategic game to achieve other goals. Please write your own addon if you want silent-dataloss. What's the point of composing not-convertible HTML and then send in plain text. We're not a converter, and even converters would have to ask you before dataloss downgrading.

I actually wonder if your full proposal of attachment 8672017 [details] is really serious, or you are silently hoping for wontfix of that proposal because it's too complex, and you're trying to prove the point that we can't let the user control everything. But I'll give you the benefit of the doubt, and also it would be good to actually see that in XUL, where it might look much better/simpler than on paper.

Comment 99

[Thomas D. (:thomas8)]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #98)
> (In reply to WADA from comment #94)
> > Sorry, typo.
> >   Even when allHtml, even if Tb says Not-Convertible, I have right to
> > silently send as text/plain. :-)

Silent massive dataloss can never be a right, and can never be allowed as a default. We should never encourage that use case because it does not make sense. In fact, for our purposes here, it can never be allowed for cases of convertible::no, even when you manually pick "send as plain", we must warn you. But that's a total edge case, so it's not worth talking about it.

Comment 100

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #96)
> You misunderstand. You asked about case of "all unknown" or "some unknown &
> some HTML". In my proposal, these will automatically be translated according
> to the user's general translation for "unknown==prefers: plain | HTML |
> both. So "unknown" cases are no longer unknown, but known.
> And one possible translation, if user sets "unknown==html" (which makes a
> lot of sense in 2015, and I havve that as a default in my proposal), is that
> "all-unknown"=="all-html", and "some-unknown & some html"=="all-html" also.
> Simple. Actually easier, as it reduces the number of cases auto-detect needs
> to handle.

Yes, I can understand this. If we could limit unknown to either HTML or plain, that would simplify the deciding. However, if you allow unknown=Both (which I think must be the default), we still have 3 classes of recipients.

> > > > Only if there is at least one plain
> > > > recipient, consult the Send options. Would that work?
> > > 
> > > No. As I tried to explain in the proposal, imo combined send options for
> > > plain + any other preference will never work. We need separate options which
> > > are only for allPlain case, because that case can be handled more radically
> > > than any mixed cases. You can never set a radical downgrading as default for
> > > mixed cases.
> > 
> > Why? If there is at least one plain text recipient, we must send the msg in
> > the lowest common denominator.
> 
> NO. We MIGHT send the msg in lowest common denominator, if the user so
> wishes, and after warning the user about the dataloss of HTML formatting.

Yes, that is what I say. If there is at least one Plain recipient, consult Send options. Now it is up to the user if he wants to send HTML/Both/Plain automatically, or to Ask him.

> Our UI has "prefers-plain", never "accept-nothing-but-plain".

Yes I know, and I pointed that out elsewhere. But it seems nobody wants to rename it. On the other hand, why would the sender bother to go into his recipients card in AB and explicitly selected Plain text preference? Only because the recipient requested so, because he can't read HTML or Both. I think otherwise nobody would bother.

> I think multipart/alternative (send msg as
> both plaintext and html) is exactly for such mixed cases of prefers-plain &
> prefers-html. Come on, if you are really composing to recipients that would
> choke on a multipart/mime message, please use plaintext editor instead.
You may have forgotten which recipients want what. You just have the HTML composition by default. You need an app (TB) to warn you one of your recipients wants plain. THEN you decide, not before starting composing.
"You"=common user like me :)

> > For HTML recipients, it is HTML, for unknown it is Both (which I asked
> > above).
> 
> Well, I'd agree with WADA that sending "both" for unknown is not ideal, and
> not needed for most everyday scenarios, because everybody can read
> HTML-only.

I think Both should be the default, but if anybody wants to override that, I can provide that option.

> That's why I want to disambiguate "unknown" and let user decide
> what "unknown" should mean. I suggest defaulting to
> "unknown"=="prefers-html"; then we can avoid sending "both" for cases of
> "all-unknown", "some-unknown" etc. where sending both is often not needed. I
> think sending "both" is only needed when "prefers-plain" is involved, or if
> user wants "unknown"=="prefers-both", which is also reasonable if user
> suspects some "unknown" might not like or have problems with HTML, and wants
> to err on the safe side. And for conservative users who prefer plaintext,
> they can even make all their "unknown" friends to be considered as
> "prefers-plain". So everybody gets their will in a simple and transparent
> manner, instead of the current internal guesswork which makes many people
> unhappy. All that by the cost of one line in options with a single dropdown.
> 
> > For plaintext recipients, we consult Send options (so the user
> > decides whether it is Ask me, HTML, Both, Plain).
> 
> You mean, we *will* consult send options, because currently we don't. For
> allPlain, we force sendformat::Plain *without* consulting the Send options
> (but Magnus has already agreed to fix it, so we'll do that soon).

Yes. I thought I will fix this somewhere here in these bugs:) Does he have other plans now?

 And
> because current send options are mixed-scope including "unknown" with
> unknown preference, we default send options to "silently send both" because
> any other option would be unsafe or nagging for unknown.
> 
> > So the Send options now
> > decide the final result for all recipients. What is missing in that?
> 
> "Send options"== single current option "When sending messages in HTML
> format..."?
> 
> If yes, there's an inherent problem in current send options.
> Text says "not listed as being able to receive HTML".
> The correct language and current behaviour translation of that is the
> following scope:
> prefers-plain and/or prefers-unknown
> This was done under the assumption that prefers-unknown==prefers-plain.
> But that assumption is wrong in 2015.
> Worse, current send-options scope seems to cover all mixed cases, too.
> So the net result is, that we follow send options (made for
> prefers-plaintext) even for cases of
> prefers-unknown & prefers-html.

Of course, I would change the wording for the Send options.

Comment 101

[:aceman]

> I have not seen a full proposal by you, can you point me to it? Maybe too
> much information on these bugs.
I have not posted it yet:)

Here it is. Consider a message that is HTML and is not convertible (convertibility can be influenced by the user via patch in bug 136502):
Recipient types                 Resulting format
HTML                            HTML (only)
HTML & Unknown                  Both
HTML & Unknown & Plain          Send Options
Unknown & Plain                 Send Options
HTML & Plain                    Send Options
Plain                           Send Options

For this we would not need to change current UI and not change any options in Send Options (just rename it that it applies only if there are some explicit plain text recipients).

I agree that your proposal of specifying whether unknown is HTML, Both or Plain (and keeping my rules above) will move many more messages to be sent as HTML or Both. For you or some people that may mean that ALL messages will be sent as HTML (only) or Both, because there will be no Plain text recipients in AB.

E.g. if you set unknown=>HTML then you get
Recipient types                 Resulting format
HTML                            HTML (only)
HTML & Unknown                  HTML (only)
HTML & Unknown & Plain          Send Options
Unknown & Plain                 Send Options
HTML & Plain                    Send Options
Plain                           Send Options

Now if the message IS convertible, we could have these rules:
Recipient types                 Resulting format
HTML                            HTML (only)
HTML & Unknown                  Both
HTML & Unknown & Plain          I am not sure here?
Unknown & Plain                 Plain?
HTML & Plain                    I am not sure here?
Plain                           Plain

Comment 102

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #99)
> (In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment
> #98)
> > (In reply to WADA from comment #94)
> > > Sorry, typo.
> > >   Even when allHtml, even if Tb says Not-Convertible, I have right to
> > > silently send as text/plain. :-)
> 
> Silent massive dataloss can never be a right, and can never be allowed as a
> default. We should never encourage that use case because it does not make
> sense. In fact, for our purposes here, it can never be allowed for cases of
> convertible::no, even when you manually pick "send as plain", we must warn
> you. But that's a total edge case, so it's not worth talking about it.

I am sure he does not want it as default. Also, we do allow the downconversion if recipients are plain and certain values of Send options. Just today and also in my proposal we never check Send Options if the case is so clear as allHtml recipients. I tought we want to shortcut and automate most cases so that change of format or asking user is used very scarcely. So forcing TB to ask user (or check options) even if allHTML is new to me.

Comment 103

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #95, with skipping many comments again
> Now this is a completely new requirement to me :( I always thought this is a
> clear case to send HTML (only) automatically and be done with it. If you
> really want to have this optional, then this will inflate the number of
> options. I always thought we want to make most cases automatic and really
> only bother Send Options or the user when necessary.

Who mixed (B-2) "if allPlain && Covertible, send as text/plaain" case was not me. You and Magnus.
I cited problem in (C-1) "if (!allPlain && !allHTML) && Covertible), send as text/plain" only since initial.
Who re-opened this bug is not me.

Purpose of the "Full Matrix" is to find best way in each 6 cases separately.
My "Full Matrix" is a kind of thought experiment(gedankenexperiment, or Gedankenexperiment) to find best solution for any people, to find best design of Send Options panel in 21-th Century.

Who posted many comments about "Full Matrix", "possible change in Send Options panel", was not me. You and Tohmas D.

I believe that without "Table in User Story" or "Full Matrix", no one except you, who wrote one-liner patch, says that "if allText && Not-Convertible, send as text/plain" may produce "format loss", "data loss" for users.
I believe that someone won't/can't try to propose improvement in Send Options panel without idea of "if(allHtml||allPlain||(!allHtml||!allPlain)) UseSendOptions" and "Full Matrix", as seen in two bugs opened by someone, as seen in bugs where many coments were posted from someone. I believe that someone continued posting comments of complaint. I believe that someone merely repeated comment like "full control by user is needed or mandatory".

I'm simply waitng for your review comment on a pseudo code for "user's choice of default for Prefers=Unknown" in this bug for next step, after WONTFIX of this bug which is request for "one-liner-patch" by you or me.
As I already stated, I'll don't touch this bug which was not re-opened by me.

Comment 104

[WADA:World Anti-bad-Duping Agency]

To aceman. my thinkig was as follows.
1. During analysis of your one-line patch, I knew next:
   UseSendOption in both (C-1) and (C-2) === share single Send Options panel between (C-1) and (C-2).
   Because current Send Options code aaand Send options panel is designed/implemented for )C-2) only,
   shsaring panel between multipla cases may produce problem.
2. If so, if option for both (C-1) and (C-2) are placed in Send Options panel, no problem.
   If user sets text/html instead of text/plain, problem of this bug is resolved.
3. Why Send Options panel should be used for mixed case only?
   If options for (B-2) is added to panel, "data loss in allPain" is resolved.
4. If so, why "UseSendOptions in all 6 case" can be wrong or invalid or inproper?
5. Because send logic was cleanly sorted out in 2014, following change was sufficient for it in send logic.
     if(allHtml){...}elseif(allPlain){...}else if(Convertible){...}else UseSendOption
     => if( allHtml || allPlain || (!allHtml && !allPlain) ){ UseSendOptions; }
     In Send Options logic, aConvertible value can be freely utilized.

"Removing excess, not-needed, not-important element from full package" is easier than "Adding each element when it is found needed, based on importance or priority".
This is reason why I presented "A Full Solution" in this bug.
It looks a representation of ""full control by user is needed or mandatory" of Thomas D.

Comment 105

[:aceman]

(In reply to WADA from comment #103)
> Who mixed (B-2) "if allPlain && Covertible, send as text/plaain" case was
> not me. You and Magnus.

This is current behaviour which I understood is a problem (IF convertibility detection is incorrect, but as that may change I am starting to think we do not need to solve this). So is it or is it NOT behaving incorrectly now?
I think this was bug 1204379, filed by you, where you wanted a quick fix until the convertibility is fixed elsewhere.
I admit that even in that bug you say "if allPlain && Covertible, send as text/plain" is correct behaviour for you.
Can you just confirm that you still think so?

> I cited problem in (C-1) "if (!allPlain && !allHTML) && Covertible), send as
> text/plain" only since initial.

OK.

> Who re-opened this bug is not me.

So where is your proposal in the User Story table? Maybe I would then understand better.

I see that for allHTML TB should send SF::HTML in all 4 proposals. You suddenly say you want the option to send plain text in that case. That is not seen in the table therefore I wonder.

> Purpose of the "Full Matrix" is to find best way in each 6 cases separately.
> My "Full Matrix" is a kind of thought experiment(gedankenexperiment, or
> Gedankenexperiment) to find best solution for any people, to find best
> design of Send Options panel in 21-th Century.
> 
> Who posted many comments about "Full Matrix", "possible change in Send
> Options panel", was not me. You and Tohmas D.

Where is this Full Matrix? I see nothing by this name till now.

> I believe that without "Table in User Story" or "Full Matrix", no one except
> you, who wrote one-liner patch, says that "if allText && Not-Convertible,
> send as text/plain" may produce "format loss", "data loss" for users.

Yes it causes data loss. Other thing is whether users agree to it or not. OK, if detection of convertibility will work correct (bug 414299), there is no dataloss and we can keep this case as is (For those that think there is still loos, I proposed bug 136502. So then why in the User story table it is written Magnus wants "Ask user", disregarding Send options? Where did he say so? This would also mean he does not agree with you. Anyway, if that is true, this would change the situation.

> I believe that someone won't/can't try to propose improvement in Send
> Options panel without idea of "if(allHtml||allPlain||(!allHtml||!allPlain))
> UseSendOptions" and "Full Matrix", as seen in two bugs opened by someone, as
> seen in bugs where many coments were posted from someone. I believe that
> someone continued posting comments of complaint. I believe that someone
> merely repeated comment like "full control by user is needed or mandatory".

Is that "someone" Thomas?:) I have now seen request from you for "full control" even for cases that seems clear to be automated (allHtml -> HTML).
 
> I'm simply waitng for your review comment on a pseudo code for "user's
> choice of default for Prefers=Unknown" in this bug for next step, after
> WONTFIX of this bug which is request for "one-liner-patch" by you or me.
> As I already stated, I'll don't touch this bug which was not re-opened by me.

Sorry I do not understand the code. Can you please make it more readable to me?

Comment 106

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #105)
> (In reply to WADA from comment #103)
> > Who mixed (B-2) "if allPlain && Covertible, send as text/plaain" case was
> > not me. You and Magnus.
> 
> This is current behaviour which I understood is a problem (IF convertibility
> detection is incorrect, but as that may change I am starting to think we do
> not need to solve this). So is it or is it NOT behaving incorrectly now?
> I think this was bug 1204379, filed by you, where you wanted a quick fix
> until the convertibility is fixed elsewhere.
> I admit that even in that bug you say "if allPlain && Covertible, send as
> text/plain" is correct behaviour for you.
> Can you just confirm that you still think so?

As I said to Magnus "if you think data loss, please open new bug for it", I'm not interested in it :-)
I never said "nver data loss". I merely said "this bug is for (C-1), not for (B-2)".

> > Who re-opened this bug is not me.
> So where is your proposal in the User Story table? Maybe I would then understand better.

I unsertand reason of WONTFIX and accepted WONTFIX. So I'm not responsible to keep this bug meaningful or useful any more. It's all up to who re-opened this bug.
This bug is already in "state of Unable-to-understand". There is no big difference between "state of Unable-to-understand" and "state of Unable-to-understand" * "state of Unable-to-understand".
If this bug will be closed as WONTFIX in near future, there is no problem in "state of Unable-to-understand" * "state of Unable-to-understand". :-)
So, I'm watching mock-up of "Send Options panel for 21-th Century" in this bug.

> I see that for allHTML TB should send SF::HTML in all 4 proposals. You
> suddenly say you want the option to send plain text in that case. That is
> not seen in the table therefore I wonder.

"Same options in all case" is better in design/implementaation, isn't it?
I think "Remove Plain Text Only form several cases only" is wasting of code.
Why "prohibiting Text if allHtml" is mandatory? I think "no exception", "same in all cases" is better.

> Sorry I do not understand the code. Can you please make it more readable to me?

Thanks for looking into it.
OK. I'll update it with explanations, more code around change, and will ask you for comment again.

Comment 107

[WADA:World Anti-bad-Duping Agency]

Attachment: (v 4) Pseudo code for user's choice of Default for Unknown with new pref, and convert Unknown to HTML or Text to get out of mystery by Auto-downgrade

All relevant code lines are included.

Comment 108

[Thomas D. (:thomas8)]

Attachment: XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

So along my proposal T1 of attachment 8672330 [details], here's an XUL Demo of a slightly modified Proposal T2, for our Global Send Options Dialogue.

Key Benefits:

1.) Allow message-centric users who don't care about the format preferences of their recipients to opt out of recipient-centric Auto-Detect completely, by setting their default delivery format to any of Auto-Detect | HTML | Both | convert-to-plain (ask if not convertible). As seen on bugs, many users would just like to send HTML always, never plaintext. Of course they can still always change it per-message from the menu of delivery formats. Let's face it: Auto-Detect was made to send Plaintext as often as possible when HTML was still a problem. But we have already changed that to send "Both" most of the time, which is actually a bad default in times of HTML. Let the modern user choose how he normally wants to send any message. Have a global default, but also allow different defaults for different identities, e.g. Newsgroup identities etc. Here we serve users who prefer stable defaults over Auto-Detect, which unfortunately never tells you in advance which format it will finally choose. Now you can be SURE that even old Unce Sam who prefers-plain, between 100 others who prefer-html, can never cause message-degradation to plaintext.

2.) Disambiguate "Unknown" for better Auto-Detection, and in the 21st century, default to HTML-only for "Unknown". But let the user choose, perhaps he prefers "prefers-Both" for unknown, or even "prefers-plain". By default, for cases of !convertible::plain, let's send HTML-only more often where we currently default to sending "both". The current default of multipart/alternative ("Both" plaintext AND HTML) is not very good because it increases message sizes all across the board without much benefit in real life.

3.) Incorporate the ultimate fix for countless bugs - we need an explicit, global Switch allowing user to opt out of Auto-Downgrade. Aceman is trying this in Bug 136502. I'm not sure if we need this per-identity (see 1.), nor if it should be remembered per-identity when changed per-message, but I think global default setting for all identities is needed. I understand that Auto-Downgrade is mainly message-centric. It looks at the message, and if there's very little/no formatting, it sends plaintext. So if enabled, and message is convertible::plain, let it just supersede Auto-Detect and send plaintext in all cases, even when everyone prefers HTML. Mixing message-centric and recipient-centric for some, but not all cases is very confusing. I think this is good news for Magnus and Ben - all your simple, "non-formatted" messages will just get sent as plain text, no matter what. Recipients who prefer HTML can certainly read plain, too.

4.a) New, separate Send Options for allPlain case: I imagine that those users who actually have Uncle Sam who can't handle HTML (prefers-plain) will want to downgrade to plaintext as often as possible. But radical downgrading is only possible if we don't spoil it for other recipients in the process.
4.b) Keep existing send options for cases involving SOME prefers-plain. If someone else for the same message prefers HTML, user will probably want both. Or, if Uncle Sam is more important than all others, user can choose radical downgrading for mixed, too.

5.) Clean up and clarify the whole Send Options Dialogue. Fix visual hierarchy shortcomings of the current dialogue. E.g., make it clear that all options are only about HTML messages, and that most options are only relevant for delivery-format:Auto-Detect, including HTML/Plain Domains, whose description I have improved/corrected.

6.) I skipped [Restore Default] button(s) in this XUL Demo, but we need one OR perhaps several of them somewhere.

Comment 109

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #101)
> > I have not seen a full proposal by you, can you point me to it? Maybe too
> > much information on these bugs.
> I have not posted it yet:)
> 
> Here it is. Consider a message that is HTML and is not convertible
> (convertibility can be influenced by the user via patch in bug 136502):
> Recipient types                 Resulting format
> HTML                            HTML (only)
> HTML & Unknown                  Both

If "Unknown" is known (via user pref), we can be better here. E.g. for unknown==html, just send html.
Which also improves very frequent case of ALL-unknown, which unfortunately you forgot in your list.
So what's your default for allUnknown?

> HTML & Unknown & Plain          Send Options - *both*
> Unknown & Plain                 Send Options - *both*
> HTML & Plain                    Send Options - *both*
> Plain                           Send Options - *both*

It's possible to bundle send options for all cases involving plain; however, users who care about the integrity of their message can never choose anything but "Both". And it's hard choice for users who consider unknown==prefers-html. What they probably want for old uncle SAM is "Plain", but that's feasible only if ALL recipients are prefers-plain.

> For this we would not need to change current UI and not change any options
> in Send Options (just rename it that it applies only if there are some
> explicit plain text recipients).

> I agree that your proposal of specifying whether unknown is HTML, Both or
> Plain

letting the USER specify!

> (and keeping my rules above) will move many more messages to be sent
> as HTML or Both.

No. It will move many messages to be sent as HTML-only instead of BOTH, which is the current default.
Iow, it will shrink the size of many messages by default, as we avoid to send the duplicate plaintext part where not needed or wanted.

> For you or some people that may mean that ALL messages will
> be sent as HTML (only) or Both, because there will be no Plain text
> recipients in AB.

Hmmm. Depends. If you keep your Auto-Downgrade option ON, we will still send plaintext for all simple / "unformatted" messages by default. Otherwise, we'll send less Both, and more HTML-only by default, which is clearly good. Only if you want. User can choose.

> E.g. if you set unknown=>HTML then you get
> Recipient types                 Resulting format
> HTML                            HTML (only)
> HTML & Unknown                  HTML (only)
> HTML & Unknown & Plain          Send Options - *Both*
> Unknown & Plain                 Send Options - *Both*
> HTML & Plain                    Send Options - *Both*
> Plain                           Send Options - *Both*
> 
> Now if the message IS convertible, we could have these rules:

What, in your model or in mine?

> Recipient types                 Resulting format
> HTML                            HTML (only)

Ah well, if somebody agrees to downgrading HTML messages *at all*, then he'll probably be fine with it even for HTML recipients. The meaning of "prefers-html" is certainly NOT that the recipient can't handle or hates plaintext mails if there's no significant formatting anyway (after we fix auto-downgrade algorithm to preserve styles, <tt>, <pre>, links etc.). Prefers-HTML was only needed at a time where the assumption was that most recipients are Prefers-Plain. In fact, I think we could just rip out Prefers-HTML without any problem, if we provide a user-defined default setting for prefers-unknown. I think prefers-HTML is more like saying prefers-plain==false, and to avoid send option of ASK because it was too annoying. But we no longer default to ASK, so it's all history.

> HTML & Unknown                  Both
> HTML & Unknown & Plain          I am not sure here?
> Unknown & Plain                 Plain?
> HTML & Plain                    I am not sure here?
> Plain                           Plain

You see, isconvertible::Plain is ALWAYS confusing... ;)
We currently Downgrade for ALL cases except allHTML.
I propose that as soon as we provide other means of ensuring "HTML" always, it's safe and simple to have the following predictable and consistent behaviour:

auto-downgrade==True -> If the message is convertible::plain, ALWAYS send Plaintext WITHOUT looking at any recipient preferences (even when all are marked/considered "prefers-HTML").

Auto-downgrade is message-centric, so we can just skip recipient-centric auto-detect. It's weird that we currently respect all-prefers-HTML (and send HTML even if convertible), but we do NOT respect if only SOME-prefers-html. So prefers-html is meaningless even in current behaviour. And current behaviour never cares about cases of "unknown", if they want plaintext or not, nor if the user wants downgrade to plaintext for unknown recipients.

Hey Ben. The unbelievable has happened. ThomasD. advocates for an EXPANSION of the default recipient scope for Auto-Downgrade... (BECAUSE it's now optional, never forced onto user... My proposal provides 2 ways to opt out of Auto-Downgrade, either directly, or indirectly by opting out of Auto-Detect and picking a default send format.)

Comment 110

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #109)
> > Here it is. Consider a message that is HTML and is not convertible
> > (convertibility can be influenced by the user via patch in bug 136502):
> > Recipient types                 Resulting format
> > HTML                            HTML (only)
> > HTML & Unknown                  Both
> 
> If "Unknown" is known (via user pref), we can be better here. E.g. for
> unknown==html, just send html.

Yes. I said that later in my comment.

> Which also improves very frequent case of ALL-unknown, which unfortunately
> you forgot in your list.

Good catch :)

> So what's your default for allUnknown?

My default would be Both.

> > HTML & Unknown & Plain          Send Options - *both*
> > Unknown & Plain                 Send Options - *both*
> > HTML & Plain                    Send Options - *both*
> > Plain                           Send Options - *both*
> 

In my model, the Send Options would have a single setting (from the list of formats), that would be applied in all these cases. As I said, the setting would be tuned to Plain text users, the lowest common denominator.

> It's possible to bundle send options for all cases involving plain; however,
> users who care about the integrity of their message can never choose
> anything but "Both". And it's hard choice for users who consider
> unknown==prefers-html. What they probably want for old uncle SAM is "Plain",
> but that's feasible only if ALL recipients are prefers-plain.

It seems according to WADA and/or Magnus, the last line should be:
Plain                            Plain (or Ask)

> > For this we would not need to change current UI and not change any options
> > in Send Options (just rename it that it applies only if there are some
> > explicit plain text recipients).
> 
> > I agree that your proposal of specifying whether unknown is HTML, Both or
> > Plain
> 
> letting the USER specify!
> 
> > (and keeping my rules above) will move many more messages to be sent
> > as HTML or Both.
> 
> No. It will move many messages to be sent as HTML-only instead of BOTH,
> which is the current default.
> Iow, it will shrink the size of many messages by default, as we avoid to
> send the duplicate plaintext part where not needed or wanted.
> 
> > For you or some people that may mean that ALL messages will
> > be sent as HTML (only) or Both, because there will be no Plain text
> > recipients in AB.
> 
> Hmmm. Depends. If you keep your Auto-Downgrade option ON, we will still send
> plaintext for all simple / "unformatted" messages by default. Otherwise,
> we'll send less Both, and more HTML-only by default, which is clearly good.
> Only if you want. User can choose.
> 
> > E.g. if you set unknown=>HTML then you get
> > Recipient types                 Resulting format
> > HTML                            HTML (only)
> > HTML & Unknown                  HTML (only)
> > HTML & Unknown & Plain          Send Options - *Both*
> > Unknown & Plain                 Send Options - *Both*
> > HTML & Plain                    Send Options - *Both*
> > Plain                           Send Options - *Both*
> > 
> > Now if the message IS convertible, we could have these rules:
> 
> What, in your model or in mine?

Mine ;)

> 
> > Recipient types                 Resulting format
> > HTML                            HTML (only)
> 
> Ah well, if somebody agrees to downgrading HTML messages *at all*, then
> he'll probably be fine with it even for HTML recipients. The meaning of
> "prefers-html" is certainly NOT that the recipient can't handle or hates
> plaintext mails if there's no significant formatting anyway (after we fix
> auto-downgrade algorithm to preserve styles, <tt>, <pre>, links etc.).
> Prefers-HTML was only needed at a time where the assumption was that most
> recipients are Prefers-Plain. In fact, I think we could just rip out
> Prefers-HTML without any problem, if we provide a user-defined default
> setting for prefers-unknown. I think prefers-HTML is more like saying
> prefers-plain==false, and to avoid send option of ASK because it was too
> annoying. But we no longer default to ASK, so it's all history.

Well, that is the current behaviour and it didn't seem to be controversial. Do you now propose to send plain text even to allHTML recipients if the message is convertible? Yeah, you do later in the comment.

> > HTML & Unknown                  Both
> > HTML & Unknown & Plain          I am not sure here?
> > Unknown & Plain                 Plain?
> > HTML & Plain                    I am not sure here?
> > Plain                           Plain
> 
> You see, isconvertible::Plain is ALWAYS confusing... ;)
> We currently Downgrade for ALL cases except allHTML.

Yes.

> I propose that as soon as we provide other means of ensuring "HTML" always,
> it's safe and simple to have the following predictable and consistent
> behaviour:
> 
> auto-downgrade==True -> If the message is convertible::plain, ALWAYS send
> Plaintext WITHOUT looking at any recipient preferences (even when all are
> marked/considered "prefers-HTML").

I would be fine with this. Any other supporters? :)

Comment 111

[Thomas D. (:thomas8)]

WADA, FYI:

Here's how I shrinked the total experimental matrix of your attachment 8672017 [details], and why.
Here's how I integrated "Both" without increasing number of options in Matrix.

WADA's send option matrix (based on pref for unknown==plain|html)
1) allPlain <-- this case is important, and deserves its own separate option.
2) allHTML  <-- not needed
3) mixed    <-- mixed is only interesting if "plain" is involved (that's the currently existing option)

For each of these:
- convertible(::plain, I assume) - AutoDowngrade case
- not convertible

That's overkill, and I understand it was just a Gedankenexperiment.
Btw I never demanded TOTAL user control, but just that we must offer *default* control which allows sending HTML always, and regarding "unknown==html".

So WADA's Gedankenexperiment had 6 options in total.
I took out three because Auto-Downgrade is inherently prefers-plain for cases of "no formatting", so it is nonsensical to allow the user to define combinations like:
- when allHTML -> auto-downgrade
- when allPlain -> send HTML
message-centric Auto-Downgrade already supersedes recipient-centric Auto-Detect for ALL scopes except "allHTML". If optional, we think we can just allow auto-downgrade for that case, too.
It's not very predictable currently that auto-degrade ignores 5 out of 6 possible recipient scope combinations, but respects 1.

Why allowing "unknown==both" does not require expanding the matrix:

All scope combinations with "unknown==plain|html|both":
-> and the resulting delivery-format for auto-detect, in my proposal T2 of attachment 8672688 [details]
1) allPlain                   -> * send options for scope of "allPlain"
2) allHTML                    -> - force HTML (obvious)
3) mixed: Plain & HTML        -> * send options for scope of "mixed involving somePlain" (default:both)
4) mixed: Plain & Both        ->   send options for scope of "mixed involving somePlain" (default:both)
5) mixed: Plain & HTML & Both ->   send options for scope of "mixed involving somePlain" (default:both)
6) mixed: Both & HTML         -> - force Both (user deliberately set unknown=="both" so that should be respected; both is never wrong for prefers-HTML recipients)
7) allBoth (from unknown-pref)-> - force Both (obvious)

For all recipient scopes involving somePlain, we can use same option. That option is important because downgrading might be wanted to protect old Uncle Sam. Although I never came across any Uncle Sam who cannot receive HTML messages. But we default to "Both", so unknown==both will be respected by default, and no dataloss. User can decide which is more important: respect prefers-HTML/Both by defaulting to "both" (no dataloss), or respecting Uncle Sam (prefers-plain) by defaulting to try-downconverting. If user doesn't know what is good, even ASK is still available. But will happen only rarely in 2015, because most of Uncle Sam's friends and their plaintext-only mailers have already died.

So imo having only the two scopes indicated by * is sufficient for all cases, even though "unknown==both" is allowed.

Comment 112

[Thomas D. (:thomas8)]

WADA, FYI:

You suggested somewhere that when creating new contacts with "unknown==html|plain", we should actually switch each contact's preference to prefers-html|plain upon creation.
I think NOT. It's much better/safer/more sustainable to keep "unknown" in contact properties, and have global setting how to translate unknown. If you get wiser or just change your mind, you can always change global "unknown" according to your needs, which is not possible when the current value of unknown has already been hard-saved into every contact, so it's no longer distinct as a member of the unknown group. Instead of killing prefers-unknown from contact properties, we might consider killing prefers-HTML. Or just kill nothing to avoid some users shouting. You never know. They can shout from all corners. :)

Comment 113

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #110)
> (In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment
> #109)
> > > Here it is. Consider a message that is HTML and is not convertible
> > > (convertibility can be influenced by the user via patch in bug 136502):
> > > Recipient types                 Resulting format
> > > HTML                            HTML (only)
> > > HTML & Unknown                  Both
> > 
> > If "Unknown" is known (via user pref), we can be better here. E.g. for
> > unknown==html, just send html.
> 
> Yes. I said that later in my comment.
> 
> > Which also improves very frequent case of ALL-unknown, which unfortunately
> > you forgot in your list.
> 
> Good catch :)
> 
> > So what's your default for allUnknown?
> 
> My default would be Both.

That's boring ;) To allow unknown==HTML is more interesting and efficient...

> 
> > > HTML & Unknown & Plain          Send Options - *both*
> > > Unknown & Plain                 Send Options - *both*
> > > HTML & Plain                    Send Options - *both*
> > > Plain                           Send Options - *both*
> > 
> 
> In my model, the Send Options would have a single setting (from the list of
> formats), that would be applied in all these cases. As I said, the setting
> would be tuned to Plain text users, the lowest common denominator.

But sending plaintext as lowest common denominator is potentially massive HTML dataloss of user's composed HTML for all other involved non-plain-recipients, so we'd definitely need to ASK, as Magnus said, and setting plaintext would never be good default for mixed.

> > It's possible to bundle send options for all cases involving plain; however,
> > users who care about the integrity of their message can never choose
> > anything but "Both". And it's hard choice for users who consider
> > unknown==prefers-html. What they probably want for old uncle SAM is "Plain",
> > but that's feasible only if ALL recipients are prefers-plain.
> 
> It seems according to WADA and/or Magnus, the last line should be:
> Plain                            Plain (or Ask)

??? Where did they say that, and what does it mean?
You mean that they want that
scope of allPlain should not be included in user-configurable send options scope,
but be hard-coded in Auto-detect as send-Plain (or Ask if not convertible)?

Now that's a regression. We currently
> force allPlain --> send-plain silently.
That's massive dataloss potential (apart from not matching the send options text), violating ux-error-prevention, so we can't continue doing that.

But if we kill the silent dataloss by asking, it means we will
> force allPlain --> Asking (in cases with inconvertible formatting).
But we already decided that we want to avoid asking as far as possible, so we don't want that.

So we could try to
> force allPlain --> Both.
Or maybe we couldn't. Assuming that prefers-plain is really for outdated recipients, maybe they will actually choke on multipart/alternative, or really hate it and get a heart attack, or whatever...
So I think as long as we're still keeping the whole Auto-Detect circus *at all*, which is clearly only existing to protect "prefers-plain", we should *fully* support that outdated usecase and not stop half the way down.

So whichever way we force,
allPlain --> plain-silent | plain-ASK-if-not-convertible | Both-silent
None of them really works for all plain text scenarios/user types/recipient types.

So I think allPlain scope must be in options.
But if we want to really support prefers-plain scenarios, we shouldn't bundle all-Plain with mixed-plain cases where others prefer HTML. It's better to keep allPlain case separate so that user can set aggressive downgrading there if needed, but have "Both" for mixed cases. Or aggressive downgrading for mixed, if he so prefers. We default to "both" always, so we never ask unless user sends plain-only explicitly.

Or rip out recipient-centric Auto-Detect and per-recipient-format-preferences altogether, but keep Auto-Downgrade as an option.



 
> > > For this we would not need to change current UI and not change any options
> > > in Send Options (just rename it that it applies only if there are some
> > > explicit plain text recipients).
> > 
> > > I agree that your proposal of specifying whether unknown is HTML, Both or
> > > Plain
> > 
> > letting the USER specify!
> > 
> > > (and keeping my rules above) will move many more messages to be sent
> > > as HTML or Both.
> > 
> > No. It will move many messages to be sent as HTML-only instead of BOTH,
> > which is the current default.
> > Iow, it will shrink the size of many messages by default, as we avoid to
> > send the duplicate plaintext part where not needed or wanted.
> > 
> > > For you or some people that may mean that ALL messages will
> > > be sent as HTML (only) or Both, because there will be no Plain text
> > > recipients in AB.
> > 
> > Hmmm. Depends. If you keep your Auto-Downgrade option ON, we will still send
> > plaintext for all simple / "unformatted" messages by default. Otherwise,
> > we'll send less Both, and more HTML-only by default, which is clearly good.
> > Only if you want. User can choose.
> > 
> > > E.g. if you set unknown=>HTML then you get
> > > Recipient types                 Resulting format
> > > HTML                            HTML (only)
> > > HTML & Unknown                  HTML (only)
> > > HTML & Unknown & Plain          Send Options - *Both*
> > > Unknown & Plain                 Send Options - *Both*
> > > HTML & Plain                    Send Options - *Both*
> > > Plain                           Send Options - *Both*
> > > 
> > > Now if the message IS convertible, we could have these rules:
> > 
> > What, in your model or in mine?
> 
> Mine ;)
> 
> > 
> > > Recipient types                 Resulting format
> > > HTML                            HTML (only)
> > 
> > Ah well, if somebody agrees to downgrading HTML messages *at all*, then
> > he'll probably be fine with it even for HTML recipients. The meaning of
> > "prefers-html" is certainly NOT that the recipient can't handle or hates
> > plaintext mails if there's no significant formatting anyway (after we fix
> > auto-downgrade algorithm to preserve styles, <tt>, <pre>, links etc.).
> > Prefers-HTML was only needed at a time where the assumption was that most
> > recipients are Prefers-Plain. In fact, I think we could just rip out
> > Prefers-HTML without any problem, if we provide a user-defined default
> > setting for prefers-unknown. I think prefers-HTML is more like saying
> > prefers-plain==false, and to avoid send option of ASK because it was too
> > annoying. But we no longer default to ASK, so it's all history.
> 
> Well, that is the current behaviour and it didn't seem to be controversial.
> Do you now propose to send plain text even to allHTML recipients if the
> message is convertible? Yeah, you do later in the comment.
> 
> > > HTML & Unknown                  Both
> > > HTML & Unknown & Plain          I am not sure here?
> > > Unknown & Plain                 Plain?
> > > HTML & Plain                    I am not sure here?
> > > Plain                           Plain
> > 
> > You see, isconvertible::Plain is ALWAYS confusing... ;)
> > We currently Downgrade for ALL cases except allHTML.
> 
> Yes.
> 
> > I propose that as soon as we provide other means of ensuring "HTML" always,
> > it's safe and simple to have the following predictable and consistent
> > behaviour:
> > 
> > auto-downgrade==True -> If the message is convertible::plain, ALWAYS send
> > Plaintext WITHOUT looking at any recipient preferences (even when all are
> > marked/considered "prefers-HTML").
> 
> I would be fine with this. Any other supporters? :)

Comment 114

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #111)
> WADA's send option matrix (based on pref for unknown==plain|html)
> 1) allPlain <-- this case is important, and deserves its own separate option.

This was the one which was needed to use Send Options for (B-2) by aceman's one-liner patch.
Current Send Options panel is for (C-2) only and designed/implemented for (C-2) only.

> 2) allHTML  <-- not needed
> 3) mixed    <-- mixed is only interesting if "plain" is involved (that's the currently existing option)

This was the one which was needed to use Send Options for (C-1) by aceman's one-liner patch and my one-liner patch.
Current Send Options panel is for (C-2) only and designed/implemented for (C-2) only.

Without proposal of change of Send Options panel(needless to say, change of backend code for the UI change too), patch such as one-liner patch by aceman, one-liner patch by me, can't be used.

This bug is never request for for change in Send Options panel and associated UX code.
It's reason why WONTFIX of this bug is reasonable and proper action at bugzilla.mozilla.org.

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #112)
> You suggested somewhere that when creating new contacts with
> "unknown==html|plain", we should actually switch each contact's preference
> to prefers-html|plain upon creation.

I never suggested such thing. When, in which comment, such suggetion by me?
Read through comments in a bug well before posting your comment, please.
It's by Bug 56834 which was opened on 2000-10-16.
See possible pseudo code for "user's choice of default for Unknown".
I borrowed "mandatory prefs setting for that bug" only from that bug.

Needless to say, proposed new prefs setting can be used by "creation of new Contact".
However, I believe "keep Unknown in Address Book" is far better if change like "user's choice of default for Unknown" will be made,,because there is no need of Prefers=HTML or Prefers=Text of all Contacts in all Address Books, For majrity of Tb users, "Default=HTML, with setting Prefers=Text for actually required Contacts only" is sufficient.

Comment 115

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #108)
> XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

Changed to text/plain, because following error occurs in Browser.
> Remote XUL
> This page uses an unsupported technology that is no longer available by default in Firefox.
>    Please contact the website owners to inform them of this problem
text/xml is correct Content-Type?

Comment 116

[Thomas D. (:thomas8)]

Comment on attachment 8672688 [details]
XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

(In reply to WADA from comment #115)
> (In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment
> #108)
> > XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect
> 
> Changed to text/plain, because following error occurs in Browser.
> > Remote XUL
> > This page uses an unsupported technology that is no longer available by default in Firefox.
> >    Please contact the website owners to inform them of this problem
> text/xml is correct Content-Type?

WADA, please don't make it hard to view/try my interactive XUL demo by changing the content type. I think you already heard about Google. Just googling the error message takes you straight to the simple workaround for re-enabling remote XUL in browser.

1) https://support.mozilla.org/de/questions/859304
2) https://developer.mozilla.org/en/using_remote_xul
3) https://addons.mozilla.org/en-US/firefox/addon/remote-xul-manager/

Only 3 is required to render demo.xul in FF.
2 is required if you want to view local xul in FF.
1 is the first google result which lists both 2 and 3.

After installing the addon, add this URL in the addon settings:
bmoattachments.org

Comment 117

[WADA:World Anti-bad-Duping Agency]

I merey tried to see XUL soruce quickly. I dislike "install addon merely to see something which is not nerded for me or which I'm not so interested in", "locally save something merely to see see something which is not nerded for me which I'm not so interested in". Sorry, I cancel your feedback request. ,

Comment 118

[WADA:World Anti-bad-Duping Agency]

Sorry type. nerded -> needed

Comment 119

[Thomas D. (:thomas8)]

WADA, for your interest.

Your coded references like "(C-1)" to cases of your very correct, but very hard-to-read plaintext table in user story makes your comments very hard, almost impossible to understand.
By the time people have found and understood what C-1 actually means in user story, they have forgotten the sentence of your comment. And when they go back to your comment, they have forgotten what C-1 means, because these cases are all very similar so it's very easy to mix them up.

I suggest that you make your cases explicit in comments like this:

(C-1 mixed+convertible::plain)

Also, HTML table might be much more readable for user story.

Comment 120

[Ben Bucksch (:BenB)]

WONTFIX again. Justification in comment 51.

I own this code (literally - I own the copyright), so I'm asking to please respect my decisions. Comment 53 is the exact opposite.

Comment 121

[Thomas D. (:thomas8)]

Comment on attachment 8672682 [details]
(v 4) Pseudo code for user's choice of Default for Unknown with new pref, and convert Unknown to HTML or Text to get out of mystery by Auto-downgrade

>http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#4962
>   4879 nsMsgCompose::DetermineHTMLAction(int32_t aConvertible, int32_t *result)
>
>http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#4914
>
>   No line is omitted from line 4914 to line 4987.
>   I can't write C++ code.
>   So, JavaScript statement, wrong syntax, is mixed.
>
>   4914   // If allHtml is true, then everyone has specifically requested to receive
>   4915   // HTML according to the address book.

I suggest more precise variable and pref names

> +        bool UnknownIsTextDomain;

bool UnknownPrefersPlaintext
(Domains are not relevant for the code logic here.)

> +             UnknownIsTextDomain = 
> +                Value of mailnews.compose.defaultForUnknownIsText;

mailnews.compose.sendformat.unkownPrefers.default
* please bundle all relevant prefs into .sendformat. category
* Imo, this should NOT be boolean, because "prefers-unknown" is the classic case for "send-both".
I'm not sure if WADA's thesis that MIME-capable readers will also be able to render HTML is sufficient to drop "both" as an option. I'd prefer to err on the safe side. Some people who must know recommend it, too:
https://litmus.com/blog/best-practices-for-plain-text-emails-a-look-at-why-theyre-important
Even if this would actually be a two-way pref, I would not use boolean pref because it's not future-proof if we might need any other values for that pref in the future.

> +        bool UnknownIsHTMLDomain;
> +             UnknownIsHTMLDomain = !UnknownIsTextDomain;

bool UnknownPrefersHTML

>   4957       case nsIAbPreferMailFormat::plaintext:
>   4958         allHtml = false;
>   4959         break;
>   4960 
>   4961       default: // nsIAbPreferMailFormat::unknown
> - 4962         allHtml  = false;
> - 4963         allPlain = false;
> +              // If still Unknown after Prefers and HTML/Text Domain check,
> +              // use defaultForUnknownIsText setting.
> +              if     (UnknownIsTextDomain) allHtml  = false;

if (UnknownPrefersPlaintext) allHtml  = false;

> +              else if(UnknownIsHTMLDomain) allPlain = false;

else if (UnknownPrefersHTML) allPlain = false;

>   4964         break;
>   4965       }
>   4966     }
>   4967   }
>   4968 
>   4969   // If everyone supports HTML, then return HTML.
>   4970   if (allHtml)
>   4971   {
>   4972     *result = nsIMsgCompSendFormat::HTML;
>   4973     return NS_OK;
>   4974   }
>   4975 
> - 4976   // If we can guarantee that converting to plaintext is not lossy, send the
> - 4977   // email as plaintext. Also send it 
> +        // If all recipients prefers plaintext,
> +        // or if Prefers=HTML/Text/Unknown are mixed and we can consider that

That's not correct description I think. Mixed is not required.
All-Unknown is not mixed.

> +        // formatting by HTML tags is almost not used,
> +        // convert HTML to text and send as text/plain.
>   4978   if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
>   4979   {
>   4980     *result = nsIMsgCompSendFormat::PlainText;

Comment 122

[WADA:World Anti-bad-Duping Agency]

(In reply to Ben Bucksch (:BenB) from comment #120)
> WONTFIX again.

Proposed patch was rejected, so WONTFIX. What's wrong in it? Thanks for closing again.

Comment 123

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #121)

Thanks for review. But this is a psuedo code for logic check by aceman. This is not patch proposal.
There is no need to refer to variable name etc. at this stage. All is temporary name.
If name change will be needed, "Change/ABC/DEF/all by Text Editor" is sufficient. 
What is needed is "comment about logic is correct or not" only.

> > +        // If all recipients prefers plaintext,
> > +        // or if Prefers=HTML/Text/Unknown are mixed and we can consider that
> That's not correct description I think. Mixed is not required. All-Unknown is not mixed.

If this is change before proposed patch is applied, your comment is correct. It should be: 
  if recipients type is not "all are Prefer/Domain=HTML" nor "all are Prefer/Domain=Text"
And, your "mixed" in "(C-1 mixed+convertible::plain)" is also incorrect, because "all Unknown" is not mixed.
(this is wrong, if the "mixed" is "mixed case" and definition of "mixed case" involves "all Unknown" case)

Because this is change by the patch, it's absolutely incorrect.
I was aware of this mistake.
  Why Unknown can exist at this step even though Unknown is removed by the patch himself... :-) 
Forgot to modify upon quick upload of V4.

Comment 124

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #105)
> Where is this Full Matrix? I see nothing by this name till now.

Sorry, I missed this question.

"Full Matrix" is name provide by Thomas D. for following:
  "option settings for all 6 cases" of "psedu Send options panel" in my "A Full Solution"
In his proposal or mock-up, "options for allHtml cases" looks already removed from the "Full Matrix".

Comment 125

[Thomas D. (:thomas8)]

(In reply to Ben Bucksch (:BenB) from comment #120)
> WONTFIX again. Justification in comment 51.
> 
> I own this code (literally - I own the copyright), so I'm asking to please
> respect my decisions. Comment 53 is the exact opposite.

Ben, WONTFIX wont fix the problems of what you claim to be "your" code.
Nobody cares if you have the copyright of this code or not (who ever heard of copyright in a Mozilla world...). We can change it all we want, and we will.

12.000 downloads of "Always HTML" addon can't be wrong in seeing that there's an UX problem.
It is sad that you seem to to be unaware of the difference between the purpose of a bug and the approach of a particular patch. Wrong patch approach does not mean wrong purpose of the bug. Patch was just one failed attempt to solve the problem with minimal effort.

It's also sad that as usual, you have nothing constructive to say about all the efforts which are going on here to fix the problems which you left in the code and UX design.

Repeating your old sermon of an age-old balance from the times of HTML vs. Plaintext wars (I can hardly remember, how long ago was that!?) does not make it any more true. There is no balance between plaintext and HTML as long as Thunderbird's default settings are unexpectedly downgrading users' HTML compositions into plaintext without any default way of preventing that. Lossless conversion my foot. You know very well that isConvertible algorithm is currently nowhere near lossless, nor is Auto-Detect. Auto-Downgrades eats entire stylings, and Auto-Detect dumps full-fledged message formatting including content like inline images without warning.

> * Send plaintext when we can, and send HTML when the user uses formatting that we need HTML for.

In case you haven't noticed, it's not about "sending plaintext when we can", but giving the user legitimate choices and control over the default format in which he wants to send. HTML is an accepted message format, so there's nothing wrong with sending that. Let users decide for themselves instead of forcing them into your favorite behaviour.

Comment 126

[Magnus Melin [:mkmelin]]

Comment on attachment 8672330 [details]
Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

I can't believe this bug has 125(!) lengthy comments. *Please* try to keep your comments concise and to the point. I believe the summary is correct, but nobody can read that amount of complex comments... 

As for the implementation, there's certainly no need for a per profile pref. Probably just check a (new) always-send-as-html pref here: http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#4970, and add an "Always HTML" checkbox in the send format options.

Comment 127

[Thomas D. (:thomas8)]

(In reply to Magnus Melin from comment #126)

> As for the implementation, there's certainly no need for a per profile pref.
> Probably just check a (new) always-send-as-html pref here:
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.
> cpp#4970, and add an "Always HTML" checkbox in the send format options.

4970   if (allHtml)
4971   {
4972     *result = nsIMsgCompSendFormat::HTML;
4973     return NS_OK;
4974   }

Comment 128

[:aceman]

(In reply to Magnus Melin from comment #126)
> I can't believe this bug has 125(!) lengthy comments. *Please* try to keep
> your comments concise and to the point. I believe the summary is correct,
> but nobody can read that amount of complex comments... 
Actually I started to understand it and had a plan :(

> As for the implementation, there's certainly no need for a per profile pref.
> Probably just check a (new) always-send-as-html pref here:
What is the difference between "per-profile pref" and a "pref"? Did you mean per-identity?

> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.
> cpp#4970, and add an "Always HTML" checkbox in the send format options.
OK, can you post this as feedback to my patch in bug 136502? And I'll rework it.

Comment 129

[Gervase Markham [:gerv]]

(In reply to Ben Bucksch (:BenB) from comment #120)
> I own this code (literally - I own the copyright), so I'm asking to please
> respect my decisions. Comment 53 is the exact opposite.

Without taking a position on the merits or otherwise of any particular position on the technical and UX questions covered by the bug, it is a matter of both legal fact (given that the code was freely licensed by the copyright holder under the MPL, just as all other code in Mozilla is) and Mozilla practice that copyright ownership of code does not give people more say over what happens to it in the future. Mozilla has a decision-making process in the case of conflict, and "who owns the copyright?" is not one of the considered factors when deciding who takes the final decision.

Ben: you will need to argue your case on its technical and UX merits alone.

Gerv

Comment 130

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #126)
> Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf
> I can't believe this bug has 125(!) lengthy comments. 8snip)

Magnus, read top part of "User Story" of this bug, please.

One is preparation for newly found solution(probably, one of best solutions) during analysis of two one-liner patch(and your opinion) which were proposed by this bug.
Another is follow-up session of this bug.

By the way, how can we keep readability of WONTFIX'ed bug which was already sufficiently hard-to-read, hard-to-understand, hard-to-follow bug, far before WONTFIX'ed? There is no big difference between "hard" and "unable"/"impossible" in such case, isn't it? :-)
"Re-opened only once", "fortunately, not re-opened twice", is very good, isn't it?

Comment 131

[WADA:World Anti-bad-Duping Agency]

(Off-Topic)

(In reply to Gervase Markham [:gerv] from comment #129)
> Ben: you will need to argue your case on its technical and UX merits alone.
> Gerv

Thanks for comment. I wasn't aware of that you watched this bug.

Even if it's absolutely correct legally, I believe that we should respect developer's opinion/want who actually implemented modules for Mozilla community.
And, "asking for his opinion, design objectives by him, for improvements in code of Mozilla" is a good practice and a good manner, even if he doesn't have copyeight of program code(not novel, not poem, not paint, not music) legally.
For me, Bug opener of this bug, "WONTFIX'ed legally" is not needed. "WONTFIX'ed by a developer who actually implemented modules" is sufficient. By it, I could know that proposed solution was bad.

I think that culpable party is peoples who frequently/repeatedly wrote comments as if "libel against a developer" in bug at bugzill.mozilla.org in many cases.
Please read comments in Bug 414299.
Please see rederred comment by comment #63. Why such comment on code(on comment in program source code!!!) can be produced? ;-)

Comment 132

[Magnus Melin [:mkmelin]]

Comment on attachment 8672682 [details]
(v 4) Pseudo code for user's choice of Default for Unknown with new pref, and convert Unknown to HTML or Text to get out of mystery by Auto-downgrade

I don't see what problem this really resolves. With the pref from 136502 you'd be able to determine stuff more consistently.

Comment 133

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #132)
> Comment on attachment 8672682 [details]
> (v 4) Pseudo code for user's choice of Default for Unknow with new pref of
> mailnews.compose.defaultForUnknownIsText=true(Text)/false(HTML)
> 
> I don't see what problem this really resolves.
> With the pref from 136502 you'd be able to determine stuff more consistently.

This change won't resolve any problem in Tb's code or doesn't touch design policy.
And change is absolutely independent from and is irrelevant to change in Bug 136502. 

This patch merely reduces frequency or probability of "falls into (C-1) if( (!allHTML&&!allPlain) && Convertible) send as text/plain", by removing Prefers=Unknown case.

Because Prefers=Unknown is interpreted as Prefers=HTML or Prefers=Text,
"All Prefers==Unknown case" is changed to allHTML case or allPlain case.
So, mixed case( (C-1) if(mixed&&Convertible) / (C-2) if(mixed&&Not-Convertible) ) occurs only when "actually HTML Domain recipients + actually Text Domain recipients" case.
I believe that majority of Tb users keeps Prefers=Unknown in Address Book.
Even if probability of Convertible is pretty high(even if 1), probability of "actually HTML Domain recipents + actually Text Domain recipients" is low, probability of that "(C-1) if(mixed&&Convertible) send as text/plain" is sufficiently low.
And, even if "actually HTML Domain recipents + actually Text Domain recipients" occurs, who sets "actually HTML Domain recipents" and "actually Text Domain recipients" is user. So, even when HTML mail was sent as text/plain, reason why send as text/plain is understandable for user.
(Note: Who set Prefers=Unknown is Thunderbird.)

i.e.
"Probability of that following phenomenon occurs in ordinal environment" is changed to low.
> Thunderbird produces following funny phenomenon == mystery for user.
>   Even though user actually composed mail in HTML mode,
>   the mail was sent as plain/text by Tb silently, with UNKNOWN reason.
> Current Action in (C-1) "if(!allHtml&&!allPlain&&Convertible::Plain)"
> is cause of user's confusion.

As I repeatedly said, this bug refers above phenomenon only, which generated request of Bug 136502(Opened on 2002-04-09), and which produced comments/many attachments in Bug 414299, which produced useless discussion like "lossy or not lossy", which produced repeated "critical or severe format/data loss".
So, "low probability of above phenomenon" is a solution of this bug.

Do you know existence of Bug 56834(Opened on 2000-10-16)?
Preference was borrowed from that bug.
If the patch for "interpret Prefers=Unknown one of Prefers=HTML and Prefers=Text" is effective, there is no need to set Prefers=HTML or Text upon Contact creation in Address Book.
So, the patch is friend of Bug　56834, although proposed one-liner patch in this bug was friend of Bug 136502.
Because this bug was WONTFIX'ed, Bug 136502 is not friend of this bug any more :-)
If the patch resolves something in Tb's code, it's problem of Bug 56834.

Please note that "if(allHtml||allPlain||(!allHtml&&!allPlain) UseSendOptions" + "enhanced Send Options panel" is a thought experiment(gedankenexperiment, or Gedankenexperiment).
A part of the "enhanced Send Options panel" can be used as UI of option setting for Bug 136502, if profile wide preference is used in that bug. It's usable as UI of "default for Prefers=Unknown".

Comment 134

[Thomas D. (:thomas8)]

(In reply to Gervase Markham [:gerv] from comment #129)
> (In reply to Ben Bucksch (:BenB) from comment #120)
> > I own this code (literally - I own the copyright), so I'm asking to please
> > respect my decisions. Comment 53 is the exact opposite.
> ...
> Ben: you will need to argue your case on its technical and UX merits alone.
> Gerv

Thanks a lot Gerv for that clarification. EXACTLY THAT.

<snip>

Comment 135

[Thomas D. (:thomas8)]

(In reply to Magnus Melin from comment #132)
> Comment on attachment 8672682 [details]
> (v 4) Pseudo code for user's choice of Default for Unknow with new pref of
> mailnews.compose.defaultForUnknownIsText=true(Text)/false(HTML)
> 
> I don't see what problem this really resolves. With the pref from 136502
> you'd be able to determine stuff more consistently.

Hi Magnus,

this pref makes a lot of sense if we dare to see things in the big picture. It's unfortunate WADA's and my comments are too extensive so you couldn't follow them (and even I have problems to catch up ;), because much has already been said.

Bug 136502 is currently twisted because summary says "option for switch off Auto-*Detect*" which means default to an actual message delivery format whenever composing (which is very good and needed message-centric feature), but what users want most is "option to switch off Auto-Downgrade", which is never same as Auto-Detect.

Current patch for Bug 136502 (attachment 8673195 [details] [diff] [review]) does a little bit of both as it switches off "convertible::plain" by setting "convertible::no" which will actually mostly force HTML (or html question for newsgroups) which is more than just disable Auto-Downgrade - but that's actually neither necessary nor desirable. Instead, we should really just bypass the "convertible::plain -> force plaintext" condition if the pref is true, without setting convertible::no. Then, Auto-Detect can still work correctly according to user-settings and TB-settings. Because it's NOT mainly about "Always send HTML", it's more about not silently bypassing all of Auto-Detect when there's no formatting. Practically, for many cases, that will indeed send HTML (or both), but more precisely according to Auto-Detect settings and algorithms.

Which is where the default pref for "prefers-unknown==plain | html | both" comes in handy.
If we allow user to disambiguate case of "prefers-unknown", and come with a reasonable default for that disambiguation ("html"), we can be more precise, transparent and predictable in Auto-Detect.
Specifically, setting "unknown==html" will effect that allHTML cases get sent as "HTML-only" (currently Both, which is unnecessary overhead in msg size). Setting "unknown==plain" can assist those who want to err in favor of plaintext always. Setting "unknown==both" is helpful if recipients preference is actually unknown, so you want to cater for both types of recipients, Plain AND HTML. Apparently, "send multipart/alternative = Both" is not outdated practice, but still recommended even by popular email marketing agency [1], who say that "both" will reach most recipients best.

Finally, by disambiguating "both", we can also reduce the one-size-catch-all scope of send options to "allPlain" (and perhaps add one separate send-options dropdown for "somePlain" scenarios) and make them more useful, because they are currently a conglomerate which can never really be used with all the available options because the current send options scope matches too broad:

Current scope of send options "recipients not listed as being able of receiving HTML":
 "all-prefers-plain", "some-prefers-plain", "prefers-unknown", and de-facto "prefers-html", too, if mixed with any of these.

[1] https://litmus.com/blog/best-practices-for-plain-text-emails-a-look-at-why-theyre-important

Comment 136

[Magnus Melin [:mkmelin]]

Ok I think I see what you're getting at. You want to ignore recipient prefs and basically look at the message only to determine format. 
I'd agree they outlived their usefulness, not likely used by anyone actively and could be ripped out IMO.

Comment 137

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #135)
> Bug 136502 is currently twisted because summary says "option for switch off
> Auto-*Detect*" which means default to an actual message delivery format
> whenever composing (which is very good and needed message-centric feature),
> but what users want most is "option to switch off Auto-Downgrade", which is
> never same as Auto-Detect.

The patch still allows downgrade but it now happens only for allPlain recipients. I think that was the requirement.
It basically fixes bug 1204379 in a different way.

> Current patch for Bug 136502 (attachment 8673195 [details] [diff] [review])
> does a little bit of both as it switches off "convertible::plain" by setting
> "convertible::no" which will actually mostly force HTML (or html question
> for newsgroups) which is more than just disable Auto-Downgrade - but that's
> actually neither necessary nor desirable. Instead, we should really just
> bypass the "convertible::plain -> force plaintext" condition if the pref is
> true, without setting convertible::no.

By setting convertible::no (via the pref) we bypass "convertible::plain -> force plaintext" as it now is not true.
What is bad in that?

> Then, Auto-Detect can still work
> correctly according to user-settings and TB-settings. Because it's NOT
> mainly about "Always send HTML", it's more about not silently bypassing all
> of Auto-Detect when there's no formatting. Practically, for many cases, that
> will indeed send HTML (or both), but more precisely according to Auto-Detect
> settings and algorithms.

Yes, the patch makes autodetect lean more to send HTML/both in more cases (only allPlain still auto-downgrade).
 
> Which is where the default pref for "prefers-unknown==plain | html | both"
> comes in handy.
> If we allow user to disambiguate case of "prefers-unknown", and come with a
> reasonable default for that disambiguation ("html"), we can be more precise,
> transparent and predictable in Auto-Detect.
> Specifically, setting "unknown==html" will effect that allHTML cases get
> sent as "HTML-only" (currently Both, which is unnecessary overhead in msg
> size).
Yes, this will cause most messages to be sent a HTML-only.

(In reply to Magnus Melin from comment #136)
> Ok I think I see what you're getting at. You want to ignore recipient prefs
> and basically look at the message only to determine format. 
> I'd agree they outlived their usefulness, not likely used by anyone actively
> and could be ripped out IMO.

I don't think so. Having the possibility to set a recipient to plain is still useful. Maybe most users will not use the feature ever but why removing it. I think Thomas only wants that for unknown (the majority) of recipients we by default assume they can read HTML, so we send both. Or HTML-only if set by the pref. I would like to implement what Thomas is proposing here. But I would still vote for having this pref default to Both.

Comment 138

[Magnus Melin [:mkmelin]]

(In reply to :aceman from comment #137)
> I don't think so. Having the possibility to set a recipient to plain is
> still useful. Maybe most users will not use the feature ever but why
> removing it. 

Useful for what? Do you have a use case for it? 
If it causes more work, confusion and unexpected behavior for 99+% of users...

Comment 139

[WADA:World Anti-bad-Duping Agency]

(In reply to Magnus Melin from comment #136)
> You want to ignore recipient prefs and basically look at the message only to determine format. 

It's incorrect. It's never "ignore". pseudo code is for "respect user's want always, instead of Tb's want or Tb developer's want".

"Prefers=Unknown of Contact in Address Book" is merely a value which was set by Tb as Tb's default upon Contact creation, because Tb cen not know Prefers=HTML or Prefers=Text.
I believe that "Prefers=Unknown of Contact in Address Book" is currently(I believe since initial of Prefers=Unknown) never user's intentional setting/choice or user's want of "Prefers=Unknown" for any Tb user.
If it's wrong, why Bug 56834 was opened in 2000?

Current logic for type check of a recipient.
  See Prefers of Address Book for a recipient. If HTML or Text, use it, end of check.
  Else if Unknown, see HTML/Text Domain setting. If HTML Domain or Text Domain, use it, end of check.
  If still Unknown, allHTML=false; allPlain=false => force mixed case, if Prefers=Unknown is detected.
  i.e. when a Prefers=Unknown is involved or all recipients is Prefers=Unknown, !allHtml&&!allPlain is set.
pseudo code merely tries to do next change.
  If still Unknown, allHTML=false; allPlain=false => force mixed case, if Prefers=Unknown is detected.
  =>
  If still Unknown, use HTML or Text value in a prefs setting which was set by user, not by Tb.

Why can this change be "ignore something"?
"ignore developer's want of forcing Unknown upon Contact creation, and forcing Unknown until user changes to HTML or Text in Address Book"?
If "a prefs setting which is set by user for Prefers=Unknown" will be introduced and if code will be changed, "user's intentional choice of Prefers=Unknown in Address Book" is *ignored*?

Or response to other comment which is posted before your comment #136?
(Quize: when will be comment number of 200 will be achieved.)

Comment 140

[[e-mail]]

(In reply to WADA from comment #139)
> Current logic for type check of a recipient.
1. If posting to newsgroup, ask user
2. If all recipients have the same preference, use that
3. If the message only contains basic text, use plain text
4. If the preference is not set to ask user, use that
5. Ask the user

As I commented in some other bug, I personally would be happy to switch cases 3. and 4.

Comment 141

[:aceman]

(In reply to [e-mail] from comment #140)
> (In reply to WADA from comment #139)
> > Current logic for type check of a recipient.
> 1. If posting to newsgroup, ask user
> 2. If all recipients have the same preference, use that
> 3. If the message only contains basic text, use plain text
> 4. If the preference is not set to ask user, use that
> 5. Ask the user
> 
> As I commented in some other bug, I personally would be happy to switch
> cases 3. and 4.

Now that looks interesting. So:
3. for mixed recipients, use Send Options unless they are "Ask me".
4. If the message only contains basic text, use plain text
5. only then ask

WADA, Thomas, would that solve any cases for you? Yes, that means this would send something from Send Options (e.g. both) to some recipient that is plain (when there is also some that is unknown or HTML), even when the msg is convertible.

Comment 142

[WADA:World Anti-bad-Duping Agency]

(In reply to [e-mail] from comment #140)
> (In reply to WADA from comment #139)
> > Current logic for type check of a recipient.

The "logic for type check" which I call is code part to set allHtml and allPlain.
Please see attached "pseudo code"(currently, attachment 8672682 [details])
Could you please review the psuedo code.

> 1. If posting to newsgroup, ask user
> 2. If all recipients have the same preference, use that
> 3. If the message only contains basic text, use plain text
> 4. If the preference is not set to ask user, use that
> 5. Ask the user

This is bottom part of code contained in the attachment.
(send mail, or use Send Options, according to value of allHtml, allPlain, aConvertible).
See also Table in "User Story" of this bug, please.

> As I commented in some other bug, I personally would be happy to switch cases 3. and 4.

3. 4. is currently equivallent to next. (see also Table in User Story of this bug, please)
  2. else if(allHtml) send text/html; else if (allPlin) send text/plain;
  3. else if( (!allHtml && !allPlain) && Convertible     ) send test/plain
  4. else if( (!allHtml && !allPlain) && not-Convertible ) use Send Options where Ask can be requested
What kind of change do you mean by "switch cases 3. and 4."?

Comment 143

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #141)
> that means this would send something from Send Options (e.g. both) to some recipient that is plain
> (when there is also some that is unknown or HTML), even when the msg is convertible.

"Automatically/silently downgrade to text/plain" is executed in next cases.
  (B-1) else if (allPlain && Convertible    )             send as text/plain
  (B-2) else if (allPlain && Not-Convertible)             send as text/plain
  (C-1) else if( (!allHtml && !allPlain) && Convertible ) send as text/plain

Answer from Ben Bucksch (:BenB) in comment #51.
> If somebody wants to add a pref to say "always send HTML+plain, even if the
> text is just plain text and we can down convert to plaintext", go ahead.
> That's the only possible concession still possible.
i.e.
- If user's preference is used, in both (B-1) and (C-1), other than "silently send as text/plain" can be executed.
- Setting at Send Opstions panel is a "user's preference", as far as user can set it freely.

Ben never rejects or prohibits or denies or ignores "send HTML mail as HTML mail even when Convertible".
He merely says that "skipping current logic, which was implemented for reasonable purpose, which was used by many Tb users for lo---ng time without critical/severe problems" should be done only by user's intentional request for it, and that default behaviour of Tb should be same as current one(or equivallent one, identical one).

Biggest problem in "using Send Options panel" is that current Send Options panel is designed/implemented for "(C-2) else if( (!allHtml && !allPlain) && Not-Convertible)" case only.
To use Send Options panel for other cases or other purposes, big design/implementation changes in "Send Options feature" is needed.
This is reason why I think WONTFIX of this bug(proposal of one-liner patch only) is pretty reasonable.
This is reason why I'm watching work by Thomas D. on Send Options panel in this bug.

By the way, aceman, I guess that Thomas D. wants prefs name of "Kill_feature_implemented_by_Ben" :-)

Comment 144

[:aceman]

(In reply to WADA from comment #143)
> (In reply to :aceman from comment #141)
> > that means this would send something from Send Options (e.g. both) to some recipient that is plain
> > (when there is also some that is unknown or HTML), even when the msg is convertible.
> 
> "Automatically/silently downgrade to text/plain" is executed in next cases.
>   (B-1) else if (allPlain && Convertible    )             send as text/plain
>   (B-2) else if (allPlain && Not-Convertible)             send as text/plain
>   (C-1) else if( (!allHtml && !allPlain) && Convertible ) send as text/plain
Yes.

> Answer from Ben Bucksch (:BenB) in comment #51.
> > If somebody wants to add a pref to say "always send HTML+plain, even if the
> > text is just plain text and we can down convert to plaintext", go ahead.
> > That's the only possible concession still possible.
> i.e.
> - If user's preference is used, in both (B-1) and (C-1), other than
> "silently send as text/plain" can be executed.
> - Setting at Send Opstions panel is a "user's preference", as far as user
> can set it freely.
So again, you want for the user to be able to choose something else than text/plain even in B-1?

> Ben never rejects or prohibits or denies or ignores "send HTML mail as HTML
> mail even when Convertible".
Then I think this would need a big redesign of Send options as it would allow to override most of the automatic cases TB currently decides itself. And I didn't see any hint from Ben/Magnus that a big redesign would be accepted. I do not think his "add a pref" also allows for "make the Send Options 3times bigger and make nsMsgCompose::DetermineHTMLAction 3times more complicated to allow the user to set any combination of formats" :)

> He merely says that "skipping current logic, which was implemented for
> reasonable purpose, which was used by many Tb users for lo---ng time without
> critical/severe problems" should be done only by user's intentional request
> for it, and that default behaviour of Tb should be same as current one(or
> equivallent one, identical one).
Yes, I understand that. I do not want to kill current behaviour by default. I just try to add some small tweaks/options that the user needs to set if he wants to override that behaviour. As you said, user's preferences.

> Biggest problem in "using Send Options panel" is that current Send Options
> panel is designed/implemented for "(C-2) else if( (!allHtml && !allPlain) &&
> Not-Convertible)" case only.
Yes, but it does not say so in the wording. It appears as if it applies to allPlain too.
So if we want to keep that, we need to fix the wording.

> To use Send Options panel for other cases or other purposes, big
> design/implementation changes in "Send Options feature" is needed.
> This is reason why I think WONTFIX of this bug(proposal of one-liner patch
> only) is pretty reasonable.
> This is reason why I'm watching work by Thomas D. on Send Options panel in
> this bug.
> 
> By the way, aceman, I guess that Thomas D. wants prefs name of
> "Kill_feature_implemented_by_Ben" :-)
Why not, if it makes sense? :) But is is called "override_Bens_feature_by_users_preference" :)

Comment 145

[Thomas D. (:thomas8)]

OK, the quality of discussion here is getting much better, I'm happy. I'm especially happy that we're discussing freely, no-holds-barred.

E.g. as a side note, indeed, as Magnus says, and as I hinted before, it's also possible that we *could* decide the following:
* rip out the entire complex mechanism of recipient-centric *Auto-Detect* (only useful for prefers-plain/force-plain, extreme minority usecase); and no Bens in the world could stop us from doing that
* keep message-centric *Auto-Downgrade* as an optional feature (send plain when msg looks plain).
Yet I'm not advocating for that (ripping out auto-detect), because
- the idea is still new to me
- I have no clear idea about the actual real-world usecases for prefers-plain
- Perhaps, as hinted in the article [1] (I think), there might be a renaissance of plaintext messaging for very small devices like smart watches etc.

[1] https://litmus.com/blog/best-practices-for-plain-text-emails-a-look-at-why-theyre-important

Comment 146

[:aceman]

(In reply to [e-mail] from comment #140)
> (In reply to WADA from comment #139)
> > Current logic for type check of a recipient.
> 1. If posting to newsgroup, ask user
> 2. If all recipients have the same preference, use that
> 3. If the message only contains basic text, use plain text
> 4. If the preference is not set to ask user, use that
> 5. Ask the user
> 
> As I commented in some other bug, I personally would be happy to switch
> cases 3. and 4.

You said this in bug 1202276 comment 14. So let's wait for some more reactions to this.

Comment 147

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #144)
> So again, you want for the user to be able to choose something else than text/plain even in B-1?

Yes, of course.
Because I think that Email is for recipients, is not for satisfaction of mail sender, I think "(B-2) if(allPlan&&Not-Convertible) silently send as text/html always" is pretty kind for recipients who is set in Text Domain of mail sender's settings by the mail sender. 
However, I never opposite to "user can set any format in Send Options or other settings or options".
Please don't forget that who said next is me :-)
  Even when Tb says Not-Convertible, I have right to set text/plain in Send Options.
  Even when Tb says Convertible, I have right to set text/html in Send Options,
  if I know that any Prefers=Text/Text Domain is "smart phone which can show HTML", even if owner prefers Text.

> I do not think his "add a pref" also allows for "make the Send Options 3times bigger ... (snip)

He is not owner of Send Options panel. You can freely try to improve(or making too busy) the panel, even if he dislike or hates your design :-)

> and make nsMsgCompose::DetermineHTMLAction 3times more complicated ... (snip)

If such code is moved to other module owned by other one, no problem.
Please don't forget that answer was never denial or prohibiting.
> That's the only possible concession still possible.

> > Biggest problem in "using Send Options panel" is that current Send Options
> > panel is designed/implemented for "(C-2) else if( (!allHtml && !allPlain) &&
> > Not-Convertible)" case only.
> Yes, but it does not say so in the wording. It appears as if it applies to allPlain too.
> So if we want to keep that, we need to fix the wording.

If so, required change can be minimized.
    minimum change in penel
  + minimum change like one-liner change in back-end code
If so, we can have 3 solutions for ambivalent requests from different kinds of user.
- Short term solution: minimum one
- mid   term solution: adding prefs setting support etc.
- long  term solution: one like mock-up by Tohmas D.

(Off-Topic)

However, I don't believe that big change is mandatory in send format choice, because;

User could/can choose any format via. Delivery Format menu for pretty long time.
And, if user want text/plain for Text Domain recipient who can not show HTML or who rejects HTML, user can already use Text Mode Editor by Shift Modifier at any of Button and menu and context menu.
Further, dropdown menu for HTML mode/Text mode in Write mrnubutton will be implemented in pretty near future.  And, I believe that numbwr of recipients who requires Prefers=Text/Text Domain is small nowadays in ordinal environment of majority of Tb users.

Please avoid overkill in your work in this area where pretty simple/easy worrkaround, such as msgcompose.background_color=#FEFEFE, msgcompose.text_color;#FFFFFF, are already found and are always available.
Please note that majority of user who experienced phenomenon of this bug" won't continue posting comment of complsaint once user knows such simple/easy workaround in his bug report at bugzilla.mozilla.org.
Please don't forget that Bug 414299 is for "<tt> and <a> with link text==link url", and that CSS Style case occurs only when "simple HTML including tag in Bug 414299" && "CSS Style is used".
So, "CSS Style check by you" will be implemented by you, "probability of that phenomenon of this bug occurs" will be drastically changed to low in ordinal environment.
- If tag like <b>, color, is used, Not-Convertible is returned, so problem won't occur even when CSS is used.
- If CSS Style attribute is used, Not-Convertible is returned by your patch, so problem won't occur.
- To use CSS attribute in Tb's composer, provided way in Tb is currently "Insert HTML" only.
  So, CSS Style case can occur only in Edit New/Reply/Forward of HTML mail generated by other mailer.
- HTML generated by MS is never simple HTML in almost all cases.
  "deep nest of table, with many tr/td, with CSS, with text in a few td only"
  is pretty normal/ordinal/usual HTML mail if generated by MS.
  So, "simple HTML" is pretty edge casee, pretty rare case, if HTML is generated by MS :-)
Please note that relevant conditions is "A_Func(allHtml,allplain) && aConvertible", and, "allHtml, allplain" and "aConvertible" are absolutely independent entity. So, even if "probability of one entity" is 1, if "probability of another entity" is changed to sufficiently low, probability of a "A_Func(allHtml,allplain)&&aConvertible" is changed to sufficiently low.

Again, please avoid overkill in your work in this area.

Comment 148

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #145)
> OK, the quality of discussion here is getting much better, I'm happy. I'm
> especially happy that we're discussing freely, no-holds-barred.
> 
> E.g. as a side note, indeed, as Magnus says, and as I hinted before, it's
> also possible that we *could* decide the following:
> * rip out the entire complex mechanism of recipient-centric *Auto-Detect*
> (only useful for prefers-plain/force-plain, extreme minority usecase); and
> no Bens in the world could stop us from doing that
> * keep message-centric *Auto-Downgrade* as an optional feature (send plain
> when msg looks plain).
Yes, please everybody note the difference between auto-detect and auto-downgrade. Those are 2 processes and no patch of me merges them or removes one of them. There are only some options to disable/override one or both of them by user decision.

> Yet I'm not advocating for that (ripping out auto-detect), because
"prefers" field from AB is one of the inputs into the auto-downgrade algorithm. If that is removed (proposal from Magnus), you actually only leave auto-detect as the sole input. So anything detected to be plain (no useful formatting) will be sent as plain to all recipients.

So please do not talk about removing "auto-detect".

Aaaaah, I now see you use "auto-detect" and "auto-downgrade" in opposite sense than I would use them :)

Comment 149

[:aceman]

(In reply to WADA from comment #147)
> (In reply to :aceman from comment #144)
> > So again, you want for the user to be able to choose something else than text/plain even in B-1?
> 
> Yes, of course.
> Because I think that Email is for recipients, is not for satisfaction of
> mail sender, I think "(B-2) if(allPlan&&Not-Convertible) silently send as
> text/html always" is pretty kind for recipients who is set in Text Domain of
> mail sender's settings by the mail sender. 
I do not understand this. If you have allPlain recipients and you send them HTML, how is that kind for the recipient? You have now forced the idea of the sender onto the recipient. That seems to me like opposite to what you say.

> However, I never opposite to "user can set any format in Send Options or
> other settings or options".
> Please don't forget that who said next is me :-)
>   Even when Tb says Not-Convertible, I have right to set text/plain in Send
> Options.
Yes.

>   Even when Tb says Convertible, I have right to set text/html in Send
> Options,
Currently you do NOT have this option, for unknown and plain recipients. (only allHhml recipients get HTML only).

>   if I know that any Prefers=Text/Text Domain is "smart phone which can show
> HTML", even if owner prefers Text.
If you know that, you can set that recipient to HTML in your AB. Why setting it to plain and then expect/force TB to send him HTML?

> > I do not think his "add a pref" also allows for "make the Send Options 3times bigger ... (snip)
> 
> He is not owner of Send Options panel. You can freely try to improve(or
> making too busy) the panel, even if he dislike or hates your design :-)
> 
> > and make nsMsgCompose::DetermineHTMLAction 3times more complicated ... (snip)
> 
> If such code is moved to other module owned by other one, no problem.
> Please don't forget that answer was never denial or prohibiting.
We can't just put any code in any arbitrary place :) There must be some schema to code placement and the reviewer will not allow any mess :)

> > That's the only possible concession still possible.
> 
> > > Biggest problem in "using Send Options panel" is that current Send Options
> > > panel is designed/implemented for "(C-2) else if( (!allHtml && !allPlain) &&
> > > Not-Convertible)" case only.
> > Yes, but it does not say so in the wording. It appears as if it applies to allPlain too.
> > So if we want to keep that, we need to fix the wording.
> 
> If so, required change can be minimized.
>     minimum change in penel
>   + minimum change like one-liner change in back-end code

Yes, or my proposal was to change the meaning of those Send options to apply always if there is at least one Plain recipient. If we have only a mix of unknown and HTML recipients (a subset of (!allHtml && !allPlain)), we could automatically send them Both and be happy.

> User could/can choose any format via. Delivery Format menu for pretty long
> time.
> And, if user want text/plain for Text Domain recipient who can not show HTML
> or who rejects HTML, user can already use Text Mode Editor by Shift Modifier
> at any of Button and menu and context menu.
I don't thinks this is so easy. The user would have to think BEFORE even starting composing that maybe some of his recipients (that he later puts in) will be plain text. I think this is very unreal expectation.

> Please avoid overkill in your work in this area where pretty simple/easy
> worrkaround, such as msgcompose.background_color=#FEFEFE,
> msgcompose.text_color;#FFFFFF, are already found and are always available.
This was a hack :) So in bug 136502 I make this option official.

> Please note that majority of user who experienced phenomenon of this bug"
> won't continue posting comment of complsaint once user knows such
> simple/easy workaround in his bug report at bugzilla.mozilla.org.
> Please don't forget that Bug 414299 is for "<tt> and <a> with link
> text==link url", and that CSS Style case occurs only when "simple HTML
> including tag in Bug 414299"
Yes, I'll finish that one, once the style patch is done.

> && "CSS Style is used".
> So, "CSS Style check by you" will be implemented by you, "probability of
> that phenomenon of this bug occurs" will be drastically changed to low in
> ordinal environment.
Yes.

> - If tag like <b>, color, is used, Not-Convertible is returned, so problem
> won't occur even when CSS is used.
> - If CSS Style attribute is used, Not-Convertible is returned by your patch,
> so problem won't occur.
> - To use CSS attribute in Tb's composer, provided way in Tb is currently
> "Insert HTML" only.
We also have addons, that insert even <style> into the <head> of the composed document.
We have to not assume anything about how simple the document should be just because TB's default composer does not allow inserting some code.

>   So, CSS Style case can occur only in Edit New/Reply/Forward of HTML mail
> generated by other mailer.
> - HTML generated by MS is never simple HTML in almost all cases.
>   "deep nest of table, with many tr/td, with CSS, with text in a few td only"
>   is pretty normal/ordinal/usual HTML mail if generated by MS.
>   So, "simple HTML" is pretty edge casee, pretty rare case, if HTML is
> generated by MS :-)
> Please note that relevant conditions is "A_Func(allHtml,allplain) &&
> aConvertible", and, "allHtml, allplain" and "aConvertible" are absolutely
> independent entity. So, even if "probability of one entity" is 1, if
> "probability of another entity" is changed to sufficiently low, probability
> of a "A_Func(allHtml,allplain)&&aConvertible" is changed to sufficiently low.

Yes. The option in bug 136502 is basically for those "low probability" cases. For people who replying to a plain text message via the HTML composer and consider the reply header to be useful formatting that must be kept. Or that their plain text message must absolutely be displayed in proportional font so it must be sent as HTML otherwise all their work is busted.

And of course for cases when the convertibility detection still misses some real cases of formatting we didn't thought of:)

> Again, please avoid overkill in your work in this area.
That is my main goal in all this:) I want to do small changes that allow to mostly fix this instead of totalmegasolution that makes Send options 3times bigger. I am sure the potential reviewers are completely fed up with all our talk and any slightly big solution will be rejected:)

Comment 150

[WADA:World Anti-bad-Duping Agency]

In reply to :aceman from comment #149)

aceman, I'm never opposit to "no user's option for allHtml" or "option for if(allPlain&&Not-Convertible)text/plain due to data loss".
I merely wanted to say;
- user has right to send mail, which user composed in HTML mode using the mailer, in any format.
- mailer doesn't have right to send mail, silently, with unknown reason for user, without user's request,
  in format which user never wants.
- There are peoples who have different opinion fron yours.

Comment 151

[WADA:World Anti-bad-Duping Agency]

(In addition to comment #150)

- Developer of a open source software has right to provide program code based on his philosophy, belief.

Comment 152

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #149)
> (In reply to WADA from comment #147)
> > (In reply to :aceman from comment #144)
> > > So again, you want for the user to be able to choose something else than text/plain even in B-1?
> > 
> > Yes, of course.
> > Because I think that Email is for recipients, is not for satisfaction of
> > mail sender, I think "(B-2) if(allPlan&&Not-Convertible) silently send as
> > text/html always" is pretty kind for recipients who is set in Text Domain of
> > mail sender's settings by the mail sender. 
> I do not understand this. If you have allPlain recipients and you send them
> HTML, how is that kind for the recipient? You have now forced the idea of
> the sender onto the recipient. That seems to me like opposite to what you say.

Even though I don't hate Auto-Downgrade, when I wanted to get HTML mail source by Send Later, if I forget to use <B>, background color, text color, etc., and if I forgot to set Delivery Format=HTML, Tb creates text/plain mail. So, I have to create HTML again. In such case, I want to kill Auto-Downgrade, and I want to send mail as text/html if Tb says Convertible and send mail as text/plain if Tb says Not-Convertible.
That means that I want "Damm, stupid Auto-downgrade' button" in such case :-)

Please don't post stiff and prope response to my lack in seriousness comment.

Comment 153

[WADA:World Anti-bad-Duping Agency]

Off-Topic.

(In reply to :aceman from comment #144)
> > "Automatically/silently downgrade to text/plain" is executed in next cases.
> >   (B-1) else if (allPlain && Convertible    )             send as text/plain
> >   (B-2) else if (allPlain && Not-Convertible)             send as text/plain
> >   (C-1) else if( (!allHtml && !allPlain) && Convertible ) send as text/plain
> Yes.

Thanks for your coming back to this unable-to-read/understand/follow bug.
Please note that 80% of "lo--ng comment in this bug" is not by me.

Although "Auto-downgrade" is not official term, I tried to separate "Auto-Detect feature" and "an action of Auto-downgrade in Auto-Detect feature".
However, I didn't know about correct meaning of "an action of Auto-downgrade in Auto-Detect feature" which I called. It was "when Tb want to send HTML as text/plain, Tb sends HTML mail as text/plain".
For me, it was sufficient, because "problem for me" was "not so small number of Tb users are confused by it, and some of them opened bug at bugzilla.mozilla.org due to the confusion". I was not interested in "such phenomenon occurs in what conditions".
  
But, by your one-liner patch which you attached to Bug 12043798(that was also morphed to unable-to-read/understand/follow bug ...), I could read logic for "send mail". And I could know about "where/how 'automatically/silently downgrade to text/plain' is executed by Tb".
Table in "User Story" of Bug 12043798 and this bug(Bug 1210244) is my homework　report in your "lecture by one-liner patch" :-)

After your lecture, I was surprized.
  Even though actual problem is in following case only,
  // two cases in Auto-Downgrade 
    (B-2) else if (allPlain && Not-Convertible)
               send as text/plain  <= data loss you found and indicated by patch
    (C-1) else if( (!allHtml && !allPlain) && Convertible )
               send as text/plain  <= mystery for user, cause of user's confusion,
                                      start point of allmost all Auto-Downgrade relevant bugs
  request was went to "other default than Auto-Detect in Delivery Format menu",
                      "prefs setting to 'don't use Auto-Detect' when I composed HTML mail" and so on,
  or went to "change Convertible check logic for me" only.
  (Bug 414299. I separaed CSS case to Bug 584313, based on Ben's want, will, decision, ...)

This was biggest reason why I opened this bug.

Comment 154

[Thomas D. (:thomas8)]

Attachment: Screenshot S-T2 of XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

I hope that everyone will try the actual interactive XUL demo X-T2 (attachment 8672688 [details]), because only in the demo can you see and play with the settings dropdowns. But for ease of reference, here's a screenshot of that proposal.

Comment 155

[Thomas D. (:thomas8)]

Hmmm. Viewing that screenshot attachment 8673807 [details] via BMO expands the image to the maximum available viewing port size of FF, so please be aware that this looks much bigger on your screen than in real life. Better try the XUL demo of attachment 8672688 [details] instead, so that you're not prejudiced about the size.
[off-topic] Any idea why the image auto-expands or how to prevent it?

Comment 156

[Thomas D. (:thomas8)]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #145)
> OK, the quality of discussion here is getting much better, I'm happy. I'm
> especially happy that we're discussing freely, no-holds-barred.

**************************************************************************
Typology of delivery format UX failure and how to cure it:
message-centric vs. recipient-centric
**************************************************************************

I would like to share some insight which recently revealed itself to me more clearly.
I think this typological analysis of the current design can help a lot to understand why the current behaviour is so confusing and failing to serve the needs of various users with a transparent, predictable, and efficient UX.
This is the philosophy behind my simple, full proposal of attachment 8672688 [details], which is a xul demo of an enhanced  version of attachment 8672330 [details] which comes with more technical description of benefits, implementation details, auto-detect delivery format result tables etc.

There are two main ways of deciding the send format in which HTML messages will ultimately be sent:
A) recipient-centric: (looking at recipients' preferences; especially if they prefer "plain", this approach will compromising the message formatting, iow this goes against message-centric approach.)
B) message-centric (looking at a particular message body; manually or automatically choose an appropriate send format; this tends to compromise recipients preferences, because preserving the HTML message formatting is incompatible with sending plaintext to recipients who prefer that; iow, this goes against recipient-centric approach.

* Auto-Detect is recipient-centric. Look at recipients' preferences and try to choose the best message format based on that.
* Send Options are exclusively for finetuning Auto-Detect (although we never reflect that in the UI, another shortcoming which my proposal addresses), so they are also recipient-centric.
* Auto-Downgrade is (mostly) message-centric. Look at the message body and send plaintext if it looks like plaintext; ignore recipients preferences for most cases, including "mixed" of somePlain and someHTML, or "prefers-Both" via recipient-centric send options. Confusingly, we suddenly observe recipient preferences for just one exception, when all recipients prefer HTML aka allHTML.
* Delivery formats from menu are message-centric (Auto-Detect | html | plain | both). Choose whichever send format you like best; or let Auto-Detect decide the best format for your particular message (based on recipient-centric algorithm).

*** Problems of current TB send format design: ***
Neither a) recipient-centric nor b) message-centric user choices are fully developed or controllable, because they are intertwined or not offered.

**** Magnus is a message-centric user.***
Imagine a user like Magnus who does not know any users who prefer-plaintext.
He's also a pragmatic user who can agree that "if message looks like plaintext, let me just send plaintext". Or he might be a marketing user who has read article of [1] and would like to send "both" for every message. Or he might have a girlfriend with a smart watch and might want to send "plain" for every message, and has not come across our plaintext editor because we don't default to that so it's not easy to get into.

--> We currently do NOT allow message-centric users to pick a default send format which they think is best for starting their messages (html | plain | both). Instead, we are forcing everyone to use recipient-centric Auto-Detect, even though (as Magnus points out), they might not even know about the recipient format preferences in the AB, and thus never use them.
--> To allow message-centric use cases, we need to implement at least a global pref for default send format which user can choose freely. Imo, this pref should be expanded to *allow* per-identity default send format. It's very possible that for my private communications account, I'll happily want "html-only" default send format, but for my business communicatinos account, I want default of "both". Per-identity works so that by default, it will use the global setting (and show that in the dropdown) and not create any actual pref entries for the identity until needed. Structurally much like "unknown" for recipients, where I'm also suggesting to use global pref if per-recipient pref is not set.

*** Uncle Sam is a recipient-centric user.***
He has carefully set per-recipient "prefers-plain" for his old friends who need or want that.
But most of his friends are "prefers-unknown" because defining prefs for each recipient is boring and not really needed as most will be just happy with whatever I send them. He wants to ensure that his "prefers-plain" friends get plaintext as often as possible; but he also wants to preserve his HTML-formatted messages for all other friends without dataloss of formatting so that they get nice HTML messages.
--> We currently do NOT allow recipient-centric users to downgrade to plaintext as often as possible for their "prefers-plain" friends AND at the same time send HTML as often as possible for the rest of the world. Because we have wrongly combined the send options for "prefers-plain" and "prefers-unknown", AND because we don't allow user to disambiguate of "prefers-unknown" should mean "prefers-plain", "prefers-html", or "prefers-both" (a safe and needed option, currently available as default but only in very crooked implicit ways). If you set send options to "force-downgrade" to optimize for "prefers-plain", you'll also get the same for "unknown" users. If you set "both" to optimize for "unknown-prefers-html", you'll also get the same for "plain" users in mixed cases. In both cases, it's probably not what you want.
--> When *all* recipients of a message are "prefers-plain" (allPlain), setting send options to radical downgrading (after asking) is a legitimate usecase which effectively isn't well covered now. But Auto-Detect is all about "prefers-plain", so it's not too much to add one separate send option for "allPlain" scope.
--> For mixed cases involving at least one prefers-plain, we have send options. Let's keep them, and radically simplify the scope description: "If some, but not all recipients prefer to receive message formatted as plain text". That's a tricky case because following the recipient-centric dogma (send plain to prefers-plain) will violate the message-centric dogma (preserve HTML formatting). The user must decide how to handle that case, via asking, downgrading(and ask if not possible), or just sending some format of choice.
--> It's really ironic that while we default to recipient-centric "Auto-Detect" of recipients' preferences, we offer no way for user to specify the actual format preference of the largest group in most ABs, "prefers-unknown" (which is essentially "undefined"). To complete the feature of recipient-centric auto-detection, disambiguation of "prefers-unkown" via user pref and UI of "prefers-unknown== plain | html | both" is needed. A simple dropdown box in send options does the trick.

*** Auto-Downgrade vs. Auto-Detect == message-centric vs. recipient-centric ***

Now over to my "best friend", Auto-Downgrade. More specifically, we have "Forced Auto-Downgrade to plaintext if message body is considered isconvertible::plain, AND if recpients are allPlain, mixed plain & html & unknown, but NOT allHTML. Right. Let's talk about it.
*Message-centric* Auto-*Downgrade* is currently implemented undercover without any trace, indicator, or control in the UI. But de facto, it's bundled with *recipient-centric* Auto-*Detect*, but will actually BYPASS recipient auto-detect for most cases. You get it. Two features pulling in opposite directions *invisibly* bundled together. That's what I call twisted UX. No wonder users get confused.
1) Confusion 1: Iow, an invisible, uncontrollable, message-centric feature (auto-downgrade) will for some cases disable the recipient-centric parent feature (auto-detect with Send Options). Parent feature (auto-detect) is only active option seen from UI, but suddenly in same cases not applied for reasons unknown to user. That's what WADA euphemistially calls an "element of surprise" for user; rumor has it that it's driving some affected users MAD how TB unilaterally decides above their heads to send HTML messages in plaintext. More so because the auto-downgrade algorithm does not honor its promise of lossless conversion and instead has a significant formatting dataloss potential e.g. when styles happen to be on the wrong tags which auto-downgrade wants to eliminate.
2) Confusion 2: While Auto-Downgrade, by virtue of being message-centric, happily (and correctly per feature definition) ignores any recipient preferences including "prefers-html" (when mixed with "unknown" or "prefers-plain") by just sending plain text anyway, we randomly deviate from that behaviour for the allHTML case where *all* recipients prefer HTML. This may seem plausible at first sight, but it is not. Why "someHTML" case is different from "allHTML" case? If we actually bothered to give users a choice, why would a user who has opted in to downgrade for "someHTML" case because there is "no formatting" in the message, not want the same for "allHTML" case for the same type of message?
Message-centric Auto-Downgrade supersedes recipient-centric Auto-Detect; that's clear and understandable behaviour if correctly exposed in the UI, so we should make things even more confusing by having unnecessary exception of allHTML case.

For completeness' sake:
- Auto-Detect itself is correctly placed in the list of message-centric send format options because it's an automatical way of determining the per-message send format.).
- Auto-Downgrade cannot be applied on top of any other explicit Delivery Format out of "plain | html | both", because it would either be effectless (for "plain"), or disable (lead ad absurdum) explicit delivery formats involving HTML by sending plaintext instead.
- Because of the very limited message formatting scope (convertible::plain), Auto-Downgrade always needs Auto-Detect as a fallback for determining another send format for all other formatting scopes (convertible::Yes|altering|no). But Auto-Detect does not always need Auto-Downgrade. So the options are:
Auto-Detect + auto-downgrade==false
Auto-Detect + auto-downgrade==true
This can be rewritten for UI-purposes using checkbox which is only enabled when Auto-Detect is active:
Options > Delivery Format
( o ) Auto-Detect
[ X ] Plain text when no formatting [Auto-Downgrade]
---------------------------- separator should be added in the menu
(   ) Plain text
(   ) HTML
(   ) Both Plain and Rich (HTML) Text
IMO, looking at the long history of user confusion and bugs about Auto-Downgrade, it would be worth to expose and indicate this in the primary per-message UI (delivery format menu) AND in the global options UI (send options), see Aceman's work in bug 136502.
Global options UI sets the default for Auto-Downgrade==true|false for every composition; per-message UI allows user to opt in or opt out for the current message. Exposing this in the delivery format menu is an excellent way of alerting users to this feature, so that the mystery/surprise/anger associated to the current intransparent and unpredictalbe behaviour disappears. It's actually more about exposing and indicating feature status than allowing per-message toggle (which is better done by chosing an explicit delivery-format).

Main courses are done, now for typological desert:

Massive silent dataloss of formatting occurs when convertible::NO and delivery format: plain text is chosen from menu. Which typological case?
Delivery formats are primarily message-centric.
But message-centric (per-message) delivery formats also have inherent assumption about recipients:
- delivery format "html" for inconvertible messages implies assuming recipients are capable of html.
- delivery format "plain" for inconvertible messages implies assuming recipients prefer/need plain
So, at least for messages with significant formatting, there's no significant inherent difference between following cases:
- recipients prefer plain (allPlain or somePlain scope)
- sender selects delivery format plain (most likely allPlain or somePlain scope)
Shocker surprise: Although essentially same case, send options protections by asking are only applied for explicit recipient-centric case, but never for implicit recipient-centric case.
Imo (and I understand Magnus agrees), whenever sending plaintext we should always ask instead of silent massive dataloss at least for cases of "convertible::no", regardless how plaintext format decision was arrived at (by auto-detect, or by user).

(I assume it's generally not message sender who wants downgrading just for the sake of it, because it's nonsense to compose significant HTML and then degrade it to plaintext before sending. And for deliberate conversions, there are better ways than blind conversion after sending).
*However*, only cases of delivery-format *plaintext* can involve dataloss of formatting (hence we must provide send options for them, so user decides how to handle the dataloss).

Congrats! If you've read all the way up to here, you're a brave, mad, or strong soldier of TB :)

[1] https://litmus.com/blog/best-practices-for-plain-text-emails-a-look-at-why-theyre-important

Comment 157

[:aceman]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #154)
> Created attachment 8673807 [details]
> Screenshot S-T2 of XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

Now this is what I called Send options 3 times larger than currently ;)

Comment 158

[:aceman]

(In reply to WADA from comment #150)
> - There are peoples who have different opinion fron yours.
Sure. I just want to somehow make sense from all the other opinions so that I can understand what to implement. My opinion is not important here, I mostly compose plain text ;)

Comment 159

[WADA:World Anti-bad-Duping Agency]

Attachment: kji

Comment 160

[WADA:World Anti-bad-Duping Agency]

To aceman, do you think that big change around HTML mail composion/sending is needed for (B-2) in Table of above attachment, even adter following?
(a) CSS Style Attribute support in Convertible check logic.
(b) Prefers=Unknow is interpreted as HTML or Text.
For resolving (B-2) case, I think following is sufficient.
    your one-liner patch
  + new prefs for "send text/plain or UseSendOptions in (B-2) case and (C-1) case"
  + new prefs support code
    (read prefs, if( ... && pref=true) UseSendOptions; else if( ... && pref=false) send text.plain;) 
  + wording change only in Send Options panel
  + wording change in panel by UX code according to prefs=true/false
I think same Send Options panel setting can be applied to all cases which uses Send Options, if new prefs is introduced and above change of (a)/(b) will be made.

Comment 161

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #154)
> Created attachment 8673807 [details]
> Screenshot S-T2 of XUL Demo X-T2: Full Proposal T2 - Rethink Auto-Detect

I think it's good mock-up for posible solutions with improvement or enhancement of "Send Options" panel.
1. profile wide prefs, which is currently considered in possible solutions, are considered in panel.
   - enhancement by aceman, which is currently propsed in Bug 136502(I think wrong place.) 
   - enhancement for "interpret Prefers=Unknown as HTML or Text"
2. utilizing Send Options pnanel in both of following case is considered.
   - (B-2) if(allPlain)&&Not-Convertible) send text/plain
      => data loss for sender
   - (C-1) if((!allHtmln&&!allPlain)&&Convertible) send text/plain
      => mystery for users, cause of requests in bugs, cause of comments of complaint

Comment 162

[WADA:World Anti-bad-Duping Agency]

(In addition to comment #161)
As aceman says, panel of the mock-up is too big or too heavy.
Panel size of my Notebooc PC is too small for it, and smart phone users can't view entire panel so easily :-)

Comment 163

[WADA:World Anti-bad-Duping Agency]

Attachment: hgf

Soryy for quick replace. typo, wrong statements, ... were corrected.
Key point is:
  To understand that Prefers=Unknown is a main culprit,
  "(C-x) if(!allHtml&&!allPlain) && ...)" was split to
      "replaced (C-x) if( (!allHtml&&!allPlain) && Unknown_is_involved     && ... )" 
  and "new      (D-x) if( (!allHtml&&!allPlain) && Unknown_is_not_involved && ... )"

Comment 164

[Thomas D. (:thomas8)]

(In reply to WADA from comment #162)
> (In addition to comment #161)
> As aceman says, panel of the mock-up is too big or too heavy.
> Panel size of my Notebooc PC is too small for it, and smart phone users
> can't view entire panel so easily :-)

TB is designed for desktop PCs / Notebooks, not for smart phones, so that's irrelevant.
Yes, the proposal increases the number of total lines in send options from 5 to about 15 (including headlines). That's pretty little if you consider that available options went up from 1 to 5.
So I've added 4 options with 2 lines each, which fixes a massive amount of several different design problems. I've also added section headlines, and descriptions, which provide necessary context currently missing.
Imho, panel size is a complete non-argument. Imo the only relevant question is:
Which options are necessary to provide sufficient ux-control, ux-error-prevention, ux-efficiency, ux-natural-hierarchy etc. for user?

Some ways of resolving "big options panel"
- vertical scroll bar (it's not like users are changing this every day...)
- split options, e.g. Default Message Delivery Format could be at the bottom of Options > Composition > General Tab, and change the button to "Settings for Auto-Detect..." (save 4 lines)
- Within Options > Composition, have a dedicated "Send format" tab
- Move HTML and Text domains into their own dialogue, because nobody uses them anyway
- recombine the two recipient scope options allPlain and somePlain, but I've explained why I think these should be separate if we really want to be useful for plaintext scenarios. So we could have:
"If any recipient prefers to receive messages formatted as plain text:"

Comment 165

[Thomas D. (:thomas8)]

Comment on attachment 8672330 [details]
Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf

(Clearing feedback request, In reply to :aceman from comment #87)
> Comment on attachment 8672330 [details]
> Proposal T1: Another, simpler full solution - Rethink auto-detect.pdf
> 
> What are the a) b) c) items in the Table of results?
> e.g. "3.all unknown", so how do you get into case a) or b) or c) ?

That's the default delivery format preference for "prefers-unknown" which WADA and I are suggesting.
Let the user decide: prefers-unknown == (a) plain | (b) hmtl | (c) both

> Another question: can we just send Both if the mix of recipients is "all
> unknown" or "some unknown & some HTML"? Only if there is at least one plain
> recipient, consult the Send options. Would that work?

Yes, it can work and would be arguably better than now. My proposal also suggests that "unknown" scope should NOT be dominated by send options which are really designed for recipient scopes involving "prefers-plain" only, but today it's more likely that "unknown==hmtl". That's why I suggest limiting the scope of send options (ask | plain | html | both) to cases involving "prefers-plain" only.

But if we force "prefers-unknown"=="send-both", it means that we are taking a decision over the users head again to do what WE think is best for cases of unknown. Which is bad for both types of use cases:
- users who consider "unknown==plain" will not sufficiently get what they want (we force both where they might want plain)
- users who consider "unknown==html" will not sufficiently get what they want (we force both where they might want "HTML-only").

At the cost of only a single dropdown in options, we can easily reduce the ambiguity and let everyone have their way with a much smarter recipient-centric auto-detect.

Comment 166

[WADA:World Anti-bad-Duping Agency]

Attachment: edc

Sorry for frequent update.

Comment 167

[WADA:World Anti-bad-Duping Agency]

Attachment: baz

Sorry, update again.
"Simple solutions based on aceman's one-liner patch" is added.

To aceman, what do you think about the simple solutions?

Comment 168

[WADA:World Anti-bad-Duping Agency]

Attachment: srq

Comment 169

[WADA:World Anti-bad-Duping Agency]

Attachment: (v5) (line 4980 is omitted by mistake) Pseudo code for new prefs of Default for Unknown, and convert Unknown to HTML or Text in order to remove Prefers=Unknown case

I gave up feedback from someone.

Comment 170

[WADA:World Anti-bad-Duping Agency]

Attachment: yxw

Comment 171

[WADA:World Anti-bad-Duping Agency]

Attachment: vut

Sorry, typo.

Comment 172

[:aceman]

Comment on attachment 8675255 [details]
vut

I support your suggestions in caption [2], the "simple solutions".
1. I support using Send options also for "allplain" case (whenever there is at least one plain recipient, use Send options). Or use a separate option in Send options.
2. Adding a pref on how to interpret Unknown recipient seems a good idea, I think also Thomas supports/suggests this. I think it would be easy to do and I could implement it. But I would suggest there must also be a value of "Both" for this pref.

But you guys need to file these as simple bugs with clear descriptions and no fights/discussions about what actually is to be done (what the proposal actually is).

Comment 173

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #172)

See bug 1215791, please.

> But I would suggest there must also be a value of "Both" for this pref.

There are HTML Domain and Text Domain only in Send Options and Prefers=HTML and Prefers=Text only in Address Book.
Please enhance Send Options and Address Book for "Both" by yourself, which both of you want, before saying complaint to me.
Please implement Prefers=Both in Address Book and "Both Domain" in Send Options.

Comment 174

[WADA:World Anti-bad-Duping Agency]

aceman, please note that "Prefers=HTML/Text in Address Book" is "per mail address HTML Domain/Text Domain setting".
Plese don't say same thing as Thomas D.

Comment 175

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #172) [2], the "simple solutions".
> 1. I support using Send options also for "allplain" case
>    (whenever there is at least one plain recipient, use Send options).

This is done by "change for (B-2) part in your one-liner patch".
 -  if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
  Your patch : + if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
  My   patch : + if (allPlain)
  New  patch : + if (aConvertible == nsIMsgCompConvertible::Plain)

> 2. Adding a pref on how to interpret Unknown recipient seems a good idea,

This pref + "converting Prefers=Unknown to HTML or Text" is for (C-1).
By this, "change for (C-1) part in your one-liner patch and my one-liner patch" in not needed any more.

If prefs=HTML, almost (C-1) is changed to (A-1) which is allHtml case.
If prefs=Text, almost (C-1) is changed to (B-1) which is allPlain case.
Outstanding "HTML+Text" case, I called (D-1), is cared by your patch for bug 584313.
  If "CSS style attribute" is considered as Not-Convertible,
  "CSS style attribute" case in (D-1) is changed to
  (D-2) Not-Convertible case where Send Options is used.
  Remained (D-1) case is "HTML+Text" && Convertible=Yes && No CSS Style Attribute.
    Because Text Domain recipient is requested by user, user can understand reason why sent as text/plain.
    Because CSS Style Attribute case is not involed, no one shouts "massive data loss!!!".

Comment 176

[WADA:World Anti-bad-Duping Agency]

Sorry, mistake.
  Your patch : + if (aConvertible == nsIMsgCompConvertible::Plain && allPlain)

Comment 177

[WADA:World Anti-bad-Duping Agency]

To aceman, please note following.

Almost all complaints of Thomas D. is for "Pretty simple HTML && CSS Style Atrribute && Prefers=Unknown recipient only" case.
So, if Prefers=Unknown is converted to HTML via. new prefs, problem of "flood of comments" is resolved without change/enhancement in bug 584313.
Change/enhancement in bug 584313 is needed for "Pretty simple HTML && CSS Style Atrribute && Text Domain recipient is actually requested by user".
I believe that user who sets new prefs=Text won't create "Pretty simple HTML with formatting by fully utilizing CSS Style Atrribute". So, for such users, change/enhancement in bug 584313 is not needed.
If user uses ordinal HTML tags which is relevant to formatting by HTML tag, Convertible=No is returned regardless of CSS Style attribute usage.
If change of "convert Prefers=Unknown to HTML or Text" will be done, change/enhancement in bug 584313 is needed in case of that user who creates "Pretty simple HTML with formatting by fully utilizing CSS Style Atrribute" tried to send the mail to Text Domain(Prefers=Text) recipient.

Comment 178

[WADA:World Anti-bad-Duping Agency]

To aceman, Bug 1204379 and this bug can be called following:
  My trial to understand what Thohms D. is complaining for, what Thoms D. is wanting.
And, I think I could find culprit in this bug at last: Culprit is Prefers=Unknown.

Comment 179

[WADA:World Anti-bad-Duping Agency]

Please note that Thohms D. / Thoms D. in previous comment !== Thomas D. in this bug :-)

Comment 180

[Thomas D. (:thomas8)]

(In reply to WADA from comment #177)
> To aceman, please note following.
> 
> Almost all complaints of Thomas D. is for "Pretty simple HTML && CSS Style
> Atrribute && Prefers=Unknown recipient only" case.
> So, if Prefers=Unknown is converted to HTML via. new prefs, problem of
> "flood of comments" is resolved without change/enhancement in bug 584313.

Well, that depends on the global solution which we ultimately prefer.
imo, user-preference of unknown==html == prefers-html != guarantee for sending html-only always
Iow it will just mean "prefers-html" and we use that for determining the best compromise format using auto-detect algorithm, informed by send-options etc.

And, as you know, I have proposed to make auto-downgrade-to-plaintext-if-convertible::plain feature optional AND more easy to understand by also applying that to cases of prefers-html incl. allHTML.
Reason: auto-downgrade is *message-centric* option which supersedes *recipient-centric* options (as seen in current behaviour, reason for many bugs). But we should not confuse the user even more by just superseding some, but not all recipient-centric cases. I think users who accept that plaintext bias for "unformatted" messages will also be fine with sending plaintext to "prefers-html" recipients in that case. This feature looks at the message *content* only, so it's fine to ignore recipient preferences.
And prefers-html recipients can always handle plaintext afasik, so we're not creating problems for anyone...

So with my proposal, new pref for unknown==... will NO LONGER solve all cases.

My proposal also has advantage that we can default to unknown==html and still keep the auto-downgrade algorithm as a default, which seems to be wanted by Magnus and Ben. I still don't fully see why, but I can accept that if there's really NO formatting whatsoever, it's OK to default to sending plaintext IF the user gets an explicit choice. And for me, explicit/transparent means we need Auto-Downgrade checkbox in delivery format menu (AND in global compose options).

> Change/enhancement in bug 584313 is needed for "Pretty simple HTML && CSS
> Style Atrribute && Text Domain recipient is actually requested by user".

Not if my new downgrade logic above gets implemented.

> I believe that user who sets new prefs=Text won't create "Pretty simple HTML
> with formatting by fully utilizing CSS Style Atrribute". So, for such users,
> change/enhancement in bug 584313 is not needed.

Well, then why such user is using HTML editor? Also, in the near future, any HTML formatting from HTML editor will always create CSS. So only if the message is really de-facto plaintext without formatting, it can be sent as plaintext. Sounds reasonable to me.

> If user uses ordinal HTML tags which is relevant to formatting by HTML tag,
> Convertible=No is returned regardless of CSS Style attribute usage.
> If change of "convert Prefers=Unknown to HTML or Text" will be done,
> change/enhancement in bug 584313 is needed in case of that user who creates
> "Pretty simple HTML with formatting by fully utilizing CSS Style Atrribute"
> tried to send the mail to Text Domain(Prefers=Text) recipient.

dito, see above.

Comment 181

[Thomas D. (:thomas8)]

Comment on attachment 8675255 [details]
vut

> Note:<br>Convertible : Yes : aConvertible==Convertible::Plain,
>                        No  : aConvertible==Convertible::No

WADA, I think your results table is still wrong in this corner, both for current and expected/proposed results.

You do not consider that Convertible comes with as many as *four* different, gradual return values:

convertible == plain | yes | altering | no

Current Auto-Downgrade-and-Auto-Detect algorithm:
if (aconvertible==convertible::plain && recipient-scope != allHTML) --> auto-downgrade

convertible::plain is the only condition checked for auto-downgrading.
So all other cases are !convertible::plain.
But !convertible::plain != convertible::no
Not convertible-plain is NOT necessarily same as convertible-no!!!
!convertible::plain == (yes || altering || no)

Imo this is big and confusing logical error in your otherwise pretty fine representation.

Comment 182

[Thomas D. (:thomas8)]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #181)
> Comment on attachment 8675255 [details]
> (v6) Conclusion of this bug
> 
> > Note:<br>Convertible : Yes : aConvertible==Convertible::Plain,
> >                        No  : aConvertible==Convertible::No
> Imo this is big and confusing logical error in your otherwise pretty fine
> representation.

Perhaps this error can easily be fixed by just changing the legend:
Note:<br>Convertible : Yes : aConvertible==Convertible::Plain
                       No  : aConvertible==Convertible::Yes||Altering||No

Comment 183

[Thomas D. (:thomas8)]

Comment on attachment 8675255 [details]
vut

> allPlain 	B-2 	false 	true 	No Unknown 	No 	nsIMsgCompSendFormat::
PlainText	Auto-Downgrade
Possible data loss for mail sender,
because Not-Convertible
(Note: Never for Text Domain recipent)

I don't understand "Note: Never for Text Domain recipient"? Pls explain.

NI for my previous comments, too.

Comment 184

[Thomas D. (:thomas8)]

Comment on attachment 8675255 [details]
vut

My comment on WADA's simple solution proposal [2]:

> * Add support of CSS Style atribute in Convertible check logic.
>   Simple HTML && CSS Style && Convertible case is changed to Not-Convertible case.
>   So, frustration of Thomas D. disappears.
I guess I'm not the only one who doesn't like entire stylings dumped without a choice, but yes indeed, that's the biggest painpoint.
Beyond that, I maintain that auto-downgrade must be *optional*, so we need a simple global pref for that. Ideally, to avoid any mystery and surprises, expose that pref in the primary UI of delivery format menu, with a simple checkbox below Auto-Detect menu item:
[x] Plain text if no formatting

> * In allHtml/allPlain determintion step,
>   interpret Prefers=Unknown as HTML or Text according to prefs value.
>   current: if(allPlain) send as text/plain; return;
>   new: ==> if(allPlain&&Convertible) send as text/plain; rerurn;
>   When allPlain&&Not-Convertible, code goes to otherwise step, so Send Options is used.

Where not-convertible is really !convertible::plain, so it can be any of convertible:: yes | altering | no. See my Comment 181.

> * Wording change in Send Options panel.
>   Panel is shared between
>   (B-2)  allPlain  &&Not-Convertible case and
>   (D-2) (HTML+Text)&&Not-Convertible case (mixed case)

Yes, it is possible to keep a single option for all recipient scopes involving at least one "prefers-plain" recipient (so possibly involving prefers-HTML-recipients, too). So wording could be something like
"If any recipient prefers to receive messages as plaintext:"
Only possible after disambiguation of "unknown" as proposed above.

To allow full utilization of recipient-centric approach for cases of prefers-plain, specificially allPlain case, I think we should split that up into one option each for (B-2) allPlain and (D-2) mixed (someHtml & somePlain).
For recipient scopes of allPlain, user might want more radical downgrading thatn for scope of mixed (someHTML & somePlain), where sending "both" is usually best.

> Send logic after simple change
> 
> HTML       Text       Convertible=Yes   Convertible=No   recipient scope
> recipient  recipient	(message-centric) (recipient-centric)
> ------------------------------------------------------
>   O          -        send text/html    send text/html    (allHTML)
>   -          O        send text/plain   use Send Options  (allPlain)
>   O          O        send text/plain   use Send Options  (mixed)

Wow, this is how simple auto-detect choices become if we implement that single little pref for disambiguating "prefers-unknown", here with WADA's dual choice between unknown==plain || unknown==html.

Adding a third logical choice of unknown==both will not significantly complicate that because where one recipient prefers "both", we can usually just send "both", unless explicitly told otherwise for cases involving "prefers-plain". Internal logic will be a bit harder, but still entirely predictable for user.

What you can see very nicely from this little table is where we are currently inconsistent about mixing message-centric and recipient-centric behaviour:
message-centric auto-downgrade wins over recipient-centric for all cases (allPlain, mixed), except for case of (allHTML), where we suddenly fall back to recipient-centric without real need. If user agrees to message-centric auto-downgrade as a concept, recipient preferences for that case are irrelevant (more so because prefers-HTML users can definitely handle plaintext, but perhaps not vice versa).

If auto-downgrade is global option, AND we expand scope of auto-downgrade to apply to allHTML case, too, only then can we default to unknown=="html" AND keep auto-downgrade ON by default, which seems the preferred option e.g. for Magnus and Ben, perhaps WADA, too.

Comment 185

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #172)

Everyone has their own wordings, so I'll reply to Aceman, too (same as previous comment):

> Comment on attachment 8675255 [details]
> (v6) Conclusion of this bug
> 
> I support your suggestions in caption [2], the "simple solutions".

Yes, that's a pretty good proposal. I'd agree to that with a few changes in the detail.

> 1. I support using Send options also for "allplain" case (whenever there is
> at least one plain recipient, use Send options). Or use a separate option in
> Send options.

If we want to fully support more aggressive downgrading for allPlain case, which is usually not wanted for mixed case, then separating the two recipient scopes is needed:
(B-2)  allPlain &&Not-Convertible case and (all recipients prefer plaintext)
(D-2)  mixed    &&Not-Convertible case     (some, but not all recipients prefer plaintext; some prefer HTML).

> 2. Adding a pref on how to interpret Unknown recipient seems a good idea, I
> think also Thomas supports/suggests this. I think it would be easy to do and
> I could implement it. But I would suggest there must also be a value of
> "Both" for this pref.

+1. Big thanks to WADA for pushing us into finding this really simple and effective solution.
I agree with Aceman that we should also offer "unknown==both" because it's legitimate, useful, recommended exactly for "unknown" recipient preference. This makes our internal matrix a bit more complicated, but not really hard because "both" will just be auto-detected to sending "both" unless cases involving "prefers-plain" require otherwise.

> But you guys need to file these as simple bugs with clear descriptions and
> no fights/discussions about what actually is to be done (what the proposal
> actually is).

Your voice also counts :)
Between WADA, myself, and Aceman, I can already see a far-reaching agreement on the key issues.

Aceman and I can outvote WADA, to include unknown==both, so that's solved :)

WADA, can you pls comment on the following:
- What do you think of expanding scope of optional Auto-Degrade to allHTML case, so that we have a predictable, fully message-centric feature which never looks at recipient preferences? I think that's easier to understand for user. Auto-Downgrade-if-convertible::plain supersedes recipient preferences. Always.
- What do you think about having two separate send options for allPlain vs. mixed (somePlain&someHTML)? Which allows more aggressive downgrading for allPlain, usually not wanted for mixed. Legacy case, probably mostly useful for recipients who'd really choke on multipart/alternative.

Comment 186

[Thomas D. (:thomas8)]

WADA, Aceman, FYI:

My full proposal (allowing unknown==both; separate send options for allPlain vs. somePlain recipient scopes) can still work with very simple send logic (each 'if case' exits the logic with 'return'):

if autoDowngrade==true && convertible::Plain    send Plain
if (allHTML)    send HTML
if (allPlain)   use SendOptions1 (default: both) [send option for allPlain]
if (somePlain)  use SendOptions2 (default: both) [send option for somePlain]
if (someBoth)   send both
(These send logics cover all scopes, so we must NOT default to asking after this)

Note:
autoDowngrade==pref(AutoDowngrade)
(somePlain) scope covers all combinations involving any recipient who prefers plaintext:
- somePlain & someHTML
- somePlain & someHTML & someBoth
- somePlain & someBoth
(someBoth) scope covers (allBoth), too

Pseudo code for setting the new send logic variables:
if recipient-preference==plain is found --> somePlain=true
if recipient-preference==unknown-both is found --> someBoth=true

Comment 187

[Thomas D. (:thomas8)]

Attachment: Proposal P-T2 (Recipient scopes & send logic).pdf

Here's a full breakup of possible recipient scope combinations and the resulting auto-detect send formats of my Full Proposal P-T2 (XUL-Demo attachment 8672688 [details], screenshot attachment 8673807 [details]).

This is the background explanation for simple send logic of my proposal (see Comment 186), including unknown==both, and two separate send options for allPlain vs. somePlain scopes.

Comment 188

[WADA:World Anti-bad-Duping Agency]

Sorry, I wanted to protect my documents, so I set "Restrict ...", then Securty flag was set.
How can I restrict access to this bug without Security flag?

Comment 189

[WADA:World Anti-bad-Duping Agency]

To "users in all of the selected groups can view this bug" :

Please let me know how to delete files I uploaded to bmoattachments.org(attachment of a bug of bugzilla.mozilla.org), before clearing Security flag.
I wanted to protect my docuents which I uploaded from any accees from any user of B.M.O.

Comment 190

[:aceman]

Comment on attachment 8675246 [details]
(v5) (line 4980 is omitted by mistake) Pseudo code for new prefs of Default for Unknown, and convert Unknown to HTML or Text in order to remove Prefers=Unknown case

Yes, this pseudocode looks like what we aim to implement. If there are only 2 states (HTML and plain) for the new pref, maybe that is enough. I need to start coding it whether we can get away with 2 states and the code determines which of the 3 formats to emit (HTML, Both, plain), or if we need help from the user to decide and need all 3 states in the pref.

Comment 191

[WADA:World Anti-bad-Duping Agency]

"Both support in new pref" itself is pretty simple.
  xxx=new pref value (0=Text, 1=HTML, 2=Both)
  UnknownIsTextDomain = (xxx==0) ;
That's all. Mapping : 0 => Text Domain, 1/2 => HTML Domain
xxx can be freely used upon send, in addition to allHtml/allPlain.

But I can't understad reason why Both is needed.
  if(allPlain), multipart/alternative shouldn't be used because Text Domain recipients only.
  if(HTML+Text), there is no need of pref=Both, because Send Options panel is used.
For "if(allHtml)" case? If so, why text version is needed even though all recipients can show HTML?
Different option is needed for "if(HTML+Text && Convertible)" and "if(HTML+Text && Not-Convertible)"?
If so, I believe that proper method is "enhancement of Send Options panel".

These are reasins why I wrote following.
  "Repeating Both, Both, Both" sounds for me being monomaniac.

Comment 192

[WADA:World Anti-bad-Duping Agency]

Above is my last comment on pref=Both.
I'll never refer it, touch it, any more.

Comment 193

[WADA:World Anti-bad-Duping Agency]

Sorry, typo.
These are reasons why I wrote following.
  "Repeating Both, Both, Both" sounds for me being monomaniac.

Comment 194

[WADA:World Anti-bad-Duping Agency]

"if(HTML+Text&&Convertible){ if(global pref of Default for Unknown=Both) multipart/alternative"; else text/plain ; }" is wrong way.
"Global pref use for this code only" is wrong way.
If such thing is needed, "Both support in Address Book" should be implemented first and Prefers=Both/HTML/Text/Unknown should be clearly defined and mapping to HTML Domain/Text Domain should be clearly defined.
Or, it should be conrolled by Send Options panel and Send Option UX code.

Comment 195

[WADA:World Anti-bad-Duping Agency]

Such dirty trick is wrong way. "Injecting such dirty trick to code by request from pretty limited users only" is wrong way. "Injecting such dirty trick to code for pretty limited users only" is wrong way.

Comment 196

[WADA:World Anti-bad-Duping Agency]

I've understood and just remebered "What was Both, Both, Both".
Code is pretty simple.
  xxx=new pref value (0=Text, 1=HTML, 2=Both)
  UnknownIsTextDomain = (xxx==0) ;
  Do allHtml/allPlain setting.
  At send srep,
  if(allHtml) send text/html;
  else if(Convertible && (!allHtml&&!allPlain) && xxx==2) send nultipart/alternative;
  else if(Convertible) send text/plain;
  else use Send Options
But, aceman, please don't do such thing in clean initial version.
Please do such thing in next enhancement.
Do you want WONTFIX? :-)

Comment 197

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #190)
> Comment on attachment 8675246 [details]
> (v5) (line 4980 is omitted by mistake) Pseudo code for new prefs of Default
> for Unknown, and convert Unknown to HTML or Text in order to remove Prefers=Unknown case
> 
> Yes, this pseudocode looks like what we aim to implement. If there are only
> 2 states (HTML and plain) for the new pref, maybe that is enough.

Following change was included in proposal because you and Magnus said "a data loss" case.
 - 4978   if (aConvertible == nsIMsgCompConvertible::Plain || allPlain)
 +        if (aConvertible == nsIMsgCompConvertible::Plain)
(Not-Convertible && allPlain)
Because there are not bug reports for this case, and because this case is allPlain case, and because this logic was used for long time, I think above part is better not touched.
I think following is sufficient and safer as clean/initial version.
(1) Convert Prefers=Unknown to HTML by pref setting=HTML.
    => all Unknown case is changed to allHtml case.
    => mixed case is "HTML + Text", so at least one Text Domin recipient is contained in recipients.
       Because Text Domain recipoent is actually requested by user, "send as text/plain" is logical.
(2) CSS Style support in Convertible check logic.
    CSS Style case is changed from Convertible to Not-Covertible, so problem is surely reduced.
If above part is not touvhed, there is no need to touch Send Options panel.

Comment 198

[WADA:World Anti-bad-Duping Agency]

I think "Remove Prefers=Unknown case" and "send logic change/improvement for different purpose" should be separated, as "CSS Style support in Convertible check logic" is separated.

Comment 199

[WADA:World Anti-bad-Duping Agency]

I believe that "all-in-one solution for all complaints by a someone on all issues which is relevant to Auto-Detect" should be avoided.

Comment 200

[WADA:World Anti-bad-Duping Agency]

I probably wrote all request from me to you on minimum change.
I end touching the change. Good luck, aceman.

Comment 201

[WADA:World Anti-bad-Duping Agency]

Waynw, thanks for clearing security flag which I set by my mistake and wrong operation.

Comment 202

[Thomas D. (:thomas8)]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #186)
> WADA, Aceman, FYI:
> 
> My full proposal (allowing unknown==both; separate send options for allPlain
> vs. somePlain recipient scopes) can still work with very simple send logic
> (each 'if case' exits the logic with 'return'):
> 
> if autoDowngrade==true && convertible::Plain    send Plain
> if (allHTML)    send HTML
> if (allPlain)   use SendOptions1 (default: both) [send option for allPlain]
> if (somePlain)  use SendOptions2 (default: both) [send option for somePlain]

FTR: For the sake of simplicity, compromise and moving on faster, I'm fine with conflating (allPlain) and (somePlain) scopes into one (somePlain) two scopes into one (which makes radical downgrading for allPlain harder but maybe that's not a very important/frequent usecase).

So the total matrix becomes even simpler:

if (autoDowngrade==true && convertible::Plain) {send Plain; return;}
if (allHTML)    {send HTML; return;}
if (somePlain)* {use SendOptions (default: both); return;}
if (someBoth)   {send both; return;}
(These send logics cover all scopes, so we must NOT default to asking after this)
 
> Note:
> autoDowngrade==pref(AutoDowngrade)
> (somePlain) scope covers all combinations involving any recipient who prefers plaintext:
> - somePlain & someHTML
> - somePlain & someHTML & someBoth
> - somePlain & someBoth

- allPlain, too (in case of combined send options for somePlain and allPlain scopes)

> (someBoth) scope covers (allBoth), too
> 
> Pseudo code for setting the new send logic variables:
> if recipient-preference==plain is found --> somePlain=true
> if recipient-preference==unknown-both is found --> someBoth=true

Auto-Downgrade to bypass auto-detect, and Auto-Detect recipient scopes shown in graphical table of attachment 8675716 [details].

Comment 203

[WADA:World Anti-bad-Duping Agency]

(In reply to Thomas D. (currently busy elsewhere; needinfo?me) from comment #202)

To Thomas D.

I think latast yur mock-up of a Send Options panel is sufficient for a mock-up for possible or required changes or planned changes by aceman.
  Statements are never invalid even if it's lengthy for actul Send Options panel which size is limited.
  UI for all possible requred preference is contained.
  All send format options for possible requred change is contained.
What will be required next is "design of actual panel" according to actually chosen solution.
So, there is no need to do additional work for your mock-up now.

HTML attached to bug 1218156 is "Send Options panel which is required when all is done via Send Options setting".
Because it's merely a presentation of concept now, and purpose of bug 1218156 is for validity check of possible code change only, I don't provide required statement in panel for user yet.
  I use "Convertible", "Yes" , "No" in concept presentation/
  What kind of user can understand by such wording?
To keep "table for send format option for all 6 cases" "Compact", at least following is needed.
  Better wording than "Convertible"
  Descrition about the "Better wording than Convertible" in panel at appropriate plaace for user.
These are not needed for validity check of possible code change.
However, mock-up for design/implementation of actuaal panel will be required if such code change will be accepted.

Tohmas D., can you help me to create mock up for HTML attached to bug 1218156?

As you already know, "creating mock-up" is a "try and error".
This bug is already closed as WONTFIX, so there is no problem in frequest update of content of attachment :-)
Because basic layout of "UI for pref + send format optons + HTML/Text Domain setting" can not be changed, design of "UI for pref" and "send format optons" only is needed.
For ease of viewing mock-up, HTML is better than PDF/XUL.

If you can help, download "HTML attached to bug 1218156", modify it, upload to this bug, please.

Please never add comment to bug 1218156", because that bug is for validity check of possible code change by developers.

Comment 204

[WADA:World Anti-bad-Duping Agency]

To Thomas D.

As I wrote in new bug, there is requirement like next.
  Utilize HTML mode composition for pure text(text/plain) mail creation.
    For visibility during mail composition,
    <h3> for heading of section, italic for quoting,
    Body Text or <p> for body of mail, <pre> for attached data.
Design of a feature of software should pay attention to such requirement.
Design of a feature of software is never requesting what you want only.
And, "send multipart/alternative even when allHtml" is never wrong or incorrect or invalid action, as far as it is done by user's request.
Even when allPlain, it's same, although good manner for majority of recipient is "send as text/plain".
Because many mailers can understand multipart/alternaative and can use text/plain part for him, "sending multipart/alternative" is never wrong or incorrect or invalid, if user has such requirement in his environment, even though good manner for majority of recipient is "text/plain only" in this case.

Please never repeat "killing auto-downgrade is needed!" or "default of Both is mandatory for Prefers=Unknown!" etc. any more.

Comment 205

[Thomas D. (:thomas8)]

(In reply to WADA from comment #203)
-> answered via private mail

Comment 206

[WADA:World Anti-bad-Duping Agency]

Comment on attachment 8675246 [details]
(v5) (line 4980 is omitted by mistake) Pseudo code for new prefs of Default for Unknown, and convert Unknown to HTML or Text in order to remove Prefers=Unknown case

To aceman, pseudo code is moved to "(a) convert Unknown to HTML/Text" part of pseudo code attached to bug 1218156, because bug 1215791 for implementing it was interfered.

Comment 207

[WADA:World Anti-bad-Duping Agency]

To Thomas D.

For Prefers in Address Book.

As term of "Prefers" is used in Address Book, I also though it's perhaps "send format" like one when it was implemented in Address Book, because word of "send format" is seen some places.
And, it was corrct. In definitions, term of "send format" was actually used.
 nsIMsgCompSendFormat
 http://mxr.mozilla.org/comm-central/source/mailnews/compose/public/nsIMsgCompose.idl#48
 nsIAbAddressCollector
 http://mxr.mozilla.org/comm-central/source/mailnews/addrbook/public/nsIAbAddressCollector.idl#16
 collectAddress
 http://mxr.mozilla.org/comm-central/source/mailnews/addrbook/public/nsIAbAddressCollector.idl#33
 nsIAbPreferMailFormat
 http://mxr.mozilla.org/comm-central/source/mailnews/addrbook/public/nsIAbCard.idl#12
As known by these, term "Prefers" is used for term "send format", and as known by nsIAbPreferMailFormat, and as aceman says, it's currently has trippe values of unknown, plaintext, html. "what send format for Unknown", "what action is expected for Unknown" etc. is never defined.

However, as known by the definition of nsIAbPreferMailFormat, meaningful value is currently plaintext and html only.
And, what we are talking about is "Auto-Detect code/definition in Auto-Detect". It's never about "Address Book code/definition in ddress Book".
And, as I stated, "pref of default for Unknown" which I call is "how Prefers=Unknown in Address Book is interpreted in Auto-Detect code".

Current code is for allHtml/allPlain from HTML/Text Domain, and is enhanced for "Prefers=HTML/Text in Address Book".
So, for Auto-Detect, "Prefers=HTML/Text in Address Book" is equivallen to "HTML Domain/Text Domain".
Because "pref of default for Unknown" I call is "how Prefers=Unknown is interpreted in Auto-Detect code", and because meanigful value defined in nsIAbPreferMailFormat is currently plaintext and html only, I used bool pref/variable for it.

If you and aceman want "default=Both in Address Book", you and aceman should clearly/cleanly define "Prefers in Addrress book" as "per email address preference setting of send format used in Send Options panel", and should enhance all definitions in Address Book, and should define "mapping from Prefers=Ask,Both,HTML,Text to HTML Domain/Text Domain for allHtml/allPlain in Auto-Deect code".
Note: Curent code implies "mapping of Prefers=HTML -> HTML Domain for allHtml, Prefers=Text -> Text Domain for allPlain", as easily known by code and my psuedo code.

Because I want to share "pref of default for Unknown", which I call so, with setting in Address Book, and because meaningful value is currently plaintext and html only, I used bool pref for it.

I believe.
1. For "per email address HTML/Text Domain setting", enhancement in Address Book" was requested.
2. Because it can be used for "per email address send format in Send Options panel",
   it's morphed to Prefers, and plain/html only was defined with Unknown for Collected Address Book.
3. Nothing is done yet for "per email address send format in Send Options panel" in Address Book,

If "Prefers in Address Book" is "per email address send format in Send Options panel", user can freely use it for "user want to send mail in format specified for this Cntact regardless of other conditions" or for "recipoent of this Contact want this formt or can recieve Text only if Prefers=Text".

Comment 208

[WADA:World Anti-bad-Duping Agency]

To Thomas D.
If Address Book will be enhanced, enhancement like next will be possible.
(1) User sets Prefers=Ask temporarily because recipient may reject HTML maail.
    In Auto-Detect, if one Prefers=Ask recipient is contained, ask to user.
    In ask dialog, # of Ask/Both/HTML/Text recipients are shown, and Convertible/Not-Convertible is shown,
    and user can request to save his choice in Send Options panel, and clear of Ask for Prefers=Ask Contacts.
(2) User sets Prefers=Both for Cotacts in Address Book.
    Auto-Detect chooses appropriate send format based on # of Both/HTML/Text Contacts in recipients.
    Needless to say, Default of Both for Prefers=Uknown, which you are eager to set, can be set.

This is also a "Win back control of send format from Tb" and a "removel of forcing send format in Tb".

Comment 209

[:aceman]

Are the feedbacks from me still needed here? It seems to me the needed components have patches in other bugs now.

Comment 210

[WADA:World Anti-bad-Duping Agency]

(In reply to :aceman from comment #209)
> Are the feedbacks from me still needed here? It seems to me the needed
> components have patches in other bugs now.

It's moved to "Convert Prefers=Unknown to HTML Domain or Text Domain" part of bug 1218156.