Open Bug 699681 Opened 9 years ago Updated 3 years ago

[Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing X-Account-Key: based identity selection or "Reply-to-self" on any user, by inappropriate mail addr matching/guessing or Reply-To handling)

Categories

(Thunderbird :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

People

(Reporter: World, Unassigned)

References

(Depends on 23 open bugs, )

Details

(Keywords: meta, Whiteboard: [See URL: field for a workaround])

Preset From: upon Reply/Forward is frequently far different from user's expectation, due to (a) forcing of X-Account-Key: based identity selection for Global Inbox support on any user, and due to (b) forcing of "Reply-to-Self" on any user even though the feature is for convenience of limited users only.
This produced not-so-small number of unwanted bugs like "Wrong from in reply", "Mail was sent using wrong From:".
This bug is meta bug(tracking bug) for analysis of such bugs and duping of them.

For (a), bug 327713 exists.
For (b), I don't know about existent bug.

According to bug 327713 comment #53, Add-on of "Correct Identity" looks effective for victims of the problem.
> https://addons.mozilla.org/en-us/thunderbird/addon/correct-identity/
Whiteboard: [See URL: field for a workaround]
Depends on: 397975
Meta bug for general "unexpected identity selection" is found.
xref-ing to that bug via Blocks: field of this bug.
> Bug 228980 : (IdentityPickTracker) [Meta] Identity selection based on recipients should be more correct
Blocks: 228980
Depends on: 701681
Depends on: 279846
Depends on: 704613
Depends on: 684259
Note-1:
If account number is changed at a Tb, phenomenon produced by "identity selection based on X-Account-Key: header" becomes funnier.
Note-2:
Because bug 426651 exists, "identity selection based on X-Account-Key: header" can occur even on mail in IMAP folder. xref bug 426651.
See Also: → 426651
> According to bug 327713 comment #53, Add-on of "Correct Identity" looks
> effective for victims of the problem.
> https://addons.mozilla.org/en-us/thunderbird/addon/correct-identity/

As noted in the "Additional information" section of the original Description of bug 684259, installing Correct Identity did not help in that situation.
FYI.
Identity selection for preset From: is done at following code.
http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#1930
> 1930       switch (type)
> 1931       {
> 1932         default: break;
> 1933         case nsIMsgCompType::Reply :
> 1934         case nsIMsgCompType::ReplyAll:
> 1935         case nsIMsgCompType::ReplyToList:
> 1936         case nsIMsgCompType::ReplyToGroup:
> 1937         case nsIMsgCompType::ReplyToSender:
> 1938         case nsIMsgCompType::ReplyToSenderAndGroup:

I think enhancement like next will reduce many user's current confusion.
(a) mailnews.reply_to_self_is_enabled = true/false (default=false)
    If possible, per account option is preferable.
    Global setting & per account setting like retention policy may be better.
(b) mailnews.account_key_based_identity_selection_when_non_Global_Inbox_owner
        = true/false (default=false)
    If possible, per account option is preferable.
    When Global Inbox owner account, this option should be ignored.
    Global setting & per account setting like retention policy may be better.
Use cases of them are independent from use case of "per folder identity", although "per folder identity" can reduce current user's confusion too.
Note-3:
Because bug 270068 exists, "identity selection based on X-Account-Key: header" can occur even on mail which is sent as attachment, saved as .eml, then imported by Drag&Drop of the .eml file. xref bug 270068.
See Also: → 270068
Depends on: 696652
Note-4:
Reply-to-self is affected by next setting.
  mailnews.reply_to_self_check_all_ident = true/false(it looks false is defaulted)
  perhaps next;
    true  : any identity is considered as "self"
    false : identities of selected account only is considered as "self"
Note-5:
Identities are defined in prefs.js like next;
 (account definition)
  mail.account.accountA.identities = idAa,idAb, ... ,idAz
                                     Left most one=Main identity of this account
  mail.account.accountA.server     = serverS
 (mail server definition)
  mail.server.serverS.realhostname/hostname = mail.server.name
  mail.server.serverS.realuserName/userName = login-useid-of-this-mail-account
 (identity definition)
  mail.identity.idAa.useremail = mail-addr-Aa@mail-domain-for-Aa
                   |
  mail.identity.idAz.useremail = mail-addr-Az@mail-domain-for-Az
 (identity's SMTP server choice)
  mail.identity.idAa.smtpServer = smtpP
 (SMTP definition)
  mail.smtpserver.smtpP.hostname = smtp.server.name
  mail.smtpserver.smtpP.username = login-useid-of-this-smtp-account
Depends on: 706451
Depends on: 706461
Tb 8 on Win-XP showed next in Account column.

account1/server1 to account8 is defined. No Global Inbox is used.
  account1/server1 : POP3, not default account
  account2/server2 : POP3, default account
  account3/server3 : Unified Folders
  accountP/serverP : POP3, not default account
  accountS/serverS : IMAP account
  accountL/serverL : Local Folders  
Checked at POP3 accountP/serverP(not server1, not default account, not Local Folders, not Unified Folders), Local Folders, and IMAP account.

                            accountP         accountL         accountS
                            serverP          serverL          serverS
 X-Account-Key: of mail     POP3 folder      Local Folders    IMAP folder 
 
 mail-X : accountN          accountN         accountN         serverS.name of    
   (accountN is defined)    -> serverN.name  -> serverN.name  this IMAP account
   (non Local Folders  )

 mail-Y : accountL          accountL         accountL         serverS.name of    
   (accountL is defined)    -> serverL.name  -> serverL.name  this IMAP account
   (as Local Folders   )    ==Local Folders  ==Local Folders   

 mail-Z : accountC          serverP.name of  serverL.name of  serverS.name of    
   (accountC is        )    this account     this account     this IMAP account
   (  not defined      )                     ==Local Folders   

 mail-W : no X-Account-Key  server1.name of  server1.name of  serverS.name of    
                            account1         account1         this account
     
Note:
Even when existent accountN is hidden account for Unified Folders(smart mail boxes), corresponding serverN.name is used and "Unified Folders" is shown.

I wasn't aware of next until I execute above check.
  account1 or accountX for server1 is always used when no X-Account-Key: header.
This is possibly one of big causes of phenomenon of "preselected reply address is far from user's expectation" in no X-Account-Key: header case.
Some funny phenomena may be no server1/account1 case.
Note.
mail.accountmanager.accounts in above check is as follows (account9999/server9999=Local Folders).
> account2,account4,account9999,account1,account3,account7,account5,account6,account8
So order of accounts is perhaps irrelevant to "always select server1 or account1 if no X-Account-Keys:".
Summary: [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing of X-Account-Key: based identity selection on any user and by forcing of "Reply-to-self" on any user → [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing of X-Account-Key: based identity selection on any user, by forcing of "Reply-to-self" on any user, by inappropriate mail addr matching/guessing)
Depends on: 724792
FYI.
Actually WRONG pre-set From: case existed - Bug 397975 (fixed by Tb 13.0).
If pqr@xx.yy.zz is contained in To:, CC:, and if pqr@xx.yy.zz is not defined as Identity but abc@xx.yy.zz is defined as Identity, abc@xx.yy.zz is selected as Identity for reply mail due to funny mail address guessing without localpart matching.
Depends on: 679217
FYI.
There are three known/major kinds of "Unwanted/Unexpected pre-set From:".
(1) Due to Tb's identity selection based on identity related Tb's message header,
.   even when user doesn't want it.
    Based on X-Account-Key: header, X-Identity-Key: header.
(2) Due to Tb's "Reply-to-self always even when user doesn't want it".
(3) Due to mail address guessing without localpart matching (Bug 397975).
I have again a false sender address when I transfer an email received on one account thunderbird send it with an different account as sender 

I have made a screen copy but it is not possible to post it heare. But the contents is, when I klick on "transfer to",  similar to 

sender  aaaaaa@xx.com 
response to: aaaaaa@xx.com 

and in the text field the header is 

subject
date
from: bbbbbb@xx.com 
respone to bbbbbb@xx.com 


Please take also note that the default account is cccccc@xxy.com  ... will say he do not use the default account instead of the original used account
To say it in a simpla way: when I receive a mail on the account aaa@xx.com the transfered mail must be send by aaa@xxx.com with reply address aa@xx.com .

In the moment thunderbird use a different account to transfer the mail.
Depends on: 523799
Depends on: 760784
Depends on: 768877
Depends on: 767829
No longer depends on: 767829
Blocks: 680500
No longer depends on: 680500
No longer blocks: 680500
Depends on: 680500
No longer blocks: 228980
Depends on: 228980
Depends on: 621139
Depends on: 474725
Bug 608513 is for user's inconvenience in utilizing current reply-to-self feature. It sounds that mailnews.reply_to_self_check_all_ident=true(defaulted as false) is a solution in his case. So that bug looks "reply to his identity which is associated to different account" case.
Following enhancement may be a solutuion for users who don't want automatic reply-to-self application.
  mailnews.reply_to_self_check_all_ident : Boolean -> Integer
    0 : reply-to-self is disabled
    1 : reply-to-self is enabled, check identities of an relevant account only
    2 : reply-to-self is enabled, check identities of all accounts
Depends on: 520812
Depends on: 815732
FYI.
As for problem due to "X-Account-Key: based identity selection", powerful sord has been implemented.
  Re-define POP3 account
By bug 485839, mail.account.lastKey was introduced, and mail.account.lastKey + 1 is used as account number upon next account definition. So, delete/re-define POP3 account alters account number of POP3 account, then accountN in existent X-Account-Key: header is changed to obsolete.
(1) If default account, select appropriate account as default account,
    in order to avoid problem due to "defaultaccunt=Local Folders" etc.
(2) Delete POP3 account, re-define POP3 account
    => account number is incremented.
    => server number may be altered, and new directory-rel is used.
    Old one : mail.server.serverX.directory-rel = .../<servername>-XX
    New one : mail.server.serverY.directory-rel = .../<servername>-YY
(3) Use mail directory assinged to Old one.
    mail.server.serverY.directory-rel = .../<servername>-XX
    restart Tb.
(4) Select appropriate account as default account
(5) If you start to feel "X-Account-Key: based identity selection" is annoying, do (1) to (4) each time.
Just fix the #$@% bug. Add an option to ignore X-Account-Key.
Depends on: 888957
Depends on: 903390
Recently, problem around Reply-To: has been reported on Tb 24.
  For example, bug 932774 bug 933377, bug 933555.
Addin "Reply-To:" to ug summary.
Depends on: 932774, 933377, 933555
Summary: [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing of X-Account-Key: based identity selection on any user, by forcing of "Reply-to-self" on any user, by inappropriate mail addr matching/guessing) → [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing X-Account-Key: based identity selection or "Reply-to-self" on any user, by inappropriate mail addr matching/guessing, or Reply-To handling)
Quick summaary of current problems.
(1) Problem due to wrong mail addr matching or inappropriate mail addr guessing.
    Fixed in Tb 13.0 by Bug 397975.
(2) Problem due to "forcing X-Account-Key: based Identity selection, even when Global Inbox is never used by Tb user".
(3) Problem relevant to some other X-...: headers used by Tb.
(4) Problem due to "forcing Reply-To-Self, even when Tb user doesn't want it".
(5) Problem which may be relevant to not-so-appopriate behavior or deffernt-from-user-expectation behavior on Reply-To:, Sender: etc.
(6) Others.
Summary: [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing X-Account-Key: based identity selection or "Reply-to-self" on any user, by inappropriate mail addr matching/guessing, or Reply-To handling) → [Meta] Preset From: upon Reply/Forward is frequently far different from user's expectation, due to forcing X-Account-Key: based identity selection or "Reply-to-self" on any user, by inappropriate mail addr matching/guessing or Reply-To handling)
No longer depends on: 933555
No longer depends on: 932774
No longer depends on: 933377
Duplicate of this bug: 680500
Depends on: 1001899
Duplicate of this bug: 1034718
Surely the idea of multiple accounts is that an account is setup for each 
email identity that the user has, ie Home, work, charity etc. 
Then it make sense to keep the mail relating to that identity in folders 
in the relevant account, ie any work related emails would be kept in the 
work\inbox, or work\sent etc. 
Emails could get there by either downloading from different pop servers, 
or by dragging from another account if all incoming mails are consolidated to one server.

Composing a new message when a certain account is highlighted works correctly, 
ie the correct formatting is selected (text/html) and the correct from address 
is selected. 

When it comes to replying or forwarding a message that is highlighted, 
it should work the same way, BUT IT DOESNT !!

Please correct me if I am wrong, but I believe the solution is blatantly obvious and very simple:
1 Ignore the X-Account-Key, or add an option to ignore the X-Account-Key
2 When replying to a message, or forwarding a message, use the same subroutine that creates a new message. 
   That correctly Opens a new edit window
   Uses the correct formatting 
   Uses the correct signature, CC and Bcc addresses
   Uses the correct From address 

To cater for different setups/users then it would be sensible to have some options 
that the user could select in their preferences.
(In reply to David from comment #21)
> Surely the idea of multiple accounts is that an account is setup for each 
> email identity that the user has, ie Home, work, charity etc. 
> Then it make sense to keep the mail relating to that identity in folders 
> in the relevant account, ie any work related emails would be kept in the 
> work\inbox, or work\sent etc. 
> Emails could get there by either downloading from different pop servers, 
> or by dragging from another account if all incoming mails are consolidated
> to one server.
> 
> Composing a new message when a certain account is highlighted works
> correctly, 
> ie the correct formatting is selected (text/html) and the correct from
> address 
> is selected. 
> 
> When it comes to replying or forwarding a message that is highlighted, 
> it should work the same way, BUT IT DOESNT !!
> 
> Please correct me if I am wrong, but I believe the solution is blatantly
> obvious and very simple:
> 1 Ignore the X-Account-Key, or add an option to ignore the X-Account-Key
> 2 When replying to a message, or forwarding a message, use the same
> subroutine that creates a new message. 
>    That correctly Opens a new edit window
>    Uses the correct formatting 
>    Uses the correct signature, CC and Bcc addresses
>    Uses the correct From address 
> 
> To cater for different setups/users then it would be sensible to have some
> options 
> that the user could select in their preferences.

@david  ..

I am OK with you, but not with "relating to that identity in folders 
in the relevant account,"  Folders does not have any relation with the senders mail address ... The only criteria is the mail address where I receive a mail ... this must be every time the senders mals and the reply mail addres. 

brainstuff
Depends on: 1076384
(See Bug 1076964 for copious documentation.)

Something has changed in this area of the code.

Before 31.1.2 I never had a problem with TB selecting the correct 'From:' identity in a 'Reply' or 'Forward'.  Now, it consistently picks the wrong identity for replies to certain incoming mail.  Not only is the identity not associated with the address to which the email was sent, it's not the default identity of my default account either.

This occurs with mail from any listserver that insert its own address in the 'To:' header, and mail sent 'To: undisclosed-recipients:;'.

In these cases, TB determines the correct address to which the email was sent (I assume from the 'Received:' headers) because the 'X-Account-Key:' header contains the correct account.

Since 31.1.2 it appears that when TB cannot determine the identity from the 'To:' or any 'Delivered-to:' headers, it ignores any 'X-Account-Key:' or 'Envelope-to:' headers AND the default account/identity setting and picks an identity apparently at random.  It is not 'id1' nor is it an identity associated with 'account1'. (I have no idea what the comment "// Still no matches? Give up and pick the first one" means in the code.)

Is there anything I can do to force TB to use the 'X-Account-Key:' or 'Envelope-to:' headers if present?  I haven't gotten a 'Delivered-to:' header since 2012.  Neither it nor the 'Envelope-to:' headers are held in high regard, so why not use the 'X-Account-Key:' information (if present) as a hint for the identity if you have nothing but "pick the first"?

This is obviously a messy area (judging by Comment 18 if nothing else) but the wholesale elimination of 'X-Account-Key:' as a determinant of the proper reply 'From:' identity has already broken a common email occurrence.  Any way to un-screw this pooch?
Some simple variant of this (along bug 903390) has just bitten me and this SUCKs big time. Can result in MASSIVE privacy violation when TB selects unexpected identity and user doesn't notice before sending.
Depends on: 1051589
Depends on: 878806
Depends on: 664947
Duplicate of this bug: 497560
more potential duplicates
Depends on: 1125978, 1173279
You need to log in before you can comment on or make changes to this bug.