397975 - Identity incorrectly picked based on matching domain without matching username (in identity guessing, a@b.c.d matches x@b.c.d and bc@x.y matches abc@x.y.z, so irrerevant identity is picked up)

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect)

Description

[S. Kerman]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3
Build Identifier: 1.5.0.10 (20070221)

I have 2 identities defined: [e-mail] and [e-mail].  My default it JPS.  If I attempt to reply to a message in which there is a gmail address among the other recipient of the message I am replying to, the my outgoing message will contain my gmail address even though I have not selected it.  This of course causes replies to listserves to be rejected.  I must remember to inspect the addressees list and manually change my identity back to JPS to prevent rejection of my message if any of the other addressees have gmail addresses.

Reproducible: Always

Steps to Reproduce:
1. described above
2.
3.
Actual Results:  
obvious to anyone who has read the above.

Expected Results:  
obvious to anyone who has read the above.

used my default JPS address unless I manually selected the gmail address.

Comment 1

[S. Kerman]

sorry for the typos!  You understand anyway...

Comment 2

[Magnus Melin [:mkmelin]]

Have you tried tb2.0? If it doesn't work there, attach a sample of such a mail (as .eml).

Comment 3

[1ac7b2edaa08e4edd3334c5dc4b966af]

So you don't want to reply using the same address as sender as the original mail was sent to?

Comment 4

[S. Kerman]

Apparently, I did not describe the problem clearly enough.  Here is the scenario. 

A message is sent to my JPS address.  In the specific problem case, it's a message distributed by a listserv.  One of the CCs in the received message by random chance happens to have a gmail address.  The presence of a gmail CC addressee in the message header causes Thunderbird to improperly select my own gmail address as the FROM address in my reply even though my default is the JPS address and I have not chosen to reply using the gmail address.  

Since my reply to the message is going to a listserve which has me registered using the JPS address, if the gmail address is used in my reply, something I would have to watch out for and over-ride to prevent, my reply to the list will bounce since my gmail address is not registered as a list member.

In the past, sometimes I received mail at my JPS address, replied and did not notice a reply back until days later when I checked my gmail.  At that point I discovered that even though I had replied from JPS and not chosen the gmail address as the outgoing address, my reply went out with the gmail address causing the next reply sent TO me to go to gmail, something I did not expect.  I did not understand why people I replied to from JPS were replying back to me at gmail until I noticed this aberrant behavior in Thunderbird, which was inserting the wrong OG address in my replies.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Dup of Bug 377998, isn't it?

Comment 6

[S. Kerman]

No, it is not.  377998 problem description reads as follows:

==== START OF QUOTE =====
"when you try to reply to an email, the recipient address proposed in the
compose window is wrong.

Reproducible: Always

Steps to Reproduce:
1. select an incoming email
2. click reply or reply to all button
3. 
Actual Results:  
the recipient address proposed is the recipient of the original email, that is
my own email address"
===== END OF QUOTE =====

The problem I am reporting is that MY alternate ID, which is my GMAIL address, is included in my outgoing message as the return address even though I have not selected it.  This occurs if there is a GMAIL addressee in the CC list.  The result is that under the specific conditions I described, when I receive a message at JPS and reply from JPS, the next reply back to me goes to GMAIL instead of JPS because my outgoing message inadvertently contains the GMAIL address instead of the JPS address as the FROM address.  My reply however goes to the person I am responding to,as it should, which is not the case for 377998.  So whatever the underlying cause, the symptoms are entirely different.

Comment 7

[WADA:World Anti-bad-Duping Agency]

Are you talking about problem/phenomenon of Bug 327713 ? 

Comment 8

[S. Kerman]

No.  And this time I'm not going to explain why not.  The problem description is plainly different for anyone to read.  I'm not going to spend my time playing silly cat and mouse games.

Comment 9

[Magnus Melin [:mkmelin]]

Please attach a sample mail, as asked in comment 2.

Comment 10

[jenweb]

I believe I've experienced the same bug. I too manage multiple identities in Thunderbird and, for me, what happens is that if I try to respond to any e-mail that's already in one of the folders of my default account, the outgoing address for the message isn't the right one -- it uses one of my other account e-mail addresses as the outgoing address rather than the address of the default account.

However, if I reply to any message in the default account inbox, the proper address is used.

I've tried playing around with all my account settings - including setting the "reply-to" address for the default account -- but that makes no difference. As soon as I try to reply to any message in a folder within my default account, it chooses to try to send it from one of my other accounts.

For me, there doesn't appear to be any difference as to whether I'm replying to a message I received as the "To" recipient or as a "cc" recipient.

This is extremely frustrating -- I very specifically use certain e-mail accounts for different purposes and have had the same trouble with listserves as S. Kerman who posted the bug report originally.

So, when I simply reply to a message -- forgetting this bug and assuming the e-mail will be sent via the same account to which the original message was sent -- I end up with problems. 

So, when responding to anything in my default account's folders, I have to check which account each e-mail is being sent from --and manually change it so that my default account address is used. 

Thunderbird never used to do this!!

(In case it helps, I'm a Mac user and am using Thunderbird 2.0.0.9.

If anyone could fix this it would be much appreciated.

Comment 11

[Wayne Mery (:wsmwk)]

does bug 404393 sound similar to you?

Comment 12

[Jay Rossiter]

I am experiencing this exact bug, in the exact scenario.

Replying to a listserv which was sent to my default account (IMAP).  A sub-identity of that account is my gmail address.  Even though the "Delivered-To:" headers in the message clearly use my primary account's email, it will reply using my gmail address.

This only seems to happen if the To: or CC: address is another gmail user (the listserv is CCed), even though the To/CC: address is not mine.

Comment 13

[Dave Miller [:justdave]]

What the reporter is implying but has failed to actually say in any of his replies is that someone *else's* gmail address (not his) was on the CC list of the message.  And because there was *any* gmail address on the list, it picked his for the reply from, even though it was delivered to his non-gmail account.

I run into this problem constantly with mail sent to mailing lists in the @mozilla.org domain.  Even though the message was delivered to my mozilla.com account, it wants to reply with my mozilla.org address because the To: header on the message had a mozilla.org address in it, even though it wasn't *my* mozilla.org address.

Pretty sure this is a duplicate of one of bug 228980's dependencies, but I'm not sure which one.

Comment 14

[Dave Miller [:justdave]]

Actually, after re-reading those, I'm not sure any of those are actually dupes, so I'm going to confirm this, fix the summary up a little, and make it an additional dependency.

Comment 15

[Ludovic Hirlimann [:Usul]]

(In reply to comment #13)
> I run into this problem constantly with mail sent to mailing lists in the
> @mozilla.org domain.  Even though the message was delivered to my mozilla.com
> account, it wants to reply with my mozilla.org address because the To: header
> on the message had a mozilla.org address in it, even though it wasn't *my*
> mozilla.org address.

This sounds like bug 585134.

Comment 16

[Jay Rossiter]

This doesn't sound like bug 585134 at all, to me.

585134 basically says that since TB has no information to choose from it simply uses the default identity instead of an alias of that identity.

In this case, TB is choosing to match domains with the incoming user based on the assumption that you're in the same organization.  In actuality, it has plenty of information to choose the correct response address based on To, CC, Delivered-To, etc.

Comment 17

[Magnus Melin [:mkmelin]]

Looking at the code it's pretty clear why this happens
http://mxr.mozilla.org/comm-central/source/mail/base/content/mailCommands.js#74

One option would be to simply remove that part... or check that all emails in optionalHint have the same domain and only do it for that case.

Comment 19

[bugzilla_reg]

I posted the above mentioned duplicate: 699432. I searched but did not find this.

My experience is the same as the OP but in my case it only occurs when using Reply or Reply All from the global Sent folder. 
Also, I use multiple accounts however I'm not sure that's the same as multiple identities. When I open Manage Identities I only have the one account showing in each.

==================================

Here is my duplicate report.

Steps to reproduce:

Selected an email in the global Sent folder and clicked Reply All. This bug occurs only when the selected email has at least one recipient email address on the Gmail, Yahoo or Hotmail domains. 

Why would I Reply All to a message in the Sent folder that I myself sent?
In order to quickly compose an email with all the same recipients. I just have to remove myself from the list.


Actual results:

The From field takes the address of my Gmail account if one of the recipients is also on Gmail. Likewise for Hotmail and Yahoo.

I have accounts in TB on Gmail, Yahoo and Hotmail, however they are not my default account and they are not any of the recipients in the email. In fact they appear nowhere in the message source. There's no reason why these accounts should be used as the from address when replying to messages sent from my default account.


Expected results:

The From address should have been my default account address.

Comment 20

[bugzilla_reg]

(In reply to bugzilla_reg from comment #19)
Just to add, this is with TB 7.0.1 on Win XP SP3.

Comment 21

[Magnus Melin [:mkmelin]]

Attachment: proposed fix

I'd just remove this check. For most company-internal lists i'd think the correct account is likely selected correctly anyway, based on the receiving account. If that account has multiple identities, wouldn't those normally have same domain, making the existing check do nothing? Besides, those lists are primarily used for memos and other one-way communication, at least what i've seen myself.

The other changes are more cleanup-type. We missed the subaddress check for the delivered-to check, and i also added delivered-to as a last hint, in case it's there.

The messageGenerator.js changes let's you set a raw address header yourself if you prefer, which i find reasonable.

Comment 22

[Magnus Melin [:mkmelin]]

The patch also fixes bug 228980, since with the patch only "full" matches would be considered.

Comment 23

[Magnus Melin [:mkmelin]]

(Or at least the initial complaint there.)

Comment 24

[bugzilla_reg]

I'd like to reiterate my earlier point that I have a very similar problem to the original report but I do NOT have separate identies. I accepted the marking of my own report, 699432, as a duplicate because of the similarity but given all the discussion is centred on identities I'm now unsure it was a duplicate.

I'm going to try to describe it one more time with examples because I don't sense this has been properly understood. Apologies if I'm wrong but I've just installed 9.0.1 and the problem is still there.

The problem:

My default account is on my private domain, [e-mail]. I also have accounts [e-mail], [e-mail] and [e-mail] set up in Thunderbird.

I send an email from [e-mail] to a bunch of people: [e-mail], [e-mail], [e-mail].

That email ends up in the Local Sent folder. A while later, before anyone has replied, I want to add another point to that email. A quick way to compose this follow-up is to Reply All to my own mail in the Sent folder because it fills in all the same recipients. It's not perfect because it copies me in too but removing that is quicker than adding the recipients again one by one.

Now, I sent the original mail from [e-mail] and this is my default account. There's no question that this address should therefore be the From address of the new mail I'm composing. However, because one of the recipients is [e-mail], TB uses MY gmail account [e-mail] in the From field.

Please let me know if you consider this to be a different issue to that in question here. If so I will try to get my orginal report, 699432, reopened.
Thanks.

Comment 25

[Jim Porter (:squib)]

(In reply to Magnus Melin from comment #21)
> Created attachment 588141 [details] [diff] [review]
> proposed fix

If we're changing the behavior of these functions, I expect we'll get out-of-sync with the identity selection when you just click an email address inside a message body (see bug 64267). It would be nice to have the identity-selection code callable from C++ so that we don't have to worry about this stuff getting out of sync now or in the future.

Comment 26

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 588141 [details] [diff] [review]
proposed fix

Review of attachment 588141 [details] [diff] [review]:
-----------------------------------------------------------------

So, I like it.  There are some comments below, but I think you can fix them up well enough.  :)  r=me with the comments addressed.

Thanks,
Blake.

::: mail/base/content/mailCommands.js
@@ -43,3 +48,3 @@
> >  function getBestIdentity(identities, optionalHint)
> >  {
> > -  var identity = null;
> > +  let identityCount = identities.Count();

Do we need to check for a null "identities" parameter, or will the developer get what they deserve in that case?  :)

::: mail/test/mozmill/message-header/test-reply-identity.js
@@ +1,4 @@
> +/* ***** BEGIN LICENSE BLOCK *****
> + * Version: MPL 1.1/GPL 2.0/LGPL 2.1
> + *
> + * The contents of this file are subject to the Mozilla Public License Version

Can we use the much-shorter MPL 2 header from http://www.mozilla.org/MPL/headers/  instead?  ;)

@@ +37,5 @@
> +/**
> + * Tests that actions such as replying choses the most suitable identity.
> + */
> +
> +// make SOLO_TEST=message-header/test-reply-identity.js mozmill-one

I don't think we really need this comment.  (Also, I'm not entirely sure it'll work on Windows, due to / vs. \ issues... ;)

@@ +145,5 @@
> +  assert_selected_and_displayed(mc, msg);
> +
> +  let replyWin = composeHelper.open_compose_with_reply();
> +  // Should have selected the default identity.
> +  checkReply(replyWin, identity1Email);

I think we should close the reply window after each test, so that they don't all pile up.  (But that's mostly a preference, I wouldn't block the review on it.)

::: mailnews/test/resources/messageGenerator.js
@@ -477,2 +478,4 @@
> >     */
> >    set from(aNameAndAddress) {
> > +    if (typeof aNameAndAddress === "string") {
> > +      let name = aNameAndAddress.replace(/<(.+@.+)>/, "").trim();

I seem to remember the email address parsing spec as being a little more complicated than this.  Should we mention in the docstring that we only accept "Name Here <email@here>"-formatted header values, instead of the full crazy email address spec?

@@ -515,0 +525,5 @@
> > +    if (typeof aNameAndAddresses === "string") {
> > +      this._to = [];
> > +      let people = aNameAndAddresses.split(",");
> > +      for (let i = 0; i < people.length; i++) {
> > +        let name = people[i].replace(/<(.+@.+)>/, "").trim();

Since we have this in more than one place, I would prefer to make a utility function that we can call from other places (and potentially make improvements to, too).

Comment 27

[Magnus Melin [:mkmelin]]

(In reply to Jim Porter (:squib) from comment #25)
> (In reply to Magnus Melin from comment #21)
> > Created attachment 588141 [details] [diff] [review]
> > proposed fix
> 
> If we're changing the behavior of these functions, I expect we'll get
> out-of-sync with the identity selection when you just click an email address
> inside a message body (see bug 64267). It would be nice to have the
> identity-selection code callable from C++ so that we don't have to worry
> about this stuff getting out of sync now or in the future.

Agreed, but that's more work than i'm willing to do at the moment.

Comment 28

[Magnus Melin [:mkmelin]]

(In reply to Blake Winton (:bwinton - Thunderbird UX) from comment #26)
> Do we need to check for a null "identities" parameter, or will the developer
> get what they deserve in that case?  :)

Yes, i find it much easer to code if the code isn't trying to hide everything wrong you do.

> I seem to remember the email address parsing spec as being a little more
> complicated than this.  Should we mention in the docstring that we only
> accept "Name Here <email@here>"-formatted header values, instead of the full
> crazy email address spec?

I think what i have is more or less it, actually (as long as we're ascii). Made it a function and added documentation.

Comment 29

[Magnus Melin [:mkmelin]]

Attachment: proposed fix v2

Carrying fwd r=bwinton

Comment 30

[David :Bienvenu]

Comment on attachment 590310 [details] [diff] [review]
proposed fix v2

Review of attachment 590310 [details] [diff] [review]:
-----------------------------------------------------------------

thx, Magnus!

Comment 31

[Magnus Melin [:mkmelin]]

->FIXED
http://hg.mozilla.org/comm-central/rev/4f4b9352555e

Comment 33

[Ben Bucksch (:BenB)]

Magnus, does the fix here fix the issue originally raised in bug 228980?

(Please note that the bug since has been morphed by others in to a meta-bug, so please don't close it.)

Comment 34

[Magnus Melin [:mkmelin]]

Yes, see comment 22.