Closed Bug 804042 Opened 8 years ago Closed 4 months ago

Case sensitivity for email address triggers invalid reply-to-self reply

Categories

(Thunderbird :: Message Compose Window, defect)

10 Branch
defect
Not set
minor

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 78.0

People

(Reporter: alex, Assigned: mkmelin)

Details

Attachments

(1 file)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.14 (KHTML, like Gecko) Version/6.0.1 Safari/536.26.14

Steps to reproduce:

My email environment is a little unusual. I have one IMAP email account per person, and many common IMAP email accounts as shared folders. This is all working fine.

This problem is a little convoluted, so I will try and explain myself as clearly as possible.

Lets say I have a staff member at email address john.smith@company.com and a shared IMAP folder called team@company.com.

I have found this issue is triggered when I forward an email from team@company.com to John.Smith@company.com AND the case of the email address in the email (John.Smith@company.com) does not match the case of the email address in the identity (john.smith@company.com). 

If they don't match, when you look at the email, it doesn't have the word "Me" next to the recipient email address (John.Smith@company.com), but if they do match, the word "Me" appears next to the recipient (john.smith@company.com).

Ok, so far the problem doesn't appear too bad, but the problem manifests itself when you try to reply to that email.


Actual results:

Case #1
If we reply to the email (From: team@company.com, To: John.Smith@company.com), we end up with a compose window as expected, but the envelope is wrong (From: john.smith@company.com, To: John.Smith@company.com)

Case #2
If we reply to the email (From: team@company.com, To: john.smith@company.com), we end up with a compose window as expected, and a correct envelope (From: john.smith@company.com, To: team@company.com)

As per the RFC, local part of the email must be treated as case sensitive so John.Smith@company.com != john.smith@company.com, but in the real world nobody requires that and the local part is commonly case insensitive so John.Smith@company.com == john.smith@company.com.




Expected results:

Case #1 should have the same behaviour as Case #2. I believe there is a bug in the code to determine if an email address is "Me" or someone else, and this is always case sensitive, where is should have the option to be case insensitive.

This might be related to bug 377998, and reply-to-self email logic.
OS: Mac OS X → Windows 7
But as you said, TB is behaving correctly according to spec.

If you wish an option for it to behave case insensitively, then that can be an enhancement request.
Severity: normal → enhancement
Status: UNCONFIRMED → NEW
Component: Untriaged → Message Compose Window
Ever confirmed: true
OS: Windows 7 → All
Hardware: x86 → All
The RFC addresses the fact that while the mail server must preserve the case of the local part of the email address, for reasons of interoperability they should be treated as case insensitive.

---

Here is an extract from rfc5321 (http://tools.ietf.org/html/rfc5321).

The local-part of a mailbox MUST BE treated as case sensitive. Therefore, SMTP implementations MUST take care to preserve the case of mailbox local-parts.  In particular, for some hosts, the user "smith" is different from the user "Smith".  However, exploiting the case sensitivity of mailbox local-parts impedes interoperability and is discouraged.  Mailbox domains follow normal DNS rules and are hence not case sensitive.
Assignee: nobody → mkmelin+mozilla
(In reply to Alex Ferrara from comment #0)
> From: john.smith@company.com, To: John.Smith@company.com)

Difference in case of local part is;
  From: in reply mail is set from your identity definition.
    To: in reply mail is set from To: header or From: header of replied mail.
If you define John.Smith@company.com as identity with smaller identity.idP than identity.idQ for john.smith@company.com, Tb perhaps uses identity.idP for From: of reply/forward/compose mail.
- Add one more identity for relevant account.
    mail.account.accountN.identities = idP,idQ
    if idQ<idP, swap them => mail.account.accountN.identities = idQ,idP
    Restart Tb.
- Set mail address for identities.
  First  identity : John.Smith@company.com
  Second identity : john.smith@company.com

> Actual results:
> Case #1
> If we reply to the email (From: team@company.com, To: John.Smith@company.com),
> we end up with a compose window as expected,
> but the envelope is wrong (From: john.smith@company.com, To: John.Smith@company.com)
> Case #2
> If we reply to the email (From: team@company.com, To: john.smith@company.com),
> we end up with a compose window as expected,
> and a correct envelope (From: john.smith@company.com, To: team@company.com)

What is difference between "we end up with a compose window as expected" and "envelope is wrong(or correct)?
What do you call by "envelope" in this context?
Does your "wrong" mean "mail address in From:/To: of message source generated by Tb" != mail address requested to SMTP server via FROM:/RCPT: command?

In Reply-To-Self, "Mail address in message header of a mail is Self or not-Self"
  == "mail address is defined as identity in Tb or not".
Identity check is done by statement like "for(ID inidentity_list)".
I don't know the "identity_list" is ordered id number of identity.idX or by mail.account.accountN.identities.

Identity check may be case insensitive currently.
I think it's already case insensitive as domainpart is case insensitive. However, because "quoted local part" should be case sensitive and case should be kept", comparison of localpart may be case sensitive.
Case #1 looks for me "John.Smith@company.com != identity defined in Tb".
(See bug 933555 for Identity check relevant source code, please)

Is defferent result obtained by next;
- mail.account.accountN.identities = idP,idQ (idP < idQ)
  - idP=john.smith@company.com, idQ=John.Smith@company.com
  - idP=John.Smith@company.com, idQ=john.smith@company.com

If team@company.com is also defined as identity in Tb, check difference of following, please.
- mail.account.accountN.identities = idP,idQ,idX,idY (idP < idQ < idX < idY)
  - idX=team@company.com, idY=Team@company.com
  - idX=Team@company.com, idY=team@company.com

Note:
If From: mail address == To: mail address, bug 933555 is exposed. Please don't test From:==To: case until that bug will be fixed.
I think this bug is already understood (though i didn't try to reproduce). This check (and the one earlier) should be made case insensitive for any real world scenarios: http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#2547
(In reply to Magnus Melin from comment #4)
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsMsgCompose.cpp#2547
Thanks for pointing. I can't know curIdentityEmail.Equals/toEmailAddresses.Contains is case sensitive or not, without your comment on it... :-)
Reason why I requested test of identity setting change is;
- Find a simple/easy workaround in case #1
- My understanding is;
    "From: of Reply mail becomes john.smith@company.com or John.Smith@company.com"
    depends on order of identity definition when isReplyToSelf==true.
    This is true even when case insensitive comparison is done. 
  If it's correct, From:john.smith@company.com for mail of From:John.Smith@company.com
  or vice versa can happen. I think it's not uncomfortable for user.
  If local part is "quoted string" and if user defines both upper case version and lower 
  case version as identity, and if user uses both for Reply-To-Self mail,
  it's RFC violation by Tb for user, because case in quoted local part is not retained :-)

  Magnus, this won't occur or this can't be a problem?
  When isReplyToSelf=true, From: mail address of reply mail is set from From: header in
  replied mail instead of identity for the From: mail address in replied mail?
Third reason.
If your "This check should be made case insensitive for any real world scenarios" is true, why this bug can be Enhancement request.
Another question.
Case of quoted local part should be retained. If case insensitive check, RFC violatio by Tb can occur. What should be done in quoted local part processing?
(In reply to WADA from comment #5)
>   Magnus, this won't occur or this can't be a problem?
>   When isReplyToSelf=true, From: mail address of reply mail is set from
> From: header in
>   replied mail instead of identity for the From: mail address in replied
> mail?

It always uses the From settings the self identity has set. 

I think there is consensus having the same address going to different people depending on case sensitivity is just nonsense - even if the RFC says the local part is case sensitive. This bug would be the least of the problems you would face in real use. We should just create software that works with real situations and real people, so I think it's very much ok to treat it case insensitively at least locally.
Severity: enhancement → minor
(In reply to Magnus Melin from comment #8)
I see. I can process complaint of "From:A@x.y.z was set by Tb even though From: is a@x.y.z!!!" by user in following case.
(A) non-quoted@x.y.z :   a@x.y.z and   A@x.y.z is always same mail box
(B) quoted@x.y.z     : "a"@x.y.z and "A"@x.y.z is same mail box
Note:
In these cases, if user wants to generate both From:a@x.y.z and From:A@x.y.z using single Tb only, user has to create two identities in the Tb, regardless of case sensitibity in comparison by Tb. So, this bug can't occur.
(I have identity#1=M.Wada@x and identity#2=m.wada@x, and choose #1 on sunny day, #2 on rainy day :-)
This bug can occur only when identity=a@x.y.z only is defined in Tb and other mail client, including Tb on other PC, sent mail with From:A@x.y.z.

How about following case?
(C) quoted@x.y.z : "a"@x.y.z and "A"@x.y.z are different mail boxes.
This is not real situation even if RFC permits it, so there is no need to care?

I don't think such mail service provider actually exists, and I believe, if intra-net mail server, even if quoted local part is needed, "First.Last-US"@... and "First.Last-JP"@... like naming is used when two same name persons are hired.
However, user freely configure server if personal use or limited use. Following is actual request.
     POP3 account : userName=uu on hostname=s.s.s / port=A
  != POP3 account : userName=uu on hostname=s.s.s / port=B(!=A) in his environment.
  So Tb should support such account creation.
Frankly I don't care about quoted addresses. I've never ever seen one used in real life (even in bugzilla!). I don't know if we actually handle them elsewhere either.
(In reply to Magnus Melin from comment #10)
> I've never ever seen one used in real life (even in bugzilla!).
Bug 941861 is an example of "issue around quoted local part handling in Tb, which actually occurred in real world", although that bug is for "space in quoted local part", not for "case sensitivity in local part".

> Frankly I don't care about quoted addresses. I don't know if we actually handle them elsewhere either.
I see. I can manage user's complaints on next case, even if such user actually exist. 
> (C) quoted@x.y.z : "a"@x.y.z and "A"@x.y.z are different mail boxes.
Simplest/easiest/surest solution in such case is always "stop such definition".
And, "case insensitive comparison of identity in Tb" is sufficient in both following cases.
> (A) non-quoted@x.y.z :   a@x.y.z and   A@x.y.z is always same mail box
> (B) quoted@x.y.z     : "a"@x.y.z and "A"@x.y.z is same mail box
Status: NEW → ASSIGNED
Comment on attachment 9147225 [details] [diff] [review]
bug804042_reply_self_case.patch

Review of attachment 9147225 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM!

::: mailnews/compose/src/nsMsgCompose.cpp
@@ +2480,5 @@
>  
>        bool isReplyToSelf = false;
>        nsCOMPtr<nsIMsgIdentity> selfIdentity;
>        if (!identities.IsEmpty()) {
> +        nsTArray<nsCString> toEmailAddressesLower;

I'm pretty sure you can set the capacity (as opposed to length) of an nsTArray in it's ctor,
eg:
nsTArray<nsCString> toEmailAddressesLower(toEmailAddresses.Length());

@@ +2485,5 @@
> +        for (auto email : toEmailAddresses) {
> +          ToLowerCase(email);
> +          toEmailAddressesLower.AppendElement(email);
> +        }
> +        nsTArray<nsCString> ccEmailAddressesLower;

nsTArray<nsCString> ccEmailAddressesLower(ccEmailAddresses.Length());
Attachment #9147225 - Flags: review?(benc) → review+

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/ec00afab1f69
make sure reply-to-self works case insensitively. r=benc DONTBUILD CLOSED TREE

Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 78.0
You need to log in before you can comment on or make changes to this bug.