Stubborn recipient autocomplete silently swaps recipients: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com)

NEW
Unassigned

Status

defect
P2
critical
5 years ago
2 years ago

People

(Reporter: bugzilla2007, Unassigned)

Tracking

(Depends on 2 bugs, Blocks 1 bug, 7 keywords)

Dependency tree / graph

Firefox Tracking Flags

(firefox38-, thunderbird38 affected, thunderbird_esr31 affected, thunderbird_esr38? affected)

Details

(Whiteboard: [ux-wysiwyg][Better STR comment 1][possible solution(?): bug 533109][workaround: add comma after new recipient])

+++ This bug was initially created as a clone of Bug #131692 +++

This is essentially Bug 131692, but that bug isn't in a good state, and this will now happen more frequently on TB31, so I'd like to roll this out cleanly for better understanding.

STR

1) have john.doe@asdf.com in your AB
2) try to compose a msg to new recipient doe@asdf.com (not yet in your AB; with recipient autocomplete enabled)

Actual result

Cannot compose a msg to perfectly valid and non-special email address, doe@asdf.com, in spite of typing out the address in full(!)
(The only way to do it is to open Address Book, create a new contact there, then go back to the composition and retrieve the contact from AB.)

Expected result

TB must let me compose msg to any valid recipient, regardless if that recipient is already saved in my AB or not.

Note:

This age-old bug is now MUCH more exposed after recent Bug 529584 as we now helpfully show partial matches for email addresses.
This is NOT an unlikely edgecase scenario at all, if you think about common local parts of email addresses on big mail providers:

     johnson(at)gmail.com
   j.johnson(at)gmail.com
 p.j.johnson(at)gmail.com
    jjohnson(at)gmail.com
   pjjohnson(at)gmail.com
 pauljohnson(at)gmail.com
paul.johnson(at)gmail.com

Possibilities are endless. There's really nothing special about a shorter address being a full partial match of a longer address. All of those shorter addresses will fail if the respective longer address is already in TB's AB and the shorter one isn't.

Depending on scenario and localization, a regular shorter surname might easily be fully contained in a longer surname:

      Bauer(at)gmail.com (full surname)
    gebauer(at)gmail.com (full surname; or: Gerda Bauer; or Gerd Emil Bauer etc.)
 Neugebauer(at)gmail.com (full surname)
    Mueller(at)gmail.com (full surname)
   dmueller(at)gmail.com (David Mueller, Dan Mueller, Doris Mueller etc.)
Waidmueller(at)gmail.com (full surname)

For big mail providers, even making your local part more unique with numbers might not prevent this bug, because other users might have had the same idea:

     johnson01(at)gmail.com
   j.johnson01(at)gmail.com
 p.j.johnson01(at)gmail.com
    jjohnson01(at)gmail.com
   pjjohnson01(at)gmail.com
 pauljohnson01(at)gmail.com
paul.johnson01(at)gmail.com

Depending on your personal datasets, many of these will fail unless they are already in your address book.

This can even make whole ranges of new addresses of a certain domain or TLD fail if that domain happens to be a substring of a longer domain already found in AB:

helpdesk(at)microsoft.co (valid TLD for Colombia)
helpdesk(at)microsoft.co.uk
helpdesk(at)microsoft.com

In fact, for business setups with more than one domain for the same company, it's pretty likely that they might have similar organizational local address parts for each of their domains.

I believe not accepting valid, regular email addresses is pretty bad behaviour for a mailing application, and 13 years after this was first filed as Bug 131692, it should really be fixed.
(In reply to Magnus Melin from Bug 131692 comment #26)
> The solution here is probably to have forcecomplete=false for the
> composition autocomplete widget. We should investigate if that's good once
> the other current autocomplete issues have been fixed...
> 
> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XUL/
> Textbox_%28Toolkit_autocomplete%29#a-forcecomplete
>  "If true, the textbox will be filled in with the best match when it loses
> the focus. If false, it will only be filled in when the user selects an
> item."

Magnus, would forcecomplete=false disable inline-autocompletion?

But I think there might be something wrong at a deeper level in the autocomplete code, because manually correcting the autocomplete inline suggestion has absolutely no effect, which violates ux-error-prevention:

0) Have "john.doe@asdf.com" in AB, but not "doe@asdf.com"; try to compose to "doe@asdf.com"
1) Type "doe" and TB shows
-> doe >> john.doe@asdf.com
2) continue typing: "doe@asdf.com"
-> doe@asdf.com >> john.doe@asdf.com
3) delete unwanted inline autocompletion part " >> john.doe@asdf.com"
-> doe@asdf.com (so now the new address is correctly shown in the input)
4) optionally close autocomplete dropdown with Alt+Cursor up 
-> doe@asdf.com (so now there's no sign of autocompletion in the UI whatsoever)
5) confirm address or move focus away (Enter | tab | click elsewhere etc.)
-> john.doe@asdf.com (start swearing at TB)
Summary: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com) → Stubborn recipient autocomplete: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com)
Whiteboard: [patchlove] → [patchlove][ux-wysiwyg]
Whiteboard: [patchlove][ux-wysiwyg] → [ux-wysiwyg]
Thomas D's last post has bitten me in a big way, and caused sensitive material to be sent to the wrong person. This bug should be elevated to critical.
(In reply to James Rome from comment #2)
> Thomas D's last post has bitten me in a big way, and caused sensitive
> material to be sent to the wrong person. This bug should be elevated to
> critical.

It's not my last post which has bitten James, but the *bug* described in my last post ;)

James correctly points out the critical nature of this bug, which has been insufficiently expressed in Actual Results:

(In reply to Thomas D. from comment #0)
> Actual result
>
> Cannot compose a msg to perfectly valid and non-special email address,
> doe@asdf.com, in spite of typing out the address in full(!)
> (The only way to do it is to open Address Book, create a new contact there,
> then go back to the composition and retrieve the contact from AB.)

The cunning/critical part of this bug is that TB silently swaps the correctly entered shorter (new) address against the longer (existing, unintended) address at a time where it's least expected: *After* confirmation of the valid and correctly displayed address with Enter or such. This can easily go unnoticed by user because we usually only need to check things *before* confirming them with Enter, and the wrong address will be very similar to the correct address. Also, after typing the new address in full and ensuring that there's no visual indication of any auto-completion, again there's no need to for the user to recheck the address in a well-behaved program.

> Expected result
> 
> TB must let me compose msg to any valid recipient, regardless if that
> recipient is already saved in my AB or not.

TB must never silently swap a correctly entered, valid address against a random address from my Address book.
Summary: Stubborn recipient autocomplete: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com) → Stubborn recipient autocomplete silently swaps recipients: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com)
A workaround is to end the subset address with a comma (,) which seems to disappear leaving the correct address in the field.
Maybe, re-enable the condition that, whenever '@' is entered as a part of the search term, we stop autocompletion?
Flags: needinfo?(mkmelin+mozilla)
(In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from comment #5)
> Maybe, re-enable the condition that, whenever '@' is entered as a part of
> the search term, we stop autocompletion?

That doesn't sound like a sustainable solution, more like a workaround. There's a bug here which needs to be examined and fixed. A non-ambiguous input in the recipient input field must never be changed unless somehow confirmed by the user (accepting selected autocomplete suggestion with Enter, Tab, etc.).
There's absolutely no reason why "foo@bar.com" should be swapped with any other address after all the autocomplete stuff is no longer seen in the UI because it was deliberately ignored/deleted/dismissed by user.
(In reply to Thomas D. from comment #6)
> (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> comment #5)
> > Maybe, re-enable the condition that, whenever '@' is entered as a part of
> > the search term, we stop autocompletion?
> 
> That doesn't sound like a sustainable solution, more like a workaround.
No this isn't a workaround. This is one of the ways to fix this.
> There's a bug here which needs to be examined and fixed. A non-ambiguous
> input in the recipient input field must never be changed unless somehow
> confirmed by the user (accepting selected autocomplete suggestion with
> Enter, Tab, etc.).
Ya, so how do you tell a system its an unambiguous input?
If the user enters '@', we know that there is a very high probability that he is trying
to enter a complete email address himself. So, we'll just stop autocompleting it (disabling the force
autocomplete is also a solution, but this one also adds to it). Maybe, we can also add this email to
collected addresses.

> There's absolutely no reason why "foo@bar.com" should be swapped with any
> other address after all the autocomplete stuff is no longer seen in the UI
> because it was deliberately ignored/deleted/dismissed by user.
That's fine, but how do you expect a system to know "foo@bar.com" is done now. For knowing that,
I proposed to test for '@'.

Thanks.
(In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from comment #7)
> (In reply to Thomas D. from comment #6)
> > (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> > comment #5)
> > > Maybe, re-enable the condition that, whenever '@' is entered as a part of
> > > the search term, we stop autocompletion?
> > 
> > That doesn't sound like a sustainable solution, more like a workaround.
> No this isn't a workaround. This is one of the ways to fix this.

Why was the condition disabled in the first place? We probably had reasons for that. Here are some:

Imo it's not a good idea at all to stop autocompletion after the first @ is entered. Lots of legitimate scenarios where users wish autocompletion would be needlessly disabled, e.g. 

1) multiple same/similar local parts with different domain
same person with different email addresses
john.doe@foo.com
john.doe@bar.com
john.doe@baz.com
same company with different local domains
admin@mozfoofoofoo.de
admin@mozfoofoofoo.fr
admin@mozfoofoofoo.com

2) addresses having @ in the display name (unless special-cased)

> > There's a bug here which needs to be examined and fixed. A non-ambiguous
> > input in the recipient input field must never be changed unless somehow
> > confirmed by the user (accepting selected autocomplete suggestion with
> > Enter, Tab, etc.).
> Ya, so how do you tell a system its an unambiguous input?
> If the user enters '@', we know that there is a very high probability that
> he is trying
> to enter a complete email address himself. So, we'll just stop
> autocompleting it (disabling the force
> autocomplete is also a solution, but this one also adds to it). Maybe, we
> can also add this email to
> collected addresses.

No, adding to collected addresses happens when sending, no need to complicate here.

> > There's absolutely no reason why "foo@bar.com" should be swapped with any
> > other address after all the autocomplete stuff is no longer seen in the UI
> > because it was deliberately ignored/deleted/dismissed by user.
> That's fine, but how do you expect a system to know "foo@bar.com" is done
> now?

Huh? There's an input field which after my input looks exactly like this:
[foo@bar.com         ] (new address not yet in AB)
(Note that there's no visual trace of autocomplete suggestions any more, neither in the input, nor below!).
I press ENTER on that input. So the system knows exactly that and when I'm done, and what I want. Really, no ambiguities here.
The UX of this is currently wrong, probably even for force-complete, or otherwise force-complete is not compatible with the UX needs here.

Can somebody enlighten me if setting force-complete to false will also remove the inline completion?
(In reply to Thomas D. from comment #8)
> (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> comment #7)
> > (In reply to Thomas D. from comment #6)
> > > (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> > > comment #5)
> > > > Maybe, re-enable the condition that, whenever '@' is entered as a part of
> > > > the search term, we stop autocompletion?
> > > 
> > > That doesn't sound like a sustainable solution, more like a workaround.
> > No this isn't a workaround. This is one of the ways to fix this.
> 
> Why was the condition disabled in the first place? We probably had reasons
> for that. Here are some:
> 
> Imo it's not a good idea at all to stop autocompletion after the first @ is
> entered. Lots of legitimate scenarios where users wish autocompletion would
> be needlessly disabled, e.g. 
> 
> 1) multiple same/similar local parts with different domain
> same person with different email addresses
> john.doe@foo.com
> john.doe@bar.com
> john.doe@baz.com
> same company with different local domains
> admin@mozfoofoofoo.de
> admin@mozfoofoofoo.fr
> admin@mozfoofoofoo.com
> 
> 2) addresses having @ in the display name (unless special-cased)
> 
> > > There's a bug here which needs to be examined and fixed. A non-ambiguous
> > > input in the recipient input field must never be changed unless somehow
> > > confirmed by the user (accepting selected autocomplete suggestion with
> > > Enter, Tab, etc.).
> > Ya, so how do you tell a system its an unambiguous input?
> > If the user enters '@', we know that there is a very high probability that
> > he is trying
> > to enter a complete email address himself. So, we'll just stop
> > autocompleting it (disabling the force
> > autocomplete is also a solution, but this one also adds to it). Maybe, we
> > can also add this email to
> > collected addresses.
> 
> No, adding to collected addresses happens when sending, no need to
> complicate here.
> 
> > > There's absolutely no reason why "foo@bar.com" should be swapped with any
> > > other address after all the autocomplete stuff is no longer seen in the UI
> > > because it was deliberately ignored/deleted/dismissed by user.
> > That's fine, but how do you expect a system to know "foo@bar.com" is done
> > now?
> 
> Huh? There's an input field which after my input looks exactly like this:
> [foo@bar.com         ] (new address not yet in AB)
> (Note that there's no visual trace of autocomplete suggestions any more,
> neither in the input, nor below!).
Really? If you have this "foo@bar.com" with no autocomplete suggestions, what is this bug about then?

> I press ENTER on that input. So the system knows exactly that and when I'm
> done, and what I want. Really, no ambiguities here.
> The UX of this is currently wrong, probably even for force-complete, or
> otherwise force-complete is not compatible with the UX needs here.
> 
> Can somebody enlighten me if setting force-complete to false will also
> remove the inline completion?
Maybe I forgot to enlighten you. force-complete to false will just not complete the suggestion unless you explicitly choose to do so. And will just leave your input (whatever is entered) there.

See here: http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/components/autocomplete/nsIAutoCompleteInput.idl#50
(In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from comment #9)
> > Huh? There's an input field which after my input looks exactly like this:
> > [foo@bar.com         ] (new address not yet in AB)
> > (Note that there's no visual trace of autocomplete suggestions any more,
> > neither in the input, nor below!).
> Really? If you have this "foo@bar.com" with no autocomplete suggestions,
> what is this bug about then?

Really! :) Pls use detailed STR of comment 1. Use backspace to delete the " >> suggestion1" part, then (optionally, for the sake of demonstration), use Alt+Cursor up to close the results dropdown. Now there's nothing but a plain email address foo@bar.com in the input box. Press Enter, curse as TB swaps this against the first address from autocomplete which is no longer visible on screen at all and has been actively dismissed.

> > I press ENTER on that input. So the system knows exactly that and when I'm
> > done, and what I want. Really, no ambiguities here.
> > The UX of this is currently wrong, probably even for force-complete, or
> > otherwise force-complete is not compatible with the UX needs here.
> > 
> > Can somebody enlighten me if setting force-complete to false will also
> > remove the inline completion?
> Maybe I forgot to enlighten you. force-complete to false will just not
> complete the suggestion unless you explicitly choose to do so. And will just
> leave your input (whatever is entered) there.
> 
> See here:
> http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/components/
> autocomplete/nsIAutoCompleteInput.idl#50

46  /* 
47    * Option for completing to the default result whenever the user hits
48    * enter or the textbox loses focus
49    */

Right, so this makes me think that this bug might want this:
When user has explicitly dismissed the inline suggestion " >> suggestion1" by backspacing it away, the default result must now be the actual user input, not the first suggestion.

If that violates the idea of force-complete (perhaps to not enter any custom values), then we must switch off force-complete. If switching off force-complete also prevents inline-completion, then I think that's another bug. Inline completion (after confirmation) can be useful even without force-complete to a closed set of values.
Whiteboard: [ux-wysiwyg] → [ux-wysiwyg][Better STR comment 1]
forceComplete is absolutely needed in thunderbird - that's what makes the address complete to the correct capitalization, not leave >> and other "non-completed stuff" in the address field
Flags: needinfo?(mkmelin+mozilla)
(In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from comment #5)
> Maybe, re-enable the condition that, whenever '@' is entered as a part of
> the search term, we stop autocompletion?

I don't think that's likely a good idea. 
We should just not force if delete was the last thing you pressed.
(In reply to Magnus Melin from comment #12)
> (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> comment #5)
> > Maybe, re-enable the condition that, whenever '@' is entered as a part of
> > the search term, we stop autocompletion?
> 
> I don't think that's likely a good idea. 
> We should just not force if delete was the last thing you pressed.

Okay, that can be tried. I'll try a patch by tomorrow.
If someone gets to this before that, great!

Thanks.
Duplicate of this bug: 1148906
[Tracking Requested - why for this release]:

Kent, I believe that 10 years after TB was born, it's about time that we allow users to compose to valid, normal new email addresses even if they happen to resemble longer email addresses which are already in AB...

In the same line as bug 1152517, autocomplete must NEVER mess around with user's explicitly and correctly entered recipients.

Starting from TB31, this is now much more exposed because we helpfully use "contains" matching instead of "startsWith" on email addresses (Bug 529584), so that searching for "Doe" will find "john.doe@foo.bar".

What do you think?
Whiteboard: [ux-wysiwyg][Better STR comment 1] → [ux-wysiwyg][Better STR comment 1][possible solution(?): bug 533109]
This bug has at least a dozen of full and partial duplicates which however are still spread around between bugs. E.g. on Bug 131692 alone (which is essentially an untidy old version of this), there are 6 dupes, coming in regularly over time, including recent reports.
This is a SECURITY hazard as well. It caused me to send proprietary information to the wrong person. It needs a higher priority.
This is not a Firefox bug. Not tracking.
(In reply to Thomas D. from comment #16)
> This bug has at least a dozen of full and partial duplicates which however
> are still spread around between bugs. E.g. on Bug 131692 alone (which is
> essentially an untidy old version of this), there are 6 dupes, coming in
> regularly over time, including recent reports.

Nobody disputes that this is a long-standing bug that needs fixing, and if you can motivate someone to get it fixed, we'll try to get it into tb-38. But at this late date, tracking is meant more for blockers, and we are not going to block tb-38 on this bug.
(In reply to Kent James (:rkent) from comment #19)

> Nobody disputes that this is a long-standing bug that needs fixing, and if
> you can motivate someone to get it fixed, we'll try to get it into tb-38.
> But at this late date, tracking is meant more for blockers, and we are not
> going to block tb-38 on this bug.

I see.
(In reply to Suyash Agarwal (:sshagarwal) from comment #13)
> (In reply to Magnus Melin from comment #12)
> > (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
> > comment #5)
> > > Maybe, re-enable the condition that, whenever '@' is entered as a part of
> > > the search term, we stop autocompletion?
> > 
> > I don't think that's likely a good idea. 
> > We should just not force if delete was the last thing you pressed.

That does not sound like a sufficient condition to avoid this bug.
I can use Ctrl+X or Shift+Del or Backspace or Delete to remove the highlighted part of the inline autocomplete suggestion. Perhaps there are other ways to do it that I'm not currently thinking of.
Please don't forget the workaround given in comment #4: Type the address, then a comma, then <enter>.
The address will be red, since it's not in the address book, so that gives the user time to check it.
(In reply to Suyash Agarwal (:sshagarwal) from comment #7)
> (In reply to Thomas D. from comment #6)
>> (In reply to Suyash Agarwal (:sshagarwal) away till 9th of March from
>> comment #5)
>>> Maybe, re-enable the condition that, whenever '@' is entered as a part of
>>> the search term, we stop autocompletion?

>> That doesn't sound like a sustainable solution, more like a workaround.

> No this isn't a workaround. This is one of the ways to fix this.

Stopping at '@' is too early.

But you're on the right track...

What should happen, in my opinion, is autocomplete should NOT kick in if the user has entered a fully complete email address - ie, localpart@fqdnpart. If the address is *complete*, quto-correct should NOT kick in, period.
(In reply to Kent James (:rkent) from comment #19)
> Nobody disputes that this is a long-standing bug that needs fixing, and if
> you can motivate someone to get it fixed, we'll try to get it into tb-38.
> But at this late date, tracking is meant more for blockers, and we are not
> going to block tb-38 on this bug.

I don't normally disagree with you Kent, but in this case - and as one who has been bitten by it - I do.

This is absolutely critical, and should absolutely BLOCK. It needs to be fixed, and whoever changed the original code and caused the bug should be brought in to do so. This is something I've commented on in the Libreoffice circles.

I understand the nature of free software and such, but there should be an understanding that, if you are going to be allowed to change some core code/component in software such as TB, there should be a very strong obligation (and expectation from the other developers) that if you introduce bugs in this new code, you must fix them in a timely fashion, or the code gets ripped out and the old code put back in place.

The Auto-Completion bugs introduced in TB 31 hung around way too long in my opinion, and this being one of the last ones - but more importantly one of the most serious ones - well, it simply must be fixed asap.

Who wrote this new code? Get them in here and have them fix this! Or if they refuse, shame them publicly by naming them and calling them out.
Can you please apply some moderation here!
This bug 1138033 is a continuation of bug 131692, which was logged in 2002.
Therefore this was NOT introduced in TB31, it's been there long before.
In TB31 the auto-complete feature was switched over to Mozilla Core's toolkit (bug 959209). Sadly this had some problems which have been fixed or will be fixed on TB38 (red addresses, bug 1042561).
This bug here together with its predecessor is ancient and will therefore not block anything. Kent is 100% correct in his assessment.
The decision to block release on a bug is also a decision to delay giving users hundreds of other fixes and improvements. A bug virtually has to be a regression in the new release to justify that. There is no significant difference from the perspective of this bug between to blocking 31.0 by six weeks to include this bug, or shipping 31.0 on time and including the fix to the bug in 31.0.1 six weeks later.

But please keep advocating to fix this, Charles and Thomas! We need to hear about critical issues that are affecting our users.
Apologies, I missed that it was an old bug - and unbelievable that it has been around for 12+ years...

And now I'm even a bit unnerved - this probably means that I have sent emails to the wrong people and didn't even know/realize it...

Scary stuff...

Can it really be that hard to fix?
A good rule of e-mail etiquette is to check the recipients (and CC's and BCC's) before you press "send".

As for whether it's hard to fix. Yes, it's hard to fix. To understand this, you need to understand the Mozilla architecture. At the base you have Mozilla core. On top of that, there is Firefox.

Thunderbird is really a client package, which uses a lot of code from the Mozilla core. For the autocompletion it uses the core toolkit. Before the completion of bug 959209 it used XPFE, also a Mozilla core module.

Both these didn't/don't deliver the functionality we need 100%. It is hard to either modify the core component or supply an equivalent client component.
As I said before, this is a SECURITY issue. I sent proprietary information to the wrong person because of this bug. That makes it a showstopper to me!
Agreed, it's very annoying that you can't enter the recipient you want. However, this bug exists since 2002, it hasn't stopped any show so far. With all due respect: How can this be a security issue?

The program clearly shows the recipients, and when you press send, it sends the e-mail to the shown recipients. If it affects you, enter the recipients into the address book and then select the correct recipient.
Sending proprietary material to the wrong person IS a security issue!
While I feel your pain, the short answer is:

So don't do it. You can easily see the list of recipients before you click send.

Now that you are aware of the bug, just act on that awareness...
(In reply to James Rome from comment #31)
> Sending proprietary material to the wrong person IS a security issue!

Hi James, I'm sorry that you've been affected by this bug and I've made it clear that I consider this a critical bug which deserves attention. However, as explained by Jorg K and Kent James, it looks we can't make it for TB 38. Per comment 28, this is non-trivial to fix, and the approach isn't even clear yet.
If we just switch off force-complete which seems to cause this, we'll probably end up with other unacceptable regressions of the desired UX.

In the meantime, now that you're aware of this bug, pls verify your recipients before pressing send, as suggested by Jorg.

As another, safer workaround to this bug, add a comma after new recipients which aren't in your AB yet. This will avoid the forced completion to the wrong recipient:

foo@bar.com, [Enter]

We understand what you're trying to say about "security", namely that your privacy has been violated by sending proprietary information to the wrong recipient. Pls accept that technically, according to the terms used here and elsewhere, this is a serious privacy violation issue, involving datalossy behaviour, but NOT a security issue. Security keyword is reserved for bugs that are dangerous because their public description might help hackers to exploit them (something like that).
Whiteboard: [ux-wysiwyg][Better STR comment 1][possible solution(?): bug 533109] → [ux-wysiwyg][Better STR comment 1][possible solution(?): bug 533109][workaround: add comma after new recipient]
Thanks Thomas. Actually for me, both the desired and undesired recipient were in my address book. TB swapped them as I typed. They had similar names.
See Also: → 1130858
See Also: → 1152517
There's an impressive variety of scenarios where TB reverts user's explicitly confirmed recipient and cunningly picks another unrelated recipient instead (which is dataloss and privacy violation):

Bug 1130858 - Recipient autocomplete suggestion overrides ANY manual address input if quickly entered/pasted and confirmed with Enter/Tab before autocomplete suggestions disappear

Bug 1138033 - Stubborn recipient autocomplete silently swaps recipients: Cannot compose message to valid, normal, new email address (doe@asdf.com/admin@foo.co) if similar longer address already exists in AB (john.doe@asdf.com/admin@foo.com)

Bug 1152517 - Recipient autocomplete wrongly considers last mouse-hovered contact from results dropdown "selected" and then uses that unintended, random recipient upon blur (via Tab, Enter, or when moving to subject or body)

So to work around all of them, users will need to:
- Slow down when entering recipients to avoid Bug 1130858
- Put a comma even after a single recipient, or add every recipient to your AB first, to avoid Bug 1138033
- Move their mouse pointer into some safe corner of the screen before(!) entering a recipient, to avoid Bug 1152517

Easy, ain't it? :p
Duplicate of this bug: 1159556
Priority: -- → P2
Duplicate of this bug: 1168086
The cesspool of comment 35 is a nasty and ridiculous set of failures in one of the core functions of a mailer: defining message recipients. I'd love to see some developer focus on these for TB 38 ESR.
Thomas:
Please restrict comments to the particular bug, unless it's a summary/meta bug which this is not. Please avoid language that could offend. If possible, focus on technical aspects.
In comment #35 you write about three bugs: Bug 1130858, bug 1138033 (this bug) and bug 1152517. Please note that two of them have patches scheduled for review. The reviewer was on leave and is now at Whistler.

While we're already off topic: My personal experience shows that as much as I tried: Words don't fix bugs, patches do!
--- off topic ---

(In reply to Jorg K from comment #39)
> Thomas:
> Please restrict comments to the particular bug, unless it's a summary/meta
> bug which this is not. Please avoid language that could offend. If possible,
> focus on technical aspects.

Hi Jorg,
as a key member of TB QA and TB UX (see [1]), and as a member of TB Core team, I'm commenting from inside, and I know exactly what I'm doing when I comment here, after 8 years of massive activity and experience in BMO (see [2]).

(Fwiw, sorry if "cesspool" sounds offensive but it comes with a figurative meaning which is simply "pit of sins", which imo describes the situation of multiple nasty bugs in an important corner of the UI quite appropriately. Plus I'm quite sure the more literal connotation of "sh**!" is not very far-fetched when TB sends your private messages randomly to the wrong recipients!).

It's part of my bug triage and UX responsibilities to point out relevant relationships between bugs. We don't and we can't have meta bugs for everything (and wrt recipient auto-complete, there are actually lots of bugs - 103 currently -  which are relatively easy to find using quicksearch [3]). Comment 35 purposefully places this bug in context of the two other bugs in the very same corner of the UI, so that developers and users can have a better understanding of what exactly might be going on when users just complain that "TB sends to the wrong address which I never picked". That context provides valuable information for evaluating the priority of these bugs in terms of the very bad UX they cause in combination. Also valuable for developers, even for the technical solution as these bugs are in the very same corner of the code and are likely to interact with each other. It's required information for our valued users on BMO because it prevents unnecessary duplicates and discussions about which bugs they might be seeing, and why their bug might still occur although we claim to have fixed one bug which they wrongly thought was their case.

> In comment #35 you write about three bugs: Bug 1130858, bug 1138033 (this
> bug) and bug 1152517. Please note that two of them have patches scheduled
> for review. The reviewer was on leave and is now at Whistler.

Thank you. That's valuable and somewhat positive information, if we ignore the painful fact that users have been suffering from this bug for more than 12 years. As advocates of our users, let's note that it's not fixed until it's fixed (also considering that it might take even longer until it actually gets rolled out on release). We've seen patches rotting away for years because reviewers didn't have time at the time and others didn't follow up, and nobody cared.

> While we're already off topic: My personal experience shows that as much as
> I tried: Words don't fix bugs, patches do!

My personal experience also shows that without words, patches might never come to a bug.
I'm much better at words than at patches, and between thousands of bugs, we definitely need clear arguments (and occasional UX summaries including emotions how our users feel about this) for prioritizing certain bugs over others. After that, I'm hoping for and depending on talented coding people like you to have mercy and dedicate their valuable time and skills to fix the issues at hand :)

Finally, please read comment #26 where there's an explicit and personal invitation by the project leader, Kent James, to continue advocating for this bug so that we don't lose it off the radar:

(In reply to Kent James (:rkent) from comment #26)
> But please keep advocating to fix this, Charles and Thomas! We need to hear
> about critical issues that are affecting our users.

As another point in support of cross-bug summaries as in my comment 35, we do need to understand the other bugs in the very same corner to appreciate just HOW MUCH and HOW EASILY our users will be affected by one or several of these bugs, all with the same devastating result, sending messages to the wrong recipient (although users do everything right!), which is a massive violation of privacy (with bad repercussions for the general reputation of our product). Imho, returning the buck to our users (as you might appear to do in your comment 30) is putting the cart before the horse. Notwithstanding all the practical problems that we might have as a small volunteer community to get things done timely and comprehensively, we need to appreciate the experience of our users as much as we possibly can and should always strive to understand, acknowledge and also FIX the real-life problems caused by the bugs and shortcomings in our software. Pointed words are a vital part of understanding and improving UX!

(In reply to Jorg K from comment #30)
> The program clearly shows the recipients, and when you press send, it sends
> the e-mail to the shown recipients. If it affects you, enter the recipients
> into the address book and then select the correct recipient.

[1] https://wiki.mozilla.org/Thunderbird:Summit_2014#Invited_Attendees
[2] https://bugzilla.mozilla.org/user_profile?login=bugzilla2007%40duellmann24.net
[3] https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Athun%2Cmailn%20auto%20complet [sic]
...and if you think about it a bit more, there might be millions of users out there who'll never report this bug
a) because they might never find out that they've sent to the wrong address until the wrong recipient lets them know, which is painful for both parties
b) even if they find that they've sent to the wrong address, they might just think it was themselves who got it wrong, and it might take considerable time to realize that it's actually TB doing the wrong thing
c) it's not all that easy to find your way to bmo and find the right bug to comment on (plus we don't always encourage "me-too" comments...)

So as somebody from inside the team with a pretty good understanding of what's happening here, I wouldn't know who else should speak up on behalf of users if not somebody like me?

As reporter of this bug, I rewrote the description from scratch because the old bug did NOT have sufficient clarity any more - again, pointed words matter...

I also pointed out why this bug deserves attention now more than ever because starting from TB 31, the number of cases where this can occur has significantly increased, as I describe with detailed illustrations in comment 0. Which is yet another reason to advocate for this particular bug: We've had a number of complaints claiming that after Bug 529584 and Bug 558931, recipient autocomplete gets it wrong more often than before. However, there's a subtle difference between cause and trigger. Bug 529584 and Bug 558931 were just the triggers which exposed a whole range of existing bugs and shortcomings which were more hidden before, but are the actual causes or at least massive factors of the undesired behaviour. You'll understand that we must really strive to eliminate those bugs whose symptoms and causes are known and defined, so that we don't rush to all the wrong conclusions about other design factors. If you read through user comments, you'll be surprised how little they appreciate when you tell them "look, it's not really because we changed the search pattern, but there's something else under the hood which makes this fail...". And those "something else under the hood" things are often age-old bugs like this one (or its predecessor for the avoidance of doubt). And yes, there are very few people out there with sufficient overview in bugs to make clear statements about which is which, as in my comment 35...
Can we conduct this discussion elsewhere, please. Long comments that have little to do with the problem make the bug hard to read and hinder finding relevant information and fixing the bug.
Jorg, please stop telling Thomas what to do and what not to do. Thanks.
(In reply to Thomas D. from comment #35)
> There's an impressive variety of scenarios where TB reverts user's
> explicitly confirmed recipient and cunningly picks another unrelated
> recipient instead (which is dataloss and privacy violation):
> 
> Bug 1130858 - Bug 1138033 - Bug 1152517

Bug 1012397 - PostSearchCleanup improvement needed - Address doesn't auto complete if you tab too quickly.  NO address filled in.
See Also: → 1012397
My 2cents worth. Hope it offers some insight

I just noticed in Nightly that a manually edited address in the To field is auto replaced with the incorrect entry from the address book on the field loosing focus.

I sent a mail adding a period after the address.  It bounced for obvious reasons.  I clicked edit as new and edited the address.  On leaving the field the value was replaced with the original.

This behaviour continued until I removed the incorrect address entry from the collected address book.
(In reply to Guy McArthur from bug 131692 comment #27)
> Was just bitten by this bug, pasting in (e.g.) anne@someplace.xyz (who is
> not in the database) and Thunderbird changes it to jeanne@someplace.xyz (who
> is)... and there is no way around it!
> 
> Thunderbird 31.8.0.
Jorg, could this long-standing and nasty bug in one of our key functions (adding recipients) be a coding challenge for you?
Flags: needinfo?(mozilla)
(From bug 1192739 comment #9):
The autocomplete code is surprisingly complex in the details with the controller basically supporting two different modes plus a lot of minor options. I don't know anything about it; I already had a bad time hunting down bug 1042561.
Flags: needinfo?(mozilla)
Duplicate of this bug: 1367801
You need to log in before you can comment on or make changes to this bug.