1045753 - Compose email LDAP erases text (autocompletes to blank entry when result email not set)

Status: RESOLVED FIXED

(Thunderbird :: Message Compose Window, defect)

Description

[Ted Huntington]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0 (Beta/Release)
Build ID: 20140717132905

Steps to reproduce:

You have to have an LDAP server configured and enabled.
Click "Write", start typing letters in "To:", a blank entry appears on the first line below and erases all the text you have. It's as if TB31 is matching on the blank entry in the LDAP- but there is no blank LDAP entry according to the LDAP administrators. 


Actual results:

text the user types in are made blank- as if a blank LDAP address is matched


Expected results:

no LDAP address should be matched- the text in "To:" should not disappear.

Comment 1

[Wayne Mery (:wsmwk)]

Is ldap the only address book you use?
In other words, at Tools | options | composition | addressing
is local address book unchecked(off), and ldap check(on)?

Comment 2

[Ted Huntington]

Yes "Local Address Books" is unchecked. This is in the beta2 release too. I'm testing earlier versions to see when it was introduced. We upgraded from v24.7.0.

Comment 3

[Ted Huntington]

I just found that v30.0 doesn't have the error. The blank ldap entry appears but it doesn't blank the To: text

Comment 4

[markus.jansen]

Same problem here under Mac OS (tested on 10.7.5 & 10.9.4) since upgraded to TB 31.

Comment 8

[spam-mails-here]

Same Problem under Ubuntu 12.04 LTS with TB 31
(Linux pc 3.2.0-67-generic #101-Ubuntu SMP Tue Jul 15 17:46:11 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux)

Ubuntu recently marked this Version of TB as stable.

Comment 9

[Magnus Melin [:mkmelin]]

Does anyone have a publicly accessible ldap server where these symptoms can be seen?

Comment 10

[spam-mails-here]

Try LDAP-settings from:
https://howto.adams.edu/index.php/Thunderbird_-_Add_LDAP_to_Address_Book
(untested, because I use TB<31 at the moment)

All Mails ends with "@adams.edu", so try this as a filter.

Comment 11

[markus.jansen]

(In reply to spam-mails-here from comment #10)
> Try LDAP-settings from:
> https://howto.adams.edu/index.php/Thunderbird_-_Add_LDAP_to_Address_Book
> (untested, because I use TB<31 at the moment)
> 
> All Mails ends with "@adams.edu", so try this as a filter.

Hm, I can't resolve this symptons with this public LDAP server and TB31 (Mac OS 10.7.5).

Comment 12

[spam-mails-here]

Yes, Mr. Jansen, you're right. The proposed server works fine.
We use
slapd 2.4.28-1.1ubuntu4.4 OpenLDAP server (slapd)
and here auto-completion presents the first proposed entry which is empty and all letters written to the "To"-Field are deleted immediately.

Comment 13

[spam-mails-here]

Attachment: LDAP-Autocomplete.png

Comment 14

[Gus]

(In reply to Magnus Melin from comment #9)
> Does anyone have a publicly accessible ldap server where these symptoms can
> be seen?

Here's one:

    LDAP:
        server: directory.utexas.edu
        port: 389
        base: dc=directory,dc=utexas,dc=edu
        maximum entries returned: 50

Comment 15

[spam-mails-here]

Gus, thx
Try to insert "araid" in the To-Field and wait after each letter for about a second.

Comment 16

[Gus]

(In reply to spam-mails-here from comment #15)
> Gus, thx
> Try to insert "araid" in the To-Field and wait after each letter for about a
> second.

OK, I have done so. It does not behave as with most other letter combinations, in that I can type the whole "araid" sequence without what I'm typing disappearing.
Not sure what this tells you...

Comment 18

[John Liebson]

Bug present in Tb 31.1. I've reverted to 24.6 again.

Comment 19

[John Liebson]

Solved, for me: I created a new Tb profile, added existing mail accounts, updated Tb to 31.1.1, and am able to enter addresses in the composition module, addresses that remain in the To: line.

Existing default profile creation date: 18Nov2011; problem most likely related to old extension data, etc. left in the prefs.js file in that profile, supposition supported by trying to write using that profile, which does not work.

Comment 20

[Volker Englisch]

Adding a new profile did not fix the problem for me.  
I started the profile manager, created a new profile, setup my mail and LDAP server but the problem still exists.

Comment 22

[Jonathan Nicol]

Version 31.1.0 (thunderbird-31.1.0-1.fc20.x86_64)

Same problem here, I'm fairly certain I've identified the LDAP entry causing problems, but I'm not sure why. It does appear there is a group and username with the same CN, and the user is a member of the group. Don't know if that's relavant. 

(domain and usernames changed to protect the innocent)

dn: uid=app-reporting,ou=People,dc=domain,dc=com
sn: Automation
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
givenName: Reporting
uid: app-reporting
uidNumber: 5012
displayName: Report Automation
cn: Report Automation
homeDirectory: /home/app-reporting
gidNumber: 5012


dn: cn=app-reporting,ou=Groups,dc=domain,dc=com
gidNumber: 5012
objectClass: top
objectClass: groupOfUniqueNames
objectClass: posixGroup
cn: app-reporting
uniqueMember: uid=app-reporting,ou=People,dc=domain,dc=com
uniqueMember: uid=realuser1,ou=People,dc=domain,dc=com
uniqueMember: uid=realuser2,ou=People,dc=domain,dc=com

Comment 23

[Rob Pearson]

I definitely confirm this as a bug, annoying one too!   I care about getting this fixed, so I'm offering USD 300.00000000 via FreedomSponsors to the first person who fix it.   https://freedomsponsors.org/issue/548/compose-email-ldap-erases-text-autocompletes-to-blank-entry

You can also join me and throw in a few bucks there and we'll get it fixed faster :)

If you fix this issue (see my acceptance criteria there) please use that site to request your payment.

Comment 25

[Magnus Melin [:mkmelin]]

I was able to reproduce this on tb31, using details from comment 14 and entering "ara". 
Doesn't seem to have the problem on trunk though, or at least it's harder to reproduce. I don't reproduce it 100% of the timest on 31 either.

Comment 27

[Magnus Melin [:mkmelin]]

Whatever fixed it for trunk went in some time between 2014-05-28 -> 2014-06-05. Can't check a more precise range on linux as apparently we had a bustage then.

Comment 28

[Wayne Mery (:wsmwk)]

(In reply to Magnus Melin from comment #27)
> Whatever fixed it for trunk went in some time between 2014-05-28 ->
> 2014-06-05. Can't check a more precise range on linux as apparently we had a
> bustage then.

was seamonkey busted in the same range?

Comment 29

[:aceman]

Is the autocomplete here matching entries from the LDAP server exclusively, or does it also look into Collected addresses addressbook or similar?

Comment 30

[Magnus Melin [:mkmelin]]

(In reply to Wayne Mery (:wsmwk) from comment #28)
> was seamonkey busted in the same range?

Yes. But windows builds are available for both, so I guess I'll check those.

(In reply to :aceman from comment #29)
> Is the autocomplete here matching entries from the LDAP server exclusively,
> or does it also look into Collected addresses addressbook or similar?

During testing I used LDAP only, turned off autocomplete for local address books.

Comment 31

[:aceman]

(In reply to Magnus Melin from comment #30)
> (In reply to :aceman from comment #29)
> > Is the autocomplete here matching entries from the LDAP server exclusively,
> > or does it also look into Collected addresses addressbook or similar?
> 
> During testing I used LDAP only, turned off autocomplete for local address
> books.

Yes, I now see this can be configured in Options->Composition->Adressing .
Are we sure all the reporters have "local addressbooks" turned off when testing this?
Otherwise a half empty contact may cause this. The original report only claims there is no blank entry on the server.

Comment 32

[Volker Englisch]

(In reply to :aceman from comment #31)
> Yes, I now see this can be configured in Options->Composition->Adressing .
> Are we sure all the reporters have "local addressbooks" turned off when
> testing this?

I have "local addressbooks" turned off and I'm seeing this problem.

I did notice that not all entries entered are blanked out for me.  Most of them are blanked but for some search strings submitted I do receive the correct address autocompleted without a blank entry in the search results.

Comment 33

[Ted Huntington]

Just to answer the question about local address books, in my testing I found that it didn't matter whether "Local Address Books" is checked or unchecked. I confirmed that this problem exists even when "Local Address Books" is unchecked.

Comment 34

[Gabriel]

The importance of this bug should be raised.  It makes Thunderbird unusable in the many corporate settings with LDAP address books, because you cannot write e-mails.  Those trying Thunderbird in these settings will see this and give up right away.

Besides the bug per se, I would favor a less aggressive behavior that does not replace what you type unless you hit enter with an auto-complete entry highlighted in the pop-up drop-down.  If that were the behavior, the current bug might show up as an annoying empty autocomplete suggestion, but at least it would let you write the e-mail.  More importantly, the current behavior has the risk of autocompleting to the wrong address.  This makes it possible for users to inadvertently send sensitive e-mails to the wrong addressees.  What do others think?

Comment 35

[Ted Huntington]

I agree- it would be nice if there was a drop-down with possible matches, allowing the user to select, and would solve this error at the same time. If that feature or option could be added, we would be able to upgrade to the latest TB. Either way, hopefully this error will get solved- I can't believe that it is too difficult to figure out- and now there is a $300 incentive to solve it- ahaha.

Comment 36

[Magnus Melin [:mkmelin]]

The fix range is http://hg.mozilla.org/comm-central/pushloghtml?startdate=2014-06-04&enddate=2014-06-05+03%3A00%3A00 - so I bet one of bug 790855 and bug 858337 fixed it.

The issue seems to be with entries that don't have an email address but only display name. In the test case server there is a few of these, e.g. "Jorge L Arellano".

I have a potential fix, but I need to investigate it a bit more.

Comment 37

[Magnus Melin [:mkmelin]]

Attachment: bug1045753_autocomplete_blank.patch

I don't know what went on in the tb24 days, but for 31 (and before 2014-06-05/bug 790855) the makeMailboxObject returned "" if there was a display name, but no email. After that, it returns "displayname <>". (That should probably be discussed in another bug.)

For the test case server, on trunk one can actually scroll down the results and find entries that have show this behavior.

For trunk this is problem is minor, but on 31 the whole autocomplete row gets blank and therefore the input text disappears.

Only LDAP is affected as for local addressbooks we do | if (email) | before calling the _addToResult. I've added the check for local abs too so it could handle possible future parser bugs and core reorg, but it's not strictly necessary. Therefore the test I added doesn't fail before this patch either, but we don't have any ldap tests.

Comment 38

[:aceman]

(In reply to Magnus Melin from comment #37)
> Created attachment 8498917 [details] [diff] [review]
> bug1045753_autocomplete_blank.patch
> 
> I don't know what went on in the tb24 days, but for 31 (and before
> 2014-06-05/bug 790855) the makeMailboxObject returned "" if there was a
> display name, but no email. After that, it returns "displayname <>". (That
> should probably be discussed in another bug.)

I also noticed this new <> in another bug. May that be some of the jcranmer's header parsing and mime stuff?

Comment 39

[Joshua Cranmer [:jcranmer]]

(In reply to Magnus Melin from comment #37)
> I don't know what went on in the tb24 days, but for 31 (and before
> 2014-06-05/bug 790855) the makeMailboxObject returned "" if there was a
> display name, but no email. After that, it returns "displayname <>". (That
> should probably be discussed in another bug.)

I have a fix for this issue in attachment 8497221 [details] [diff] [review] on bug 1005413. As for when that lands... well, bug the reviewers to review patches more quickly.

> For the test case server, on trunk one can actually scroll down the results
> and find entries that have show this behavior.
> 
> For trunk this is problem is minor, but on 31 the whole autocomplete row
> gets blank and therefore the input text disappears.

Before I go off testing this, is this patch for trunk or 31?

Comment 40

[Gabriel]

(In reply to Joshua Cranmer [:jcranmer] from comment #39)
> Before I go off testing this, is this patch for trunk or 31?

I'd like to see it patched for 31, unless there is a newer release for Ubuntu repositories coming up very soon.

Comment 41

[Magnus Melin [:mkmelin]]

(In reply to Joshua Cranmer [:jcranmer] from comment #39)
> Before I go off testing this, is this patch for trunk or 31?

It applies to both, it's just that the symptoms are much worse for 31. In trunk the symptom is that you can get a contact with name but without email, which is of course useless but it doesn't cause other harm.

Comment 42

[Magnus Melin [:mkmelin]]

Joshua: ping on the review. It *might* still be possible to get it in for 31.2.

Comment 43

[Joshua Cranmer [:jcranmer]]

I won't have time to look at this for at least 24 hours.

Comment 44

[Gabriel]

(In reply to Magnus Melin from comment #42)
> Joshua: ping on the review. It *might* still be possible to get it in for
> 31.2.
For the benefit of users following this bug but not familiar with the Thunderbird source code, could you summarize the status of fixing this bug?  What part of the code is responsible for the bug?  It seems you are waiting for improvements on some string tokenization code that should fix this, instead of patching the code that is directly responsible for clearing the addressee field from the message compose window.  Is my that correct?  When do you think a stable release of Thunderbird (v31 or newer) will be available without this bug?  Thanks in advance for the update.

Comment 45

[Magnus Melin [:mkmelin]]

This patch is preventing addresses that do not have an email from appearing in the autocompletion.
If we can get it in very very soon, it could go into tb 31.2 which is already build but will have a respin. If not, it would have to wait until tb 31.3, out in ~6 weeks.

Comment 46

[Magnus Melin [:mkmelin]]

Comment on attachment 8498917 [details] [diff] [review]
bug1045753_autocomplete_blank.patch

Review of attachment 8498917 [details] [diff] [review]:
-----------------------------------------------------------------

Mark, adding you as reviewer if you happen to have time to look at this

Comment 47

[Ted Huntington]

Just confirming that this error is in TB version 31.2.0 for those out there waiting for it to be repaired. But it looks like 31.3 will not have it.

Comment 48

[Mark Banner (:standard8)]

Comment on attachment 8498917 [details] [diff] [review]
bug1045753_autocomplete_blank.patch

Review of attachment 8498917 [details] [diff] [review]:
-----------------------------------------------------------------

Looks reasonable to me. Might just be worth checking performance of adding the call to makeMailboxObject, though I suspect its not significant actually.

Comment 49

[Daryl Tucker]

This has been resolved in the latest build: 1:31.2.0+build2-0ubuntu0.14.04.1

Comment 50

[Gabriel]

(In reply to Daryl Tucker from comment #49)
> This has been resolved in the latest build: 1:31.2.0+build2-0ubuntu0.14.04.1

I'm afraid not.  I just upgraded to that and the issue persists.  I am using MIT's LDAP.

Hostname: ldap.mit.edu
Base DN: dc=mit,dc=edu
Port number: 389

In a message composition window, if I type "asdf" it gets cleared.

Comment 51

[Daryl Tucker]

I apologize for my previous comment, as it is *not accurate*.

Comment 53

[Magnus Melin [:mkmelin]]

https://hg.mozilla.org/comm-central/rev/4a2869fc899a -> FIXED

Comment 54

[Magnus Melin [:mkmelin]]

Comment on attachment 8498917 [details] [diff] [review]
bug1045753_autocomplete_blank.patch

[Approval Request Comment]
Regression caused by (bug #): bug 790855
User impact if declined: ldap autocomplete will erase input if a contact without email is matched
Testing completed (on c-c, etc.): landing on trunk
Risk to taking this patch (and alternatives if risky): not risky

Comment 55

[:aceman]

(In reply to Mark Banner (:standard8) from comment #48)
> Looks reasonable to me. Might just be worth checking performance of adding
> the call to makeMailboxObject, though I suspect its not significant actually.
That was my first thought too. However it appears there is no call added, the line is just rewrapped.

Comment 56

[Mark Banner (:standard8)]

https://hg.mozilla.org/releases/comm-aurora/rev/d8c36ca11fb1

Comment 57

[Mark Banner (:standard8)]

https://hg.mozilla.org/releases/comm-beta/rev/0c826c10f86e

Comment 58

[Mark Banner (:standard8)]

https://hg.mozilla.org/releases/comm-esr31/rev/b384386f3e69