LDAP address autocomplete does not work when writing new e-mail

RESOLVED FIXED in Thunderbird 30.0

Status

defect
--
major
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: msmat, Assigned: neil)

Tracking

({regression})

Trunk
Thunderbird 30.0

Thunderbird Tracking Flags

(thunderbird27 affected, thunderbird28 affected, thunderbird29 fixed, thunderbird30 fixed, seamonkey2.24 affected, seamonkey2.25 affected, seamonkey2.26 fixed, seamonkey2.27 fixed)

Details

Attachments

(2 attachments, 2 obsolete attachments)

13.46 KB, application/x-xpinstall
Details
8.47 KB, patch
standard8
: review+
Details | Diff | Splinter Review
User Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20101206 SeaMonkey/2.0.11 Firefox/4.0 (Nightly/Aurora)
Build ID: 20101206150522

Steps to reproduce:

After updating to SeaMonkey 2.22 LDAP autocomplete stopped working. In 2.21 this worked fine. No manual changes were done in the configuration.

1. Define LDAP address book
2. Create new e-mail
3. Start writing recipient address
4. No hints from LDAP are shown



Actual results:

The list box with matched addresses does not show the results from LDAP (only local address book shows some entries).
It seems SeaMonkey even does not try to send any request to LDAP server. There was no packet sent to the LDAP port in Wireshark.


Expected results:

In previous version the listbox contained names that were taken from the configured LDAP address book.
OS: Linux → Windows 7
Just chiming in with an agreement here.  2.22 broke LDAP completely for all of my users.  Update to 2.22.1 hasn't shown any improvement.
Also, to clarify - LDAP is broken when typing an address directly in the "To" line.  If you click on "Address" in the toolbar, and select the right address book from the "Look in:" box on the window that pops up, it works fine every time.

Just trying to type in an address directly in the Compose window is where the problem lies.
I'd have thought this a duplicate of bug 901771, which went out with TB 25 beta
Severity: normal → major
Keywords: regression
Duplicate of this bug: 944983
For me it works (see bug 944983) immediately after seamonkey 2.22.1 has started.
I can compose new mails and it autocompletes OK.
When it has been running for some time (an hour or so) and I go back to composing a mail it no longer works.
Disaster!   I don't dare to show up at work tomorrow.
So this bug here also happens with SeaMonkey 2.23?
This bug maybe not.  It works OK on initial startup.
But bug 944983 which was claimed to be a duplicate of this bug is still happening!
The LDAP lookup works OK at first but after about an hour of running the program (with IMAP mail open
but otherwise nothing happening) the problem as described in bug 944983 still occurs with 2.23
Disagree with working OK on startup.  Works randomly, yes, but not always at startup.  More often broken than not.  Problem tested and verified, so far, on 26 client computers as well as 3 of my own.

I have the dean of my college ready to order SM removed from machines because of this problem.  Would rather not do that, so hopefully a fix can be implemented - SOON.

Still broken in 2.23.
Duplicate of this bug: 944983
So confirming this bug based on the comments.
Do you see any error in the Error Console (Tools->Web Development->Error Console) when this problem happens? Best way to check this is to open the Error Console, click the Clear button and then try to enter a mail address so that it does an LDAP lookup.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Version: SeaMonkey 2.22 Branch → Trunk
I started Seamonkey 2.23, with -mail option, verified that LDAP was working immediately after startup,
and cleared the error console.
After some time running I came back and in the error console was this error:
Tijdstempel: 23-12-2013 19:54:27
Fout: TypeError: this.browsers[i] is undefined
Bronbestand: chrome://navigator/content/tabbrowser.xml
Regel: 1096

This was two hours after I started the program.
Then I tried to send a new mail.  LDAP lookup was not working.  No more errors appeared in the console.
Of interest may be that I have a custom html page as "mailnews.start_page.url" and this page has a
refresh of two hours.
The error above is clearly related to the refresh of the mailnews start page only.
After having the program on for a night the error appears every two hours, i.e. every refresh.
But there is no message that is related to the LDAP problem discussed in this bug.
Ok.
Do you know if the local, address book-based autocomplete still works when you try it (maybe create one or two test entries in the local address book if you don't have any contacts there)? As the local matches should appear above the LDAP entries in the autocomplete dropdown.
Yes the local address book is unaffected.
Can someone create an LDAP log like described at https://wiki.mozilla.org/MailNews:Logging ? Not sure how useful that will be as Comment 0 says no network traffic goes out at all when this bug happens. Use "ldap" as logging module.
I've tried to reproduce this bug, but looks like it's quite difficult. Probably Bug 452232 is related as this one changed LDAP autocomplete to a new code interface (starting with SeaMonkey 2.22).
On that wiki page you can also jump directly to https://wiki.mozilla.org/MailNews:Logging#Generating_a_Protocol_Log you want to use
set NSPR_LOG_MODULES=ldap:5
for the logging module (and of course replace thunderbird with seamonkey).
So what I've been able to reproduce is the following:
I enter part of some name/address in the address field, wait for local and LDAP autocomplete to happen, then I press backspace to delete everything and then enter that name again. Now sometimes only local autocomplete works for me. But when opening a new compose window, autocomplete always work for me...
For the people seeing this bug: Can you check under the URL about:config if the preference mail.compose.max_recycled_windows is set to 1 for you? Just wondering (1 is the default value).
I tried with the logging and initially (when it still works OK) it logs lines like this:
0[e0f140]: nsLDAPOperation::SearchExt(): called with aBaseDn = '[dn]'; aFilter = '[filter]'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 500
0[e0f140]: pending operation added; total pending operations now = 1
0[e0f140]: nsLDAPMessage::GetDn(): dn = '[result]'
1208[a699710]: pending operation removed; total pending operations now = 0
0[e0f140]: nsLDAPMessage::GetDn(): dn = '[result]'
0[e0f140]: nsLDAPMessage::GetDn(): dn = '[result]'
etc

when it has failed after an hour or so, and I try to send another mail, it only logs:
0[e0f140]: nsLDAPOperation::SearchExt(): called with aBaseDn = '[dn]'; aFilter = '[filter]'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 500
0[e0f140]: pending operation added; total pending operations now = 1

no more results appear.   but when looking on the LDAP server the query has not been received at all!
when I now type another address line, it logs only the first line.  the line about pending operations does not appear.
Hopefully this is useful for you.

mail.compose.max_recycled_windows is at the default value 1.
And it never recovers from this problem then (so restarting SeaMonkey is the only way to get this working again)? So opening/closing Compose windows does not change anything? I wonder if opening a second Compose window changes anything?
Rob: The LDAP server you use is not public accessible in any way, is it? I've used the server ldap.uno.edu with Base DN "o=UNIVERSITY OF NEW ORLEANS,c=US" for testing and have a hard time reproducing it the way you say it (see Comment 17 for what I'm able to reproduce, but that looks like a different problem).
No, everything is on a local network which is not accessible from outside.
The LDAP server is just openldap.  There have been no problems for >10 years and now from 2.22
there is a problem.
Note that in my case it only fails when the program has been running for some time.  I type "rob"
in the address field and when everything is OK I get a list of some people named Rob in the company,
and I close the compose window.   Then I leave that PC running for some hours, I come back and open
a new compose window, type "rob" in the To: field and nothing happens, it just sits there.  When I
type some other thing that matches the single entry in my PAB, it works.
Rob: Another question on the log :) (just to make sure): So when it stops working you never get any line like
1208[a699710]: pending operation removed; total pending operations now = 0
after the previous
0[e0f140]: pending operation added; total pending operations now = 1
? This looks like it would mean it never gets around to finishing this LDAP query (some bug in LDAP code?).

Also, can you confirm Comment 2 (searching for addresses via the "Address" button still works)?
Yes that is correct.  I get only the log lines mentioned in my comment and I checked that the LDAP
server did not receive a connection.  I have started the program again for a test re comment 2.  Wait.
Tested it, and I can confirm that when the autocomplete from LDAP does not work, the lookup in the
address button still works (have to manually select the LDAP addressbook first).
Rob: Can you take a look at my questions from Comment 19 (opening/closing compose window changes anything, restart needed to fix problem)? Fixing this problem here might turn out to be quite difficult (especially when the problem is so difficult to reproduce), but we'll see.
I think I answered those!  It works OK when I start the program, stops working after one or more hours
of uptime.  I have only the main mail window open during that time, I open a new compose window to
send a new mail.  Once it is broken, opening a new compose window does not fix it.  When I restart the
program everything is reset (i.e. it works, but stops working later).
The system itself is not rebooted.
It is not like it is difficult to reproduce, it only takes time.  It fails every time I try it, on
different systems where I install the same version.  Currently testing with 2.23 (Dutch).
For a new test, I have now restarted the program (had been running since comment 24 and was still in
failure mode) and opened a new compose window to check if that one is dead when I next check it.
Well, maybe this bug is dependent on network latencies and async APIs are involved, too as far as I see this (LDAP send queue and callback to autocomplete handler). Therefore this might make it difficult for some people to reproduce the bug. Also one question/problem that needs to be solved is why you have to wait (in some way) for one hour or two to see why this bug is happening.
Result of this test: the compose window that was opened shortly after the program was started and kept open for some hours did not do LDAP completion either.  So opening/closing compose windows has no influence at all.
There is something else that is interesting: when a name is typed and ENTER is pressed, the cursor does not move to the next input field.  Normally it does.
When using the mouse to go to the second input field, the LDAP lookup still does not work but ENTER does move the cursor to the next field.  When clicking in the first field again, ENTER now works there as well.
But no LDAP autocomplete until the program is closed and restarted.
I can confirm what Rob wrote. Just one additional comment. When LDAP autocomplete stops working first time (after restarting Seamonkey) I have in the log 3 lines:
0[1f0f140]: nsLDAPOperation::SearchExt(): called with aBaseDn = '[dn]'; aFilter = '[filter]'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 100
0[1f0f140]: pending operation added; total pending operations now = 1

but for any other attempt the log has only 2 lines:
0[1f0f140]: nsLDAPOperation::SearchExt(): called with aBaseDn = '[dn]'; aFilter = '[filter]'; aAttributes = cn,commonname,mail,objectClass; aSizeLimit = 100

Note that for all other attempts the message "0[1f0f140]: pending operation added; total pending operations now = 1" is not there. This happens even when closing composer window and opening a new one.
Only Seamonkey restart makes it working again (but only for a short time - about one hour).
On the offchance that it helps us narrow down the problem, when the problem next recurs please can you try the following steps:

1. Close the Error Console if it is already open
2. Open the Address Book
3. Select your LDAP directory
4. Open the Error Console
5. Paste the following line of code and evaluate it:

top.opener.GetDirectoryFromURI(top.opener.GetSelectedDirectory()).nsIAbLDAPDirectory.protocolVersion^=1

6. Test compose autocomplete
7. Evaluate the line of code again
8. Test compose autocomplete

(You don't need to go through the close/open steps twice, they are just to ensure that the address book is the opener of the error console.)
Just in case you don't know how to do open the Error Console: You can do that via Tools->Web Development->Error Console (from the address book window).
I started the program, came back hours later and tested autocomplete.  It failed.
Then after evaluating the line above it worked again, and when evaluating it again it still works.
I see alternate output of 2 or 3 but it does not matter which one, it still works.
After the above, again waited until it failed, and kept the error console open.
It started to work immediately after clicking the evaluate button again (output was 2 in that case).
Posted file Test extension (obsolete) —
This extension attempts to replace the built-in LDAP autocomplete component. Assuming the extension doesn't install directly from Bugzilla, download it, open the Add-ons Manager, and install the extension from the downloaded file.

You will need to restart if you have done any LDAP autocomplete since installing/enabling/disabling/uninstalling the extension. (I guess I shouldn't have made it a restartless extension...)
Test extension has made no difference for me.  Installed from link above - shows up in Add-on Manager, but after restarting SM and attempting immediate LDAP lookup, it fails instantly.
I must have let an error slip in to the extension. Sorry about that. Writing an extension at midnight local time was probably a bad move.
please determine 1 day regression range using nightly builds, where builddate+1 fails and builddate works
Flags: needinfo?(msmat)
Maybe Rob and msmat are experiencing a slightly different problem which my test extension will still help with...
I think my problem is exactly the same as Rob has. Even the last test with evaluating the expression (from Comment #30) in Error Console generated the same result (and same behavior) as Rob described (that was the reason I did not comment it anymore). However I will give the extension a try and will report the result.
Flags: needinfo?(msmat)
I installed the extension. Now when I type the e-mail address I got this error in the Error Console:

Timestamp: 1/14/2014 08:01:37
Error: JSON.parse: unexpected character
Source File: resource://gre/modules/XPIProvider.jsm -> jar:file:///C:/Users/msmat/AppData/Roaming/Mozilla/SeaMonkey/Profiles/xoy8a4wc.default/extensions/ldap@test.addon.xpi!/bootstrap.js
Line: 170

The affected line is:
let params = JSON.parse(aParam);
After adding a line in the extension to print the value of "aParam" to know why it could not be parsed, it shows "id1".
Ah, my bad, that extension only works in 2.25 or later. I'll try to fix it to work in earlier versions.
Posted file Test extension (obsolete) —
v0.2 should work in earlier versions of SeaMonkey, but untested, sorry.
Attachment #8359005 - Attachment is obsolete: true
Installed version 0.2 and this time I get different error:

Timestamp: 1/14/2014 12:56:28
Error: this._parser.makeMailboxObject is not a function
Source File: resource://gre/modules/XPIProvider.jsm -> jar:file:///C:/Users/msmat/AppData/Roaming/Mozilla/SeaMonkey/Profiles/xoy8a4wc.default/extensions/ldap@test.addon.xpi!/bootstrap.js
Line: 124
Posted file Test extension
Third time lucky, perhaps?
Attachment #8359678 - Attachment is obsolete: true
So far, so good with v0.3.  Worked immediately after installing, and continues to work an hour and a half later.  Will continue testing throughout the day.
Version 0.3 works well for me. I Could not run it for longer time today but will check tomorrow if the same problem occurs.
8 hours in, and every LDAP lookup I attempt is working.  I think we have a winner...
After several hours the autocomplete function is still working.
Rather than keeping the LDAP queries around indefinitely, this patch changes the autocomplete component so that it only keeps the most recently used query around for a minute since the last autocomplete. After that a new query is created.

The test extension was a simpler version of this whereby the old cache code was retained but the entire cache was simply cleared after a minute of inactivity.
Assignee: nobody → neil
Status: NEW → ASSIGNED
Attachment #8363033 - Flags: review?(mbanner)
Comment on attachment 8363033 [details] [diff] [review]
Proposed patch

Review of attachment 8363033 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good. r=Standard8
Attachment #8363033 - Flags: review?(standard8) → review+
Component: MailNews: Composition → LDAP Integration
Product: SeaMonkey → MailNews Core
Pushed comm-central changeset 20cb29aba951.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 30.0
Comment on attachment 8363033 [details] [diff] [review]
Proposed patch

[Approval Request Comment]
Regression caused by (bug #): Toolkit LDAP autocomplete switchover
User impact if declined: LDAP lookups stop working after an hour
Testing completed (on c-c, etc.): Test extension successfully resolved problem for several affected users. (Extension no longer installs on aurora builds.)
Risk to taking this patch (and alternatives if risky): Low, only affects LDAP users.
Attachment #8363033 - Flags: approval-comm-aurora?
OS: Windows 7 → All
Hardware: x86 → All
Attachment #8363033 - Flags: approval-comm-aurora? → approval-comm-aurora+
Duplicate of this bug: 996025
You need to log in before you can comment on or make changes to this bug.