705837 - High memory and CPU usage, and slow autocomplete during address autocomplete using large addressbook

Status: RESOLVED WORKSFORME

(Thunderbird :: Message Compose Window, defect)

Description

[krichter]

User Agent: Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
Build ID: 20111115184056

Steps to reproduce:

started typing an e-mail address when entering recipient in message compose window (three very large addressbooks (14000, 9000 and 18000 addresses))


Actual results:

message compose window freezes, very high CPU usage for 5 seconds, allocation of ~450 MB RAM, then auto-completion popup shows up


Expected results:

auto-completion popup should show up faster (possibly inefficient search algorithm) and memory resources should be freed

Comment 1

[Wayne Mery (:wsmwk)]

(In reply to krichter722 from comment #0)
> started typing an e-mail address when entering recipient in message compose
> window (three very large addressbooks (14000, 9000 and 18000 addresses))
> message compose window freezes, very high CPU usage for 5 seconds,

with the current performance characteristics of large AB, this is to be expected.

> allocation of ~450 MB RAM,

and then after the compose window is gone, the memory usage goes back to normal?
does it reproduce when using safe mode? http://support.mozillamessaging.com/en-US/kb/safe-mode

Comment 2

[krichter]

(In reply to Wayne Mery (:wsmwk) from comment #1)
> (In reply to krichter722 from comment #0)
> > started typing an e-mail address when entering recipient in message compose
> > window (three very large addressbooks (14000, 9000 and 18000 addresses))
> > message compose window freezes, very high CPU usage for 5 seconds,
> 
> with the current performance characteristics of large AB, this is to be
> expected.

My proposal is to use more efficient search trees with log(n) performance.

> 
> > allocation of ~450 MB RAM,
> 
> and then after the compose window is gone, the memory usage goes back to
> normal?
> does it reproduce when using safe mode?
> http://support.mozillamessaging.com/en-US/kb/safe-mode

No, the memory isn't freed neither after the search nor after the compose window has been closed, but for further searches no more memory is allocated, only the same high CPU usage and window blocking (semi-memory leak).
The problem doesn't exist in safe mode, but my address books are not available in safe mode.

Comment 3

[Wayne Mery (:wsmwk)]

(In reply to krichter722 from comment #2)
> The problem doesn't exist in safe mode, but my address books are not
> available in safe mode.

where then, are your address books? :)

Comment 4

[krichter]

(In reply to Wayne Mery (:wsmwk) from comment #3)
> (In reply to krichter722 from comment #2)
> > The problem doesn't exist in safe mode, but my address books are not
> > available in safe mode.
> 
> where then, are your address books? :)

I figured out that they're not available in safe mode because they were created with an add-on called MoreFunctionsForAddressbook (0.6.5). So, I created another big addressbook with add-ons disabled and the problem persists.

Comment 5

[Ludovic Hirlimann [:Usul]]

(In reply to krichter722 from comment #2)
> (In reply to Wayne Mery (:wsmwk) from comment #1)
> > (In reply to krichter722 from comment #0)
> > > started typing an e-mail address when entering recipient in message compose
> > > window (three very large addressbooks (14000, 9000 and 18000 addresses))
> > > message compose window freezes, very high CPU usage for 5 seconds,
> > 
> > with the current performance characteristics of large AB, this is to be
> > expected.
> 
> My proposal is to use more efficient search trees with log(n) performance.

Patches would be welcomed.

Comment 6

[Wayne Mery (:wsmwk)]

(In reply to krichter722 from comment #0)
> started typing an e-mail address when entering recipient in message compose
> window (three very large addressbooks (14000, 9000 and 18000 addresses))

41,000 contacts in sum

I thought we had a bug report for this but I'm not finding one.  (perhaps thinking of a bug that standard8 fixed a while back)

Comment 7

[Howard Chu]

41000 records is not "large" by any means. E.g. OpenLDAP slapd can search 1 million records in 1 second using its LMDB backend. Perhaps you should integrate LMDB here as well.

Comment 8

[Mark Sicignano]

I can tell you that with an Address Book with just over 200 entries, and on two machines, both Core i7's with 8 and 16GB respectively, and one having an SSD, that upgrading to Thunderbird 31 introduced this problem. It's slow.

I already described what is happening here: https://support.mozilla.org/en-US/questions/1011952#answer-625689

My address books are local files, and small, yet, I'm constantly faster than the Thunderbird and I am on the next field before it comes up with the matching entries and it's frustrating as hell. I constantly find myself having to go back and edit the previous entry because what used to auto expand quickly is not longer doing it quickly, and won't match, so I am left with the few characters that I know do match an address book entry, like "ryan" in my field. 

I don't think there really should be a debate as to weather or not this is a real issue. It is consistent on both of my machines and consistent with my upgrade to 31 (upgraded at different times, and the problem only showed once at 31) and consistent with many other people saying that this problem came with 31.

Comment 9

[Mark Sicignano]

(In reply to Mark Sicignano from comment #8)

Here is what I'm reporting.

http://screencast.com/t/D38IYyk2

Maybe what I'm reporting is a different bug than the comments above mine, but I hope to shed some light on this problem with the screencast.

Comment 10

[fantasai]

I'm not sure if it matches to the OP, but bug 1151520 seems to match the description in comment 8.

Comment 11

[Wayne Mery (:wsmwk)]

(this wouldn't be OS specific)

Comment 12

[:aceman]

I think we did some changes in the recipient autocomplete area. Can you please retry with TB 52 or later?

Comment 13

[Wayne Mery (:wsmwk)]

(follow up to question in comment 12)

Does your issue reproduce with version 68?

It would also be nice to know how the redesigned AB performs when using beta from https://www.thunderbird.net/#channel

Comment 14

[fantasai]

Hi wsmwk, I have never encountered this bug here (705837). As for bug 1151520, I haven't noticed in awhile. Currently on 68.7.0. However, I tried the 76 beta, and bug 1151520 definitely applies there: autocomplete fails if you press tab or enter too quickly.

Comment 15

[Wayne Mery (:wsmwk)]

autocomplete speed has improved for me with 80.0b3, which means it should also be much better in 78.2.0, due out in two weeks.

Reporter is gone, but I'm going to close this WFM

Comment 16

[Wayne Mery (:wsmwk)]

fantasai,

Thanks for testing. I'd be interested in your results of using 80.0b3 or b4. If it is much better, perhaps we will even be able to close bug 1151520.

If it is still slow, I recommend getting a profile using https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance