589185 - Slow scrolling in Folder Pane after Upgrade from 3.0.6 to 3.1.2, high cpu

Status: RESOLVED WORKSFORME

(Thunderbird :: Folder and Message Lists, defect)

Description

[Jochen Grotepass]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; de-de) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; de; rv:1.9.2.8) Gecko/20100802

After upgrading from TB 3.0.6 to 3.1.2 today, the scrolling performance on the Folder-Pane is extremely slow and bad. CPU usage raises during scrolling up to 50% (MBP 8GB, Intel Core 2 Duo 3 GHz). The delay from initiating the scrolling up to real scrolling is about a second.
There are several folders listed (internal IMAP Server with lots of public folders). Total of 6 accounts. 

The same configuration used until this morning with TB 3.0.6 showed no delay in scrolling.


Reproducible: Always

Steps to Reproduce:
1. Start TB 
2. Scroll in the folder list
3. wait 


Expected Results:  
Scroll without delay as before in TB 3.0.x

Comment 1

[Jochen Grotepass]

Ah, just for the records - there are no errors listed in the error-console. The only "INFORMATION" listed are "..... server does not support RFC 5746, see CVE-2009-3555" for some IMAP Servers.

Comment 2

[Jochen Grotepass]

I tested the patch provided by Bug 562965 because I thought this might help me. After finally getting the patch to work with TB 3.1.2 it seems to work much faster now. 
Maybe this might help in identifying where the initial delay is based.

Comment 3

[Wayne Mery (:wsmwk)]

Jochen, thanks for the great detail. I'm not aware of folder pane scroll speed issues reported against v3.1 that did not occur in v3.0, but perhaps we can use that information.

folder pane scrolling speed problems are typically tied to folders being excessively accessed on disk. 
1. can you reproduce this issue when started in safe mode?  
   https://support.mozillamessaging.com/en-US/kb/Safe+Mode
2. if you can reproduce in safe mode, please provide a short msgdb:5 log
   https://wiki.mozilla.org/MailNews:Logging
   if you can't reproduce in safe mode, you'll need to dtermine what add-on is
   causing the problem

Comment 4

[Jochen Grotepass]

Attachment: MSGDB-Loggile

Misbehavior is reproducable in "save" Mode. MSGDB-Log is attached. I removed the Patch I installed Friday.

Comment 5

[Jochen Grotepass]

Attachment: MSGDB-Logfile with Patch from Bug 562965 applied

Same MSGDB Logfile after applying Patch from Bug 562965.

Comment 6

[Ludovic Hirlimann [:Usul]]

(In reply to comment #2)
> I tested the patch provided by Bug 562965 because I thought this might help me.
> After finally getting the patch to work with TB 3.1.2 it seems to work much
> faster now. 
> Maybe this might help in identifying where the initial delay is based.

Doesn't this sounds like a dup then ?

Comment 7

[Jochen Grotepass]

Not quite sure if this is really a dup. Under TB 3.0.6 (and earlier) I had no issue with scrolling in the folder pane whatsoever. Bug 562965 does, in my opinion, not directly address any issue. It is just a performance enhancement after seeing that some logic spent more time than necessary in certain functions.

To me, there is a change in the behavior between TB 3.0 and 3.1 in the folder pane. Also, after reading the updates on the patch, it seems as if this patch will die, sooner or later. And with the massive scrolling delay and Performance impact on scrolling, TB is becoming not usable any longer for me and my company.

Comment 8

[Robert Kisteleki]

I concur with OP, and have the same issue. I can confirm that this was not an issue in TB 3.0.x, but 3.1.x is very sluggish when scrolling in the folders bar. It takes about a sec to respond to the wheel movement. So if the above patch is supposed to solve an issue that was already present in 3.0.x, then it's not the solution to this issue.

(I have two accounts enabled, a total of about 100 folders listed. Mac OSX 10.6.4, 8GB RAM.)

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3

Comment 9

[Wayne Mery (:wsmwk)]

The quickest way to identify the cause may be to find a regression range, when the issue first appears. Can you reproduce this issue in 3.1(.0) or any of the early shipments of 3.1 - 3.1a1 or 3.1b1 - found at ftp://ftp.mozilla.org/pub/thunderbird/releases/ ?

Dossy, can you reproduce?

Comment 10

[Jochen Grotepass]

OK, I tested all from 3.0.6 up to 3.1.3 on my Mac. The first time the slow scrolling occured was in 3.1.b2.

I will attach the msgdb logfile from 3.1.b1 and 3.1.b2 to show the difference.
Also I will look inside the code to find the differences as soon as I have some minutes spare ...

Comment 11

[Jochen Grotepass]

Attachment: MSGDB Log with Version 3.1.b1

Last MSGDB Log without scrolling issue 
Due to the size of the log - Zipped with gzip.

Comment 12

[Jochen Grotepass]

Attachment: MSGDB Log with Version 3.1.b2

MSGDB Logfile for Version 3.1.b2 - First version with the scrolling delay/issue

Comment 13

[Wayne Mery (:wsmwk)]

Thanks for the logs. MSGDB Log with Version 3.1.b2 maxes at 6 open DBs, so the slowness is not related excessive memory from too many open DBs

you'll want to narrow the regression range to within a day or two of when it stopped working. Using the -comm-central nightly builds between dec 9 and feb 26 (per https://wiki.mozilla.org/Releases/Old ) at ftp://ftp.mozilla.org/pub/thunderbird/nightly/2008/12/  /2009/01/  /2009/02/

Comment 14

[Wayne Mery (:wsmwk)]

Robert, does you issue also happen started in safe mode?  And match up to Jochen's regression range of 3.1b1 (~march 1 source) to 3.1b2 (may 5)? 

revising comment 13... you'll want to narrow the regression range to within a day or two of when it stopped working. Using the -comm-1.9.2 nightly builds between march 1 and May 5 (ref https://wiki.mozilla.org/Releases/ ) at
ftp://ftp.mozilla.org/pub/thunderbird/nightly/2010/03/  /2010/04/  /2010/05/

Comment 15

[David :Bienvenu]

(In reply to comment #13)
> Thanks for the logs. MSGDB Log with Version 3.1.b2 maxes at 6 open DBs, so the
> slowness is not related excessive memory from too many open DBs

A related possibility is that scrolling/painting requires us to open the dbs for each folder at paint time. This might not result in a lot of db's remaining open, but it would show a lot of db's getting opened, and perhaps closed. I don't see that in the log, but it should be mentioned for completeness sake.

Comment 16

[Jochen Grotepass]

(In reply to comment #14)
> Robert, does you issue also happen started in safe mode?  And match up to
> Jochen's regression range of 3.1b1 (~march 1 source) to 3.1b2 (may 5)? 
> 
> revising comment 13... you'll want to narrow the regression range to within a
> day or two of when it stopped working. Using the -comm-1.9.2 nightly builds
> between march 1 and May 5 (ref https://wiki.mozilla.org/Releases/ ) at
> ftp://ftp.mozilla.org/pub/thunderbird/nightly/2010/03/  /2010/04/  /2010/05/

So I finally found the time to test the nightly builds. The first version that showed the slow scrolling issue was on 2010-04-21. Nightly builds before this date (up to and including 2010-04-20) were OK.

Hope this helps to narrow the issue down.

Comment 17

[Ludovic Hirlimann [:Usul]]

mozilla central https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2010-04-20&enddate=2010-04-21

comm-central https://hg.mozilla.org/comm-central/pushloghtml?startdate=2010-04-20&enddate=2010-04-21

bug 553747 looks like a nice candidate.

Comment 18

[Wayne Mery (:wsmwk)]

Jochen, still see this issue?
No regressions marked yet against bug 553747

Comment 19

[Jochen Grotepass]

Nothing has changed yet - or did I oversee anything here in the last comment?
I still have this delay and cannot use the 3.1 version without the patch mentioned earlier. However this patch did no longer work for the newer 3.1 versions - so I am sitting here with an 3.1.2 and hope that something happens. There are currently 9 Macs and at least 5 Windows XP Systems here in the office waiting for a solution. We have blocked all updates so far because of the delayed scrolling. 

Sorry, nothing new.

Comment 20

[Wayne Mery (:wsmwk)]

Jochen is correct, nothing happened in this bug. Or in Bug 562965 (which I think Jochen indicated only provides minor improvement?)  

So part of the challenge is, there is almost no one to test for us except you and Robert. [1]

Robert, Jochen, I'd like to suggest a test of TB3.3 alpha 2. Testing alpha 2 will quickly help determine if the state of the art code is improved (after all, the code base of TB3.1 is over a year old).  Can you test http://www.mozillamessaging.com/en-US/thunderbird/3.3a2/releasenotes/ ?  

If it works much better then we have something to look forward to. And if it doesn't, then we know we have more digging to do to get a fix. (code profile for example)


[1] slow scroll
- gfsn, nothing useful found: http://www.google.com/search?hl=en&client=firefox-a&hs=nho&rls=org.mozilla%3Aen-US%3Aunofficial&q=slow+scroll+site%3A+http%3A%2F%2Fgetsatisfaction.com%2Fmozilla_messaging%2F&aq=f&aqi=&aql=&oq=
- bugzilla, nothing useful found: https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring&short_desc=scroll&field0-0-0=longdesc&bug_severity=major&bug_severity=normal&bug_severity=minor&type0-0-1=substring&field0-0-1=product&type1-0-1=allwordssubstr&resolution=---&value1-0-1=perf&classification=Client%20Software&classification=Components&chfieldto=Now&query_format=advanced&chfieldfrom=2y&value1-0-0=slow&short_desc_type=allwordssubstr&value0-0-1=thunderbird&type0-0-0=anywordssubstr&value0-0-0=thunderbird%20tbird&field1-0-0=short_desc&field1-0-1=keywords
 ** bug 482931 indicates tbird is OK for that use case
 ** bug 356201 is not a version 3 regression

Comment 21

[Wayne Mery (:wsmwk)]

(In reply to comment #20)
> Robert, Jochen, I'd like to suggest a test of TB3.3 alpha 2. Testing alpha 2
> will quickly help determine if the state of the art code is improved (after
> all, the code base of TB3.1 is over a year old).  Can you test
> http://www.mozillamessaging.com/en-US/thunderbird/3.3a2/releasenotes/ ?  

I forgot to caution that this is development code - so caution, controlled environment and backups of your thunderbird profile are appropriate to minimize risk of testing development code.

Comment 22

[Wayne Mery (:wsmwk)]

this is slow scrolling with mouse click, correct? 

(and in no way related to Bug 356201 - message-list pane: slow mousewheel scrolling and scroll bar arrows, high cpu)

Comment 23

[Robert Kisteleki]

No, it's not only with click, it's also with wheels. Nothing has change about this since my last entry: using 3.1.7 on Mac OSX 10.6.6, with roughly 100 entries open in the tab I still experience:
* when clicking the down arrow: nothing happens for about .5 sec, then the list scrolls down three times with about .2 sec intervals in between
* when clicking on the scroll bar: nothing happens for about .5 sec, then the bar jumps one page down
* when scrolling down with the wheel: nothing happens for about .5 sec, then it begins to scroll, in two-three distinct steps

Very annoying. Since TB handles all my mails now, I'm not really enthusiastic about trying out alpha versions...

Robert

Comment 24

[Jochen Grotepass]

(In reply to comment #20)
> Robert, Jochen, I'd like to suggest a test of TB3.3 alpha 2. Testing alpha 2
> will quickly help determine if the state of the art code is improved (after
> all, the code base of TB3.1 is over a year old).  Can you test
> http://www.mozillamessaging.com/en-US/thunderbird/3.3a2/releasenotes/ ?  
> 
> If it works much better then we have something to look forward to. And if it
> doesn't, then we know we have more digging to do to get a fix. (code profile
> for example)

I tested TB3.3 alpha 2 and this works perfect (I am only lacking Lightning ;) )

(In reply to comment #22)
> this is slow scrolling with mouse click, correct? 
> 
> (and in no way related to Bug 356201 - message-list pane: slow mousewheel
> scrolling and scroll bar arrows, high cpu)

The delay happens all the time - either mousewheel or by clicking.

I also tested 3.1.7 today with a complete fresh installation - just in case there is a migration issue or something like this. Same issue - slow scrolling, delays, high cpu. 

To me, TB3.3 solves this issue. 

Anything else I can test?

Comment 25

[Wayne Mery (:wsmwk)]

> Anything else I can test?

nope. thanks for testing that!  Closing per your results

regarding lightning with trunk build, see 
https://addons.mozilla.org/en-US/thunderbird/addon/lightning-nightly-updater-unof/ 
and
http://weblogs.mozillazine.org/calendar/2011/01/lightning_maclinux_64_bit_nigh.html