363831 - Saved Search with only one folder to search results in endless loading and high CPU load

Status: VERIFIED FIXED

(Thunderbird :: General, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

version 2 beta 1 (20061213)

We have a regression in Saved Search aka Virtual Folders. Starting with the build 20061213 Thunderbird never stops loading the message list / message from within such a folder.

For my IMAP account I have a Saved Search for spam report messages. Opening the folder Thunderbird starts updating the status again and again. Selecting a messages changes the toolbar icons from enabled to disabled and back - again and again.

Doing the same with the nightly build 2006-12-12-04-mozilla1.8 doesn't show this behavior.

Comment 1

[David :Bienvenu]

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

IMAP protocol log? Saved searches over imap are working fine for me...

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: IMAP log

Oh sorry, I forgot. Here the IMAP log.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: msf file of IMAP folder

Comment 4

[David :Bienvenu]

We're getting a connection error trying to download that message, and we're retrying - is this an SSL connection? Nothing has changed in the imap code in this time frame, though perhaps some networking/ssl/nspr code has changed.

I don't think this is related to saved folders, or the .msf file...

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #4)
> We're getting a connection error trying to download that message, and we're
> retrying - is this an SSL connection? Nothing has changed in the imap code in
> this time frame, though perhaps some networking/ssl/nspr code has changed.

No, it is wether a SSL or TLS connection. The thing is if I use a build from 20061212 with the same profile it is working. Only builds starting from yesterday are showing the endless loading.

> I don't think this is related to saved folders, or the .msf file...

What I've seen now, if I switch to the real folder where the message is located it is loaded without any problem. So it has to be related to saved folders.

Comment 6

[David :Bienvenu]

does it happen to every imap message, or just a particular message?

Perhaps we're interrupting the message load, i.e., the load group, for some reason. 

Do you have any extensions installed?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #6)
> does it happen to every imap message, or just a particular message?

All messages inside a saved search folder. If I only open such a folder (old one or fresh created) the status bar shows following:

Searching...
6 matches found
Searching...
6 matches found
Searching...
6 matches found

and so on. When I change the focus to another program and return to TB it only shows "Searching" but Thunderbird consumes 100% cpu load. Switching to other folders doesn't leave it. I have to close Thunderbird to get a normal cpu load.

> Do you have any extensions installed?

It still happens in Safe Mode. That's the first thing I try before filing a new bug. 

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Same happens for Trunk build version 3 alpha 1 (20061214).

Only opening this folder causes Thunderbird to eat 100% cpu. Switching to another folder reduces it to ~50% but after switching to another application and back stays at 100%.

Comment 9

[David :Bienvenu]

nasty - this isn't happening to me, at least with my debug build on a mac.

Are you in a view in the saved search folder, or is it viewing all messages?

Does this happen for all saved searches, or just this particular one? Is the saved search on a single folder, or across folders?

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #9)
> nasty - this isn't happening to me, at least with my debug build on a mac.

I can reproduce it with any of my IMAP accounts on different computers. Perhaps a windows only issue?

> Are you in a view in the saved search folder, or is it viewing all messages?

Like the msf file tells only messages with a special subject are shown. I will play around with different filter criteria.

> Does this happen for all saved searches, or just this particular one?

All saved searches for an IMAP account. Local Folders is not affected.

> Is the saved search on a single folder, or across folders?

This saved search is configured to a single folder.

Steps:

1. Expand your IMAP account
2. Right click on the Inbox of your IMAP and choose "Search"
3. Press "Save as Search Folder"
4. Specify a name for it and set as search criteria "Subject" is "Spam"
5. Save it as a subfolder of Inbox inside your IMAP account

If you select this saved search now from within the folder pane Thunderbird goes crazy and consumes all cpu power.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This bug happens if a Saved Search is configured to search on one folder only.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Could the fix on bug 331014 have been raised this issue?

Comment 13

[David :Bienvenu]

I doubt it - my guess is bug 362741

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #13)
> I doubt it - my guess is bug 362741

Yes it is. After reverting the patch on bug 362741 the issue is gone. Especially the fix in msgViewPickerOverlay.js causes the endless loop.

Comment 15

[David :Bienvenu]

removing IMAP from the summary. Local folder saved searches have a similar issue - the search is (re)executed continuously...

Comment 16

[Carsten Book [:Tomcat]]

*** Bug 363919 has been marked as a duplicate of this bug. ***

Comment 17

[Magnus Melin [:mkmelin]]

I guess this is what i'm seeing too (using todays 2.0 nightly/linux). My saved search is stored in an imap folder, the search searches the inbox of a pop account for subject contains "test". 

The messages in the folder are just jumping around, and the status bar says the Searching... x matches found. Searching... x matches found in a loop until I deselect the folder (after which everything goes back to normal, it seems).

Comment 18

[Karsten Düsterloh]

Attachment: fix reentrancy

The core problem was the call of ViewChange in the onSearchDone handler. It's quite unnecessary now that we don't quit prematurely in ViewChange - the view code makes sure all relevant folder search terms are applied before doing the actual search. 
(I also found that cross folder saved searches don't remember their view and don't update the view picker - that's the fix in commandglue.js.)

So, the /mail changes plus the msgViewPicker.js changes fix this problem for TB, the rest are the equivalent changes for SM; I had to incorporate the patch from bug 265548 since it was missing in SM.

Comment 19

[Karsten Düsterloh]

Comment on attachment 248875 [details] [diff] [review]
fix reentrancy

I forgot: this patch is against 1.8 branch, but should apply to trunk almost unchanged.

Comment 20

[David :Bienvenu]

I tried this patch on TB 2.0  - unless I'm missing something, it doesn't help at all with the case where the quick search widget is not on the toolbar :-(

Comment 21

[David :Bienvenu]

this patch also seems to have broken cross-folder saved searches for me...at least when the search bar is not on the toolbar, though I didn't try it with the search bar present.

So something's pretty wrong with the TB changes - I don't see anything interesting on the error console.

Comment 22

[[e-mail]]

I can't seem to reproduce this on current trunk. Here's what I tried:
1. Select a real folder
2. Select the "Unread" view
3. Save the view as a search folder
4. Select the saved search folder

Comment 23

[Karsten Düsterloh]

@David:
I did an update to current 1.8 branch and did a diff over whole /mail and /mailnews to ensure only my patch was used - and it still works for me (Mac G4 PowerBook under Mac OS X 10.4.8)! It works with and without Quick Search box shown. I backed my patch out and the endless loop was back. Patched again and the bug was fixed again. 
WRT cross-folder saved searches: they still seem to work for me as they should and even do remember the view and last selected message now.

My current tb18.cfg:
#
# See http://www.mozilla.org/build/unix.html for build instructions.
#
. $topsrcdir/mail/config/mozconfig
CC=gcc
CXX=g++
CPP=cpp
AS=as
LD=ld
# Options for client.mk.
mk_add_options MOZ_OBJDIR=/Users/karsten/Projekte/Mozilla/obj/tb2
mk_add_options MOZ_CO_PROJECT=suite,mail,browser,calendar,composer,xulrunner
# Options for 'configure' (same as command-line options).
ac_add_options --enable-application=mail
ac_add_options --enable-debug
ac_add_options --disable-optimize
ac_add_options --enable-chrome-format=flat
ac_add_options --enable-crypto
ac_add_options --enable-extensions=all,venkman
ac_add_options --disable-installer
ac_add_options --enable-official_branding

@Neil:
SM does not have the problem this bug is primarily about, since we (a) don't allow customizing the toolbar yet and (b) don't have the patch <https://bugzilla.mozilla.org/attachment.cgi?id=238859>. The SM fixes are mostly compatibility changes to the TB changes which coincidentally fix remembering last view and last selected message for single folder and cross folder saved searches.

Comment 24

[[e-mail]]

Comment on attachment 248875 [details] [diff] [review]
fix reentrancy

Note that neither with nor without this patch can I get views to work on multiple folder saved searches.

Comment 25

[David :Bienvenu]

If I put back the quick search bar, this patch works fine...I tried backing it out again, things worked fine, I reapplied it, things were broken again. I'll try to figure out why things aren't working with the quick search bar removed. 

Comment 26

[David :Bienvenu]

actually, the problem loading the cross-folder saved search is intermittent - if I click around enough, I can eventually get it to load. But, this would still be a serious regression, since I use the cross-folder saved search a lot.

Comment 27

[David :Bienvenu]

Comment on attachment 248875 [details] [diff] [review]
fix reentrancy

my bad, I still had a change in mailviews that I didn't notice...thx for the patch!

Comment 28

[David :Bienvenu]

OK, can you see if this works for you? :

Setup a saved search that searches over an imap folder, e.g., your imap inbox and a couple local folders.
Click the saved search.
Click the imap inbox

For me, nothing shows up in the imap inbox until I click away to an other imap folder and back to the imap inbox.

Perhaps something else is screwed up in my tree, but I think I've got your patch applied correctly, finally!

Comment 29

[Karsten Düsterloh]

Attachment: fix IMAP issue

Since cross folder searches now take the whole road through ViewChange, its view is remembered - which we don't want (and didn't do before)...
Taking over r+, rerequesting sr.

Comment 30

[David :Bienvenu]

Comment on attachment 249041 [details] [diff] [review]
fix IMAP issue

thx, Karsten.

Comment 31

[Karsten Düsterloh]

Comment on attachment 249041 [details] [diff] [review]
fix IMAP issue

Landed on trunk.

Comment 32

[Karsten Düsterloh]

Pseudo-requesting SM 1.1 approval, if we have time to let this bake a few days on trunk before. Since SM is not affected by the actual bug reported here, we can stall this until after 1.1, but we need to take this due to changed shared code then.

Comment 33

[Scott MacGregor]

Comment on attachment 249041 [details] [diff] [review]
fix IMAP issue

Normally, I'd suggest we let this bake a little more, but the symptom is kind of painful on the branch so I'd like to be a bit more agressive about landing this sooner rather than later.

Comment 34

[sathallrin]

This is fixed for me now as of version 3 alpha 1 (20061220)

Comment 35

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Broken with version 3 alpha 1 (20061216). After that I've done a software update to version 3 alpha 1 (20061220) my Saved Search folder is not visible! Trying to create the same Search Folder again tells me that a folder with the name exists. What happens here? Tested on an IMAP account.

Comment 36

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #35)
> update to version 3 alpha 1 (20061220) my Saved Search folder is not visible!
> Trying to create the same Search Folder again tells me that a folder with the
> name exists. What happens here? Tested on an IMAP account.

After collapsing and expanding the account my Saved Search is visible again and no endless loading is visible.

v.

Comment 37

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #35)
> Broken with version 3 alpha 1 (20061216). After that I've done a software
> update to version 3 alpha 1 (20061220) my Saved Search folder is not visible!

This issue is covered in bug 364519. Dunno if this patch raises the new problem.

Comment 38

[Ian Neal]

a-SM1.1=me

Comment 39

[Karsten Düsterloh]

Landed on MOZILLA_1_8_BRANCH.

Comment 40

[[e-mail]]

Comment on attachment 249041 [details] [diff] [review]
fix IMAP issue

>              if (!scrolled && pref.getBoolPref("mailnews.scroll_to_new_message"))
>+               scrolled = ScrollToMessageAfterFolderLoad(msgFolder);
As discussed on IRC, this condition is erroneous and should be removed.

Comment 41

[Karsten Düsterloh]

Attachment: SeaMonkey regression band-aid fix

Band-aid fix for a SeaMonkey-only regression: remember last selected message doesn't work if scroll_to_new_message is set to false.
This patch just removes the erroneous if-condition, not the scrolled-var-weirdness (see bug 365049 for that issue).

Comment 42

[Karsten Düsterloh]

Requesting SM 1.1 approval for regression fix.

Comment 43

[[e-mail]]

Comment on attachment 249780 [details] [diff] [review]
SeaMonkey regression band-aid fix

>+             scrolled = ScrollToMessageAfterFolderLoad(msgFolder);
Nit: we never actually use this variable any more.

Comment 44

[Karsten Düsterloh]

Comment on attachment 249780 [details] [diff] [review]
SeaMonkey regression band-aid fix

Landed on trunk as is; the nit will be part of bug 365049.

Comment 45

[Ian Neal]

a-SM1.1=me

Comment 46

[Karsten Düsterloh]

Comment on attachment 249780 [details] [diff] [review]
SeaMonkey regression band-aid fix

Landed on MOZILLA_1_8_BRANCH.

Comment 47

[lijuan wang]

Attachment: detail:Segmentation Fault - core dumped

Comment 48

[lijuan wang]

Reproduced on version 2 beta 2 (20070326)

Steps:

1. Expand your IMAP account
2. File->New->Saved search
3. Select 'inbox' folder to search and saved in 'search' folder
4. Set a subject to match
5. Click ‘OK‘
Thunderbird goes crazy and consumes all cpu power.

6.Restart thunderbird 
7.Click on 'search' folder
Words 'Searching...' is showed on the status bar.Thunderbird goes crazy again.

Comment 49

[Magnus Melin [:mkmelin]]

lijuan: this landed on branch 2006-12-30 so the fix is not in beta2. Try a recent build like the 2.0 release candidates.

Comment 50

[lijuan wang]

version 3 alpha 1 (20070401)
Search successful.

But a new problem:
1.Set a saved search folder as 'search'
2.when name a new saved search folder as 'search1',pop out a 'JavaScript application' dialog:'a folder with that name is already exists.Please enter a different name'
3.Change 'search1' as 'search 1',it will success.

Comment 51

[Karsten Düsterloh]

> But a new problem:

Please file new problems as new bugs, especially if they are unrelated to this bug. TIA.