351867 - Saved Search/Local Folder Unread view issues

Status: RESOLVED FIXED

(Thunderbird :: Mail Window Front End, defect)

Description

[Bob Clary [:bc] (inactive)]

Steps to reproduce: (this is my setup, you can use a similar one I'm sure)

1. Create a local folder for bugmail called bugzilla and a filter to move all of your bugmail there. Set the view for the folder to Unread. Click the sort by date view column.

2. Create a saved search folder on a bugzilla product header, e.g. X-Bugzilla-Product: Thunderbird called bugzilla-thunderbird. Set the view on the folder to Unread. Click the sort by date view column.

3. Click on a different local folder (not bugzilla or bugzilla-thunderbird).

4. Wait for new bugmail (both Thunderbird and non-Thunderbird) to arrive and be moved to the local bugzilla folder. The unread counter on the bugzilla folder and the bugzilla-thunderbird saved search folder will update to show new mail.

5. Click the bugzilla-thunderbird saved search folder and no mail will be displayed in the list pane. Click the bugzilla local folder and the new bugmail for Thunderbird will be displayed but not any bugmail for other products.  Click on a few of the bugmails and let them be marked read.

6. Click on a different local folder (not bugzilla or bugzilla-thunderbird), then click on the bugzilla folder. All unread mail will be displayed including non-Thunderbird bugmail. Now mark all of the mail in the bugzilla folder as read.

7. Click on the bugzilla-thunderbird saved search folder and click on the threaded view column.

8. Select a different folder (not bugzilla or bugzilla-thunderbird) and wait for new Thunderbird bugmail for bugs that you have already received.

9. Click the bugzilla-thunderbird saved search folder and you will see a previously read bugmail in the list but not the new unread bugmail.

10. While keeping the bugzilla-thunderbird folder selected, wait for new mail. You will see the new mail displayed in the list without problem.

This is a major pita for me. Your mileage may vary.

Comment 1

[David :Bienvenu]

I'll try to look at this this weekend...

Comment 2

[David :Bienvenu]

Attachment: fix issues with search caching

this fixes some issues with search caching, which might be responsible for what you're seeing. I'm going to run with this for a bit more to see if there are still issues, and if it works OK, request review...

Comment 3

[David :Bienvenu]

Comment on attachment 238024 [details] [diff] [review]
fix issues with search caching

this fixes a couple issues with caching of cross-folder search results, specifically when trying to remove invalid hits. One had to do with a new header getting added to a virtual folder while the search is going on (the m_doingSearch part) and the other was a more general problem keeping track of the folders we're searching over.

Comment 4

[David :Bienvenu]

Attachment: patch checked in.

this is the patch I'll check in - I fixed on more problem having to do with headers that arrive while we're executing the search.

Comment 5

[David :Bienvenu]

Bob, is this saved search just over a single folder, and not multiple folders? If so, then the patch I checked in won't actually help. I'll try to duplicate your saved search on my system and see what happens...

Comment 6

[Bob Clary [:bc] (inactive)]

(In reply to comment #5)

This particular issue is for saved searches on a single local folder containing bugmail.

Comment 7

[David :Bienvenu]

I may have been wrong about this patch not helping, since the code I changed does seem to get invoked. But leaving that aside for a minute:

In your steps to reproduce, things start going wrong in step 5, right? You should see the new mail in the bugzilla-thunderbird saved search folder, and you should see all messages in the bugzilla folder? 

The first thing I noticed is that the bugzilla-thunderbird folder doesn't remember that it should be using the unread view (that's your intent, right, that it stay in the unread view?)

Comment 8

[David :Bienvenu]

if your bugzilla folder has any sub-folders,  by default they will be included in the first search, and thus your saved search folder will be a cross-folder search. That's why I was hitting the code I changed.

Comment 9

[David :Bienvenu]

Also, FYI, because of the way we're caching saved search results, you might be happier with a separate saved search folder that shows unread thunderbird bugzilla bugs, instead of using a thunderbird bugzilla saved search and using the unread view. That's going to perform a lot better...what's happening with your setup is that we're caching all the thunderbird-buzilla bugs, and then applying the unread view to that - if you had an unread thunderbird buzilla saved search folder, opening it should be really fast. Not that this isn't a bug...though I am having trouble reproducing it.

Comment 10

[Bob Clary [:bc] (inactive)]

(In reply to comment #7)
> In your steps to reproduce, things start going wrong in step 5, right? You
> should see the new mail in the bugzilla-thunderbird saved search folder, and
> you should see all messages in the bugzilla folder? 

yes.

> 
> The first thing I noticed is that the bugzilla-thunderbird folder doesn't
> remember that it should be using the unread view (that's your intent, right,
> that it stay in the unread view?)
> 

Yes, that is my intent and it actually does stay in Unread view for me on winxp and today's build. I don't think I saw any behavior about it losing the Unread view status before.

(In reply to comment #8)
> if your bugzilla folder has any sub-folders,  by default they will be included
> in the first search, and thus your saved search folder will be a cross-folder
> search. That's why I was hitting the code I changed.
> 

No subfolders.

(In reply to comment #9)
> Also, FYI, because of the way we're caching saved search results, you might be
> happier with a separate saved search folder that shows unread thunderbird
> bugzilla bugs, 

Ok, I can play around with that and with customized views so I can quickly see all Thunderbird bugs.

The reason I am singling out Thunderbird here, is that I am a regular dogfooder but could never get a feel for what was known or not known about the issues I see, so I started watching before beginning to file bugs. ;-)

On an unrelated topic, how hard would it be to add customized header views? Normal doesn't show me bugzilla headers (which I would love) and All shows too much to be useful for regular use. Would you mind a bug for request for enhancement for that?

Comment 11

[David :Bienvenu]

>Yes, that is my intent and it actually does stay in Unread view for me on winxp
>and today's build. I don't think I saw any behavior about it losing the Unread
>view status before.

I only saw that when I accidentally had a cross-folder saved search, and I think it only happened once. Once I got rid of the sub-folders in the search scope, it hasn't forgotten the unread view.

I'll keep trying to reproduce this. So far, I'm not getting things to go wrong in step 5...are you in threaded view in the saved search folder, or just sorted by date?

>On an unrelated topic, how hard would it be to add customized header views?
I thought someone had done that, but I don't see it in the code. Scott, do you know if there's a patch out there? This would seem to be an oft-requested enhancement. It would be fairly trivial to add a pref for custom hdr view headers, and check for it in nsMimeHtmlDisplayEmitter::BroadcastHeaders and nsMsgHdrViewOverlay.js's initializeHeaderViewTables method.  Perhaps one pref for gCollapsedHeaderView and one for gExpandedHeaderView. The prefs could even have reasonable default values so I wouldn't have to remember to change both places when trying to add headers...

Comment 12

[Bob Clary [:bc] (inactive)]

(In reply to comment #11)

> I'll keep trying to reproduce this. So far, I'm not getting things to go wrong
> in step 5...are you in threaded view in the saved search folder, or just sorted
> by date?
> 

In step 5 it is sorted by date. I see a different problem in step 9 when sorted by thread where one of the read ancestor mails (Parent?) in the thread displays instead of the new unread one.

Would my other saved searches or accounts have any affect? I have 5 different saved searches on the bugzilla local folder...

> >On an unrelated topic, how hard would it be to add customized header views?
> I thought someone had done that, but I don't see it in the code. Scott, do you
> know if there's a patch out there? This would seem to be an oft-requested
> enhancement. 

That would be sooo cool. :-)

Comment 13

[David :Bienvenu]

OK, I see what's going on in step 5 - the problem is a bit complicated but the root problem is that the quick search view is a search listener, as is the front end js code. When we open a saved search, the front end js code waits until it gets a notification that the search is done, and then issues a new search to apply the default view. Issuing a new search adds the view as a listener on the search session, but the search session is in the process of looping through listeners telling them that the search is done. So the new listener gets notified that its search is done, even though it really hasn't started yet. This confuses the quick search view caching code, and results in incorrect views. There are a few possible solutions - the hacky one is to have the front end js code launch the new search asynchronously with a setTimeout of 0. An other solution is to make a copy of the listeners before sending out the OnSearchDone notification. There are probably others...

Comment 14

[David :Bienvenu]

Attachment: proposed fix

I went for the hacky fix, but well-commented :-) fixing it in the backend would have been pretty hacky too...

Comment 15

[David :Bienvenu]

Bob, you should file an RFE for the ability to customize the headers we can display. Oren might be interested in doing this.

Comment 16

[Bob Clary [:bc] (inactive)]

(In reply to comment #15)
Bug 353193

Comment 17

[Scott MacGregor]

Comment on attachment 238859 [details] [diff] [review]
proposed fix

What does '+result+' do? I haven't seen that syntax before.

Comment 18

[David :Bienvenu]

I'm just concatenating the strings together to create the appropriate string to execute when the timeout fires. I'm not sure what context the js that executes the string in the timeout callback runs in but this seems to work fine. 

Comment 19

[David :Bienvenu]

fixed on trunk and branch - I think this bug would account for all the problems you were seeing, but if any of the problems remain, let me know, and we can figure out if we should re-open this bug, or open a new one.

Comment 20

[Bob Clary [:bc] (inactive)]

Ok, it looks better in today's build. Setting both the bugzilla and bugzilla-thunderbird folders's view to unread will allow you to see the unread mail as it arrives just fine.

But it's not completely fixed. 

Set the bugzilla folder's view to unread and the bugzilla-thunderbird folder's view to unread. Then wait for bugmail for both thunderbird and non-thunderbird to arrive. The bugzilla-thunderbird saved search will show the count and if you click on the folder it will (after some hokey-pokey in the view) show the unread mail in the message list. Then click on the bugzilla folder. You will only see the thunderbird bugmail and not the other. To reproduce this you have to not have either folder selected when the new bugmail arrives.

Comment 21

[Bob Clary [:bc] (inactive)]

Another issue (minor nit) with the current fix:

1. set a folder's view to unread
2. select a different folder.
3. wait for new mail to arrive in the folder.
4. select the folder and press 'n' to go to the next message
5. the unread message will display in the preview pane and if you have the mark as read set to a short time, then the message will be marked read before the message list pane is updated.
6. the message list pane will be redrawn with the currently previewed mail missing since it is already marked read.

Comment 22

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #19)
> fixed on trunk and branch

Several problems seems to still remain or be newly created or be exposed.
Following is test result with SeaMonkey trunk-2006092308/Win-2K.
(1) View for a virtual folder seems to be changed to "All" internally always.
     1. Set View of virtual folder to "Last 5 days"
     2. Click usual folder of View="Unread"
     3. Click the virtual folder again
        => View selection says "View=Unread" (One for usual folder)
           But display of mail list pane looks to be View=All
(2) Unread mail count seems to be counted multiple times
     1. Virtual folder is search for 3 usual folders
        Both virtual and usual folder can be seen same time in folder pane
     2. Change a mail in usual folder to "Unread" status
     3. Unread mail count of the virtual folder in folder pane becomes 3.
        (When number of target folders for search was 2, it became 2)
        This can be observed in other mail&news window, if the virtual folder
        is displayed in the other mail&news window. 
     4. When the virtual folder is clicked, unread mail count is updated to 1,
        after refreshing of view in mail list pane.
        (You can see (1) same time, if View of usual folder is different)

Should I open new bugs?
 

Comment 23

[WADA:World Anti-bad-Duping Agency]

Bug 372372 seems to be a report of phenomenon of (2) in my Comment #22(double counting) for Thunderbird 2.0.
But phenomenon of (2) in Comment #22 didn't occur with Tb latest-trunk(2007/8/22 build).
Similar phenomenon to (1) was still observed with trunk 2007/8/22 build. But View related design is already changed. So I think separate bugs should be opened for new problems.
=> Changing back to FIXED from REOPEN.

Comment 24

[WADA:World Anti-bad-Duping Agency]

FYI.
Phenomenon of my Comment #22 couldn't be observed with Thunderbird 2.0.0.6.