267997 - Virtual folders/saved search should have a 'refresh' functionality

Status: NEW

(Thunderbird :: Search, enhancement)

Description

[Mahesh Asolkar]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041103 Firefox/1.0RC2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041103 Firefox/1.0RC2

I created a virtual folder by saving a search where "age" "is" "0" (Today's
messages). When new messages arrive, I could not find a way to update/refresh
the Today folder to see the newly received messages.

When I start Thunderbird tomorrow, I want to see Today's messages from tomorrow.

Grouping by date doesn't work here because message filters route received
messages into different folders.

I think there should be a refresh/update sort of functionality associated with
virtual folders.

Reproducible: Always
Steps to Reproduce:
1. Create a virtual folder, say 'Today' (search criteria: "age" "is" "0")
2. Wait for new messages to arrive

Actual Results:  
The new messages don't show up in Today folder, and there is no way to refresh
the virtual folder contents


Expected Results:  
There should be a refresh functionality associated with virtual folders. May be
a context menu item.

Unless, virtual folders are auto-refreshed after every 'get message' action

Comment 1

[Marcia Knous [:marcia]]

Reporter: Not seeing this on today's branch build of Thunderbird.  Can you
please retest with a more recent build? I see new messages that meet my virtual
folder criteria being dropped into the appropriate folder after hitting 'Get new
messages" 

Comment 2

[WADA:World Anti-bad-Duping Agency]

*** Bug 270448 has been marked as a duplicate of this bug. ***

Comment 3

[Danny Bloemendaal]

(In reply to comment #1)
> Reporter: Not seeing this on today's branch build of Thunderbird.  Can you
> please retest with a more recent build? I see new messages that meet my virtual
> folder criteria being dropped into the appropriate folder after hitting 'Get new
> messages" 

I tried the nightly build from yesterday (apr.14 2005) and search folders still
do not work as they should:

I have an inbox rule that moves mail to a different folder (folderX, read status
isn't affected). That works. This folderX is marked to be checked for new mail
and you can see that new mail arrives in this folder. Also, I have a search
folder that shows unread mail for several folders (inbox, folderX). Usually I
have the focus on this 'unread' search folder. 

When new mail arrives in folderX, I can see that it's label is updated and show
the number of new items between brackets. However, the search folder doesn't
change. It is not updated. If I move the focus to another folder (not folderX)
and then back to the search folder, it is still not updated. But, when I move
the focus to folderX, and when the new items are shown for folderX, then the
search folder's label is adjusted and the item count between the brackets is
updated but only with the amount of the new items in folderX. If other folders
also have new items that should show up in the search folder, then these are NOT
counted and shown. I have to specifically go through all these folders one by
one. Basically eliminating the advantage of the search folder.

To summarize: all search folders need to be updated EVERY time new mail is
retrieved in any folder that is marked to be checked for new mail. If that is
too expensive, then at least the search folder having the focus needs to be
updated with each check.

Comment 4

[ovidiu]

Is this still on? 
from summary I thought that is same as Bug 365393 though seams more like a more deep issue. If the case is only about refresh on get new mail, that is the same bug, if it doesn't refresh even if leave and back to folder it is probably a "new" flag issue and there are a bunch of bugs some in work.

Comment 5

[Patrick]

I would like to have this bug fixed for the following use case:

I have a virtual (saved search) folder which displays all tagged messages in a specific folder - the "spool folder" in my case. This folder basically equals my "todo" list. I tag only messages that I have not yet dealt with. As I compete one task - represented by a tagged message, I remove the tag from that message. On that occation I would very much like the message to immediately disappear from that folder. This, however, currently doesn't happen. The only way I found to make it disappear, is to select another folder and then select the virtual (saved search) folder again. This is a bit cumbersome, as one might agree, because it requires two clicks. I would like (1. priority) the message to disappear automatically, as soon as the tag is removed or (2. priority) to have a keyboard shortcut available to refresh the safed search view.

Regards

Patrick

Comment 6

[Patrick]

Nevermind.

I think the behavior is not going to be altered in Thunderbird. Virtual folder views are, as I read, purposely "stable", because somebody might set up an "unread messages" search folder and then not want messages to disappear as soon as he clicks them.

I included the functionality for my use case in my personal extension of custom functionalities though. Basically, what the extension does, is to redefine the "0" key shortcut (RemoveAllMessageTags() by default) to update the virtual folder and select the next message after calling RemoveAllMessageTags().

Like this:

	function() {
		i = window.gFolderDisplay.view.dbView.currentlyDisplayedMessage
		RemoveAllMessageTags()
		window.gFolderDisplay.view.refresh()
		window.gFolderDisplay.selectMessage(window.gFolderDisplay.view.dbView.getMsgHdrAt(i), false)
	}

The extension is available here and redefines the "s" key shortcut as well... (but this will not work for you...) : http://www.wuala.com/patrick2000/Public/various/spoolmail/spoolmail-0.2-tb.xpi

Comment 8

[Patrick]

fixed a bug in my addon code as it stopped working....

var i = window.gFolderDisplay.selectedIndices[0];
window.RemoveAllMessageTags();
window.gFolderDisplay.view.refresh();
window.gFolderDisplay.selectViewIndex(i);

http://www.wuala.com/patrick2000/Public/various/spoolmail/spoolmail-0.4-tb.xpi

Comment 9

[u93378]

(In reply to Patrick from comment #8)
> fixed a bug in my addon code as it stopped working....
> 
> var i = window.gFolderDisplay.selectedIndices[0];
> window.RemoveAllMessageTags();
> window.gFolderDisplay.view.refresh();
> window.gFolderDisplay.selectViewIndex(i);
> 
> http://www.wuala.com/patrick2000/Public/various/spoolmail/spoolmail-0.4-tb.
> xpi


I pulled that xpi file and I don't see any lines matching what you have in this ticket.  spoolmail-0.4 just seems to have a lot of functionality about sending things to facebook, and there is no match to "gFolder" that emacs can find in that .xpi file.

I discourage people from installing this extension until this is explained more.

Comment 10

[u93378]

(In reply to ovidiu from comment #4)
> Is this still on? 
> from summary I thought that is same as Bug 365393 though seams more like a
> more deep issue. If the case is only about refresh on get new mail, that is
> the same bug, if it doesn't refresh even if leave and back to folder it is
> probably a "new" flag issue and there are a bunch of bugs some in work.

If you have a folder open in a separate tab, you have to close the tab and reopen it to get a refresh.  This is not a desirable limitation, since opening tabs is a cumbersome interaction (if you have a large menu of folders).  A refresh button seems like an obvious enhancement: manual refresh does not contradict a desire for "stability" in the virtual folders.

Comment 11

[Patrick]

Rob, are you blind? Halluzinations? Anyway. IEEE. I get the impression that it is not a very well educated club.

Now do the followin, Rob:

 1) Download the file spoolmail-0.4-tb.xpi
 2) Rename it to spoolmail-0.4-tb.zip
 3) Unpack it with winzip or another zip archive manager.
 4) Open the file bootstrap.js inside that archive with a text editor.
 5) Go to line number 44, column 33 and look at the text there.
 6) It matches exactly the four lines I have posted! :-)
 8) Try to find the string "Facebook in one of the files inside the archive.
 9) After a day or so, think about giving up and accepting that it's not there.
10) At that point, you have
     - learned how to open zip files
     - learned how to look at text files
11) Tap your own shoulder and feel great about using open source software!

I hope things are explained for you now? Otherwise just ask.

:) Cheers

Comment 12

[u93378]

When I download from the URL you posted, I don't get a well-formed zip file as that xpi file.  I get something that's only about 14Kb, and that's a text file.  This is what head shows me:


[rpgoldman-3:~/Downloads] rpg% ls -l spoolmail-0.4-tb.zip 
-rw-r--r-- 1 rpg staff 13854 Apr 24 19:02 spoolmail-0.4-tb.zip
[rpgoldman-3:~/Downloads] rpg% head spoolmail-0.4-tb.zip 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> 
	<head>
		<title>spoolmail-0.4-tb.xpi - Wuala</title>
				
				
		<meta http-equiv="X-UA-Compatible" content="IE=7" />
		<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
		<link rel="shortcut icon" href="/img/icons/favicons/favicon.ico" />
		<meta name="author" content="LaCie AG" />
[rpgoldman-3:~/Downloads] rpg% 

I pulled that once through a web browser, and once through wget, and neither time did I get a compressed file.  Is it possible that there's something on the server that stands between me and the XPI archive?

Comment 13

[Patrick]

Well, I see now where you found the reference to Facebook.

Ok, then, let me explain further:

Downloading the file spoolmail-0.4-tb.xpi is a 2-click process and you can't actually just copy the link I have posted and "wget it".

It is far easier to download it with a decent browser, like firefox for example. Tho do that you would
 - Step 1: click on the link I have posted.
 - Step 2: click on the link called "Download 'spoolmail-0.4-tb.xpi' to your computer"
 - Step 3: instruct the browser to save the file on your hard disk

Here is a screenshot of the page containing the link mentioned in Step 2. http://i.imgur.com/ExiqA.png That page, as well as the file spoolmail-0.4-tb.xpi are both hosted on wuala.com.

Alternativeley, if you still prefer to download the file with wget, you would have to copy the link mentioned in Step 2) and instruct wget to download the file from there. This url points to "http://content...?dl=1". You have to instruct wget to honor the "Content-Disposition:" HTTP header by using the "--content-disposition" option when downloading the file that way, as it will otherwise be named "...?dl=1" on your hard disk. This is a security feature of wget to prevent malicious "Content-Disposition:" HTTP headers from harming you. I can assure you that wuala.com will not purposely put such malicious response headers into their HTTP response. You can therefore safely use the "--content-disposition" option and the downloaded file will be called spoolmail-0.4-tb.xpi, as expected.

Nevertheless, download it from the browser if you can, as it is far simpler.

Btw, no data is transmitted to Facebook or any other social media companies if you visit the page that the link I have posted points to because wuala.com uses a social media widget, especially designed to prevent that, as it is explained here: http://translate.google.com/translate?sl=de&tl=en&u=http%3A%2F%2Fwww.heise.de%2Fct%2Fartikel%2F2-Klicks-fuer-mehr-Datenschutz-1333879.html

Hope that helps... :-)

Comment 14

[u93378]

Thanks; that is indeed a big help.

I just looked at the link, and it said spoolmail-0.4-tb.xpi, so I (not unreasonably, I think) assumed it was actually a link to an XPI file!

I have the correct file now, thanks.

Comment 15

[Patrick]

You are most welcome.

Don't expect it to be really restartless btw. - I remember having tried for ages to get it to work reliably but I gave up eventually.

If you want to try to fix it or create your own Addon, I can recommend "builder":
https://builder.addons.mozilla.org/

Comment 16

[d2r2p0c3]

>> I think the behavior is not going to be altered in Thunderbird. Virtual folder
>> views are, as I read, purposely "stable", because somebody might set up an
>> "unread messages" search folder and then not want messages to disappear as soon
>> as he clicks them.

I disagree. I have been using MS Outlook since the year 2000 and finally switched to Thunderbird for many reasons. However, the virtual search folders is much better implemented in Outlook. I used one for "unread messages" and having the messages disappear from the virtual search window after a couple seconds is great behavior as it allows the user to continue working through the unread messages with the windows constantly "updating" to remove the clutter of messages that have been marketed as read.
Thunderbird, should model the virtual search folders with behavior similar to that in Outlook. Currently the "Unread saved searches" is not working for me at all because summary files continue to be rebuilt, the search starts all over again or just sits there without automatically updating itself; it misses a lot of unread messages in folders, etc.

Comment 17

[Leho Kraav (:macmaN @lkraav)]

I found a combination of about:config tweaks that have virtually eliminated the need for me to manually refresh my main search folder "Today". This is on IMAP, using Sieve server-side filtering, with TB searching a bunch of shared and subfolders. My virtual folder is now updating itself with new messages from whatever folder apparently flawlessly.

http://forums.mozillazine.org/viewtopic.php?f=39&t=2678855 for details

Comment 18

[bit]

Good news: the problem seems solved - first by right-clicking on properties of the monitored folder, change to Tab "General Information" and activate the Checkbox "When getting new messages for this aacount, always check this folder". Now the virtual folder is automatically updated.

Comment 19

[erikts]

(In reply to bit from comment #18)
> Good news: the problem seems solved - first by right-clicking on properties
> of the monitored folder, change to Tab "General Information" and activate
> the Checkbox "When getting new messages for this aacount, always check this
> folder". Now the virtual folder is automatically updated.

I have monitored Inbox for today's email in a virtual folder. I have checked that Inbox doesn't have such option in it's properties. Screenshot: http://postimg.org/image/mctpqbgp5/
[Thunderbird v.31.4.0]

Comment 20

[bit]

This Option is available for each Folder, except the Inbox.

Comment 21

[Bob]

I am using a filter to hide out a tag (=done).
There should be a shortcut to re-run the filter. 
I am aware I can assign shortcuts through keyconfig for example, but what function would I call ?

Comment 22

[ivanlopez]

Same issue for me here. Version 45.4.0

Comment 23

[Joe]

This is still an issue and still needs resolving.

I have a few saved search folders that aggregate various tags/combos of tags (todo/calendar, waiting, any tagged or starred).  I keep these opened in separate tabs so I can have views of things that need doing and/or monitoring (like delegated or waiting responses) available at hand.

These work well but do not update when changes are made.  As I go through the inbox and add or edit tags, or as I move through the search folders and edit or remove tags, the search folders are not updated.  The only way to update them is to switch to another folder and then switch back.  Occasionally this doesn't work the first time but it does work probably 98-99% of the time.  In any case it's too many clicks and I'd like for these to be updated either automatically when tags change or at least with some kind of refresh (F5 comes to mind.)

This has been an issue for many many versions and is marked -enhancement.  Would be nice to see this finally get some love.
I have a few clients that use t-bird but I'm hesitant to recommend or steer them toward search folders because an 'unfinished' feeling behavior like this.  It's a shame that little things like this too often ruin the 'polish' on an otherwise really great mail client.  Would be greatly appreciated if this could be addressed soon.

Comment 24

[Japi]

Bryan explains competently why an auto-refresh function isn't so good for many users: https://bugzilla.mozilla.org/show_bug.cgi?id=365393#c10

Thus best practice would probably be (and not alternatively but both!):
a) Any edits on the content of a virtual folder (vf) shows by visual feedback: e.g. the vf name receives a star - this is typical behaviour in many software for a file that was edited but is yet unsaved; or why not change the colour?
b) Either press a button or hit a shortcut to manually refresh the folder whenever you're ready. Shortcut preferably not a single letter, because those can be hit accidentally. Ohter software (e.g. Firefox) use F5 to refresh, but that's occupied in Thunderbird ...

See how I want to use vf: I have one folder that is my "always up-to-date area" that shows everything that happens. I configure it to show not only new mails, but also recently sent mails (tagged as "just sent") to remind me to decide what to do with them (trash, move, tag, ... whatever). I have to change from this vf to another folder and then return to keep my vf folder up-to-date. Cumbersome if not annoying!

@Patrick https://bugzilla.mozilla.org/show_bug.cgi?id=267997#c8 Is that Addon still around? Wuala is not.

Comment 25

[Joe]

I can't believe this is going on 14 years and still can't be resolved.  This a nuisance in my workflow daily.  Others have well layed out the hassle of leaving and re-entering the folder so I won't rehash it all.
I'll just add that for me it's doubly even triply frustrating since I leave my most important search folders open in a dedicated tab.  
In *theory* this is so they are "always on" and these tabs should (again, in theory) *never* be moved from their respective search folder.  In practice, well....

Now as Jan says above Bryan lays out very well why an auto refresh is undesirable and probably creates more problems than it solves (365393#c10 and perhaps elsewhere).

A manual, user-intervened solution may not be perfect but it's perfectly workable, breaks nothing and hella better than no solution for a decade and a half.  This is a classic case where even the wrong action would be preferable to just inaction (14yrs? 
 ffs.  at some point - bahh.  fk it...)


A rather simple and workable solution imho (and one that I believe all the needed "moving parts" exist for - so requiring just a small add in the UI) would be to have a simple "refresh folder" option on the context menu (right click) of the search folder itself.  This should be always available (not just if "stale") so user could "force" a refresh at will.

Context menu item should work well enough for Windows, Mac and Linux users (Mac users use whatever the equiv of right-click is on their one-button mouse systems.)  Someone correct me if I'm wrong.

I believe an implementation like this would break nothing, expose an easy enough method for anyone to find (or at least intuitive enough to remember once they've searched/found/been told), would (re)use - I expect - all the already working mechanisms that exist today and involve no logic changes or other programming changes to introduce new bugs/problems etc. here or anywhere else.
Essentially it seems to me like just a small mod of the UI.  I'm not a programmer so I could be full of sht but I play one in my dreams at night so maybe I'm not too far off the mark.

As far as a shortcut key, I see no need to burden the core with that.  Power users will already have some method of key-binding, let them bind any shortcut they want to the action (the programming should make sure it's exposed if that's not an automatic given).
No "refresh indicator" or other visual is really needed either imo.  Anyone not bothered by the current behavior need not be bothered at all.  Anyone that is bothered, will seek out a solution or catch it in the change-logs/update notes.

Simple solution, problem solved "well enough".

At least "well enough" and better than the clusterfk of now.  
And that frees up all the mind share for a "better" or "more intuitive" way to worked out some time over the next 15 years or whenever the group-mind sees fit.



One last point - If the implementer REALLY feels a visual indicator is needed, I'd pose that a simple and subtle one is best. 

It could be a mild color change (maybe an opacity change?) of the search folder icon or a slightly varied icon (like adding a little "asterisk"/"shine burst" on the corner of the folder icon.)  Much more than that could confuse users and/or be visually obtrusive.  Personally I say don't bother as there's no indicator now that's been good enough for going on 15 years.  Just having the manually forced refresh would be a plenty great fix right now and probably a lot simpler to implement.

Thanks and please please please pretty please with a cherry on top please.

Comment 26

[sxc731]

After using Thunderbird for over 7 years, this shortcoming has finally pushed me to migrate to Evolution (which - quite naturally - supports this functionality out of the box). It's a shame because I really appreciated the recent improvements that went into TB 78; Evolution actually looks dated compared to TB nowadays.

This comment isn't meant as spite; hopefully it can serve as a data point to help with prioritization.

Thanks for all the fish, as they say!