Last Comment Bug 67421 - Missing possibility to filter on text in body for IMAP when offline storage not enabled
: Missing possibility to filter on text in body for IMAP when offline storage n...
Status: NEW
:
Product: MailNews Core
Classification: Components
Component: Filters (show other bugs)
: Trunk
: All All
: -- enhancement with 61 votes (vote)
: Future
Assigned To: Nobody; OK to take it and work on it
:
:
Mentors:
: 159361 208999 222286 235010 238113 249053 272506 277336 338283 367656 464825 502808 580560 674848 (view as bug list)
Depends on:
Blocks: 66425
  Show dependency treegraph
 
Reported: 2001-02-02 04:07 PST by Joachim
Modified: 2014-09-02 07:30 PDT (History)
66 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description Joachim 2001-02-02 04:07:40 PST
In Eudora for Win you can define filters that depends
on ANY text in
	body of the mail
or
	anywhere in the mail

AND

it whould be helbfull to have at least the possibility to
forward unmodified

and

to assign multiple actions
Comment 1 laurel 2001-02-02 14:12:25 PST
Also see bug 11034 for forwarding as a filter action.
Thought we had a request for multiple actions already filed, but I don't see it 
right now. I know we decided not to implement but a single filter action 
(commercial build).
Comment 2 scottputterman 2001-05-26 13:39:15 PDT
reassigning to naving
Comment 3 Håkan Waara 2002-03-05 09:29:06 PST
duplicate of bug 13145 ?
Comment 4 Matt Fago 2002-05-26 10:45:36 PDT
Really miss ability to forward and then delete messages!
Comment 5 Eric Vaandering (no email) 2002-07-28 07:50:50 PDT
*** Bug 159361 has been marked as a duplicate of this bug. ***
Comment 6 Eric Vaandering (no email) 2002-07-28 07:53:01 PDT
Can someone explain why this can't be implemented? Sure, it wastes a little
bandwidth, but would save time, which for some of us is much more important.

For instance, I want to filter out all these "bug xxxx has been marked a dup"
messages.

Comment 7 Håkan Waara 2002-07-28 07:55:36 PDT
Who says it can't be done? I think it's just a matter of coming up with a patch.
Comment 8 Eric Vaandering (no email) 2002-07-28 08:20:07 PDT
See comment 5 in bug 127250:

"No. No body filter criteria option is/should be provided even for IMAP folders
downloaded for offline use."

Comment 9 Gervase Markham [:gerv] 2003-02-03 13:06:31 PST
Urk. I thought I'd work out a quick method of defining a throw-away filter to
deal with a particularly big bunch of bugspam - but without this, it's not going
to work. At least for those of us who use decent mail protocols...

/me goes away to think of a workaround.

Gerv
Comment 10 Sander 2003-02-06 17:14:40 PST
Modifying summary to include IMAP - this one puzzled me for a while before I saw
that part of it. Dropping the forward and multi action parts as there are other
bugs for that. Also see bug 56781 and bug 40528 for previous discussion (scant
as it is) of this option.
Comment 11 Michael Baffoni 2003-02-13 15:03:47 PST
I guess I don't understand why filters can't include body when that is exactly
what happens when junk-mail filtering occurs.  Anyway we can piggyback the calls
(or cache one of the calls for the messages, or call any body filters
before/after a pull for junk mail analysis) so we don't have to call a message
(body) more than once?  To say that we are concerned with bandwidth for standard
filters and not junkmail filtering is a bit disingenuous.
Comment 12 (not reading, please use seth@sspitzer.org instead) 2003-04-09 00:37:32 PDT
> I guess I don't understand why filters can't include body when that is exactly
> what happens when junk-mail filtering occurs.

yes, it is possible to implement body text filters (or "has attachment(s)", as
nbaca has pointed out, which I think there is a bug for), doing this.

I wouldn't tie it into the junk mail filters (as the user might not be using
it), but I'd hope we'd be smart to:

1)  improve and re-use the nsIMsgFilterPlugin.idl interface
2)  make it so if I got the body (and text parts) for the filter, we wouldn't go
get it again for junk mail (if both were being used)

all this said, futuring.
Comment 13 Matthias Versen [:Matti] 2003-06-10 19:51:28 PDT
*** Bug 208999 has been marked as a duplicate of this bug. ***
Comment 14 Bogdan Stroe 2004-06-29 11:33:54 PDT
*** Bug 249053 has been marked as a duplicate of this bug. ***
Comment 15 Mike Cowperthwaite 2004-08-18 13:37:25 PDT
*** Bug 222286 has been marked as a duplicate of this bug. ***
Comment 16 Mike Cowperthwaite 2004-11-22 15:27:59 PST
*** Bug 235010 has been marked as a duplicate of this bug. ***
Comment 17 Sorin Sbarnea 2004-12-26 03:35:36 PST
No one interested in adding this *important* functionality ?
Comment 18 Steven Boothe 2005-01-26 11:01:39 PST
Well if no one is interested in making this work as the filter advertizes
itself, then maybe someone should create some wording in the filters dialog to
indicate that this won't work so people will stop trying to use the filters in a
logically intuitive fashion which in fact does not work when all appearences
indicate differently.
Comment 19 Mike Cowperthwaite 2005-02-09 20:46:56 PST
*** Bug 272506 has been marked as a duplicate of this bug. ***
Comment 20 Mike Cowperthwaite 2005-02-27 10:49:18 PST
*** Bug 277336 has been marked as a duplicate of this bug. ***
Comment 21 Mike Cowperthwaite 2005-03-23 07:30:09 PST
*** Bug 238113 has been marked as a duplicate of this bug. ***
Comment 22 Jim Meyer 2005-08-05 10:33:18 PDT
Please see https://bugzilla.mozilla.org/show_bug.cgi?id=217034#c11 for a
possible workaround to this problem.

--j
Comment 23 Mike Cowperthwaite 2005-08-05 12:20:09 PDT
(In reply to comment #22)
> Please see https://bugzilla.mozilla.org/show_bug.cgi?id=217034#c11 for a
> possible workaround to this problem.

The workaround doesn't work for the IMAP body case.
Comment 24 Gerard Earley 2005-12-05 07:20:19 PST
I would like to see this implemented as quickly as possible.

As a system administrator I am responsible for forwarding particular emails to various people based on their content. Without this functionality I cannot recommend thunderbird to people performing similar tasks.

Also it has been reported in the mozillazine forum that it is possible to make this work with IMAP by adding a "Body" custom header. These reports do point out that this functionality does not work on the latest 1.5RC1.
Comment 25 Hemant Singh 2005-12-05 07:26:56 PST
Increasing priority as it is really pending from long time now and it is one of the basic thing needed in any email client.
Comment 26 Håkan Waara 2005-12-05 09:04:09 PST
Hemant, please do not change target, priority or any other flags when a bug is not assigned to you. They are used by the developers to track their own bugs. Thanks.
Comment 27 Magnus Melin 2006-05-17 11:42:06 PDT
*** Bug 338283 has been marked as a duplicate of this bug. ***
Comment 28 brendan.maas 2006-09-11 07:56:30 PDT
I’m currently using qmail on my system with which Thunderbird interfaces very nicely.  However I need the body filter in order to help my techs organize email into categories and even so that they can ignore email that is just meant as archival record.  A perl script could be written to filter the email before it's downloaded to the client, but I'd rather leave that server alone since it’s been so stable for us.  Hopefully this can be resolved, I see that the search feature works just fine on the Body of email, so it seems like filter capability shouldn't be that difficult.
Comment 29 Sander Lepik 2007-01-21 09:04:37 PST
*** Bug 367656 has been marked as a duplicate of this bug. ***
Comment 30 Sander Lepik 2007-01-21 09:07:46 PST
it has SO many votes + SO many duplicates and you STILL don't even bother to care????

Reported: 2001-02-02

nice guys.. keep up that "good" work :(
Comment 31 Henrik Edberg 2007-02-01 08:04:37 PST
Can some one please add a thorough explanation to this thread on why this cannot/should not be implemented (some external links in other bug reports are now dead, and with them the explanation I assume). If it can/should be implemented, what makes the implementation so difficult?

I've seen comments about bandwidth usage and IMAP, and I can see how one might not want to download the message body all the time when using IMAP. But then again, if this is what the user wants, why should Thunderbird be the limitation?  I'm using IMAP because I want manage my mail on the server, not because I need to save bandwidth.

Acatually, the search can already be done manually and if I create a new Saved Search Folder I can even do it done automatically, even with IMAP. When I create the Saves Search Folder there is a check box and a warning saying "Search Online (Gives up-to-date results for IMAP and News folders but increases time to open the folder)". Seems fair. So basically I can do this, just not have it label my messages Important/Todo/Later like I want to. So close, but yet so far away ;) Is there really now way of doing this for message filters as well?

Right now it just seems like this problem is deemed not important, and I believe a lot of people think it is. I hope someone can shed some light on this.
Comment 32 David :Bienvenu 2007-02-01 08:31:38 PST
I'm sure this is explained in one of the many duplicates of this bug, but here goes again:

This is not easy. To filter on message bodies, we'd need to download and store message bodies before applying message filters. We currently only download message headers before applying filters. We'd also need to make the imap filtering code capable of reading the offline message store to do body filters.

The way I would do this would be to somehow check if any imap message filters had body search criteria. If so, we would fetch message headers *and* message bodies at the same time, and store the message bodies in the offline store, and when all the headers and all the message bodies had been completely downloaded, then apply the filters.

I'm not sure that would be acceptable. What people would want instead is for us to only download the message bodies if we need to evaluate a body search term, and that would involve rewriting the imap filter running code to be asynchronous, which would be hard.
Comment 33 Sander Lepik 2007-02-01 08:41:58 PST
how is it possible that search works? is it some server-side function then? to search from body.. if yes, then is it maybe somehow possible to merge this function with filters?
Comment 34 David :Bienvenu 2007-02-01 10:18:56 PST
again, server search is asynchronous, so the filtering code would need to be rewritten to use server search, at least for body terms, and to deal with asynchronous results.
Comment 35 Brian Cartier 2007-03-28 08:12:14 PDT
Fetching all of the message headers and bodies would be acceptable to some (perhaps most) users.  It would be reasonable to post a warning dialog the first time a user tries to filter using "Body".

There are various posts about how to get this to work on older versions of TB (<1.5).  Can you explain why it would work in those cases, but not in newer versions?

IMO, the lack of 'Body' filtering with IMAP is a HUGE limitation that will severely limit TB adoption in the corporate world.  My #1 reason for using TB is for message filtering.  With the 'Body' limitation, there is no reason for my colleagues to make the switch from Lotus Notes -- well, there are still reasons but they are less compelling. 
Comment 36 Armin Fuerst 2007-04-01 13:02:21 PDT
I don't see any problem downloading all bodies for filtering - I have to download the whole e-mail for Junk-Filter anyway. Hopefully, the mail doesn't have to be downloaded twice...
Comment 37 Shad Sterling 2007-06-07 06:06:16 PDT
My network is fast, and my computer is faster; the only bottleneck for my use of this feature is the development time, and I'm sure I'm not alone in that.  For users who are concerned about bandwidth, apply body filters as late as possible (like a short-circuit evaluation, in order of increasing expensiveness); it might be a pain to code, but in the long term - tho less than the 6 years it's taken for this to be addressed - it's a far bigger pain to not have the feature.  I think the warning dialog is a good idea, and I'd add a warning that informs the user when a filter is running and is required to take longer or download more than some threshold setting.

I suppose when we get to where we can download the body without the attachments people will fight about weather we should be able to have filter rules that need to download attachments to be evaluated.  &sigh;.
Comment 38 (not reading, please use seth@sspitzer.org instead) 2007-06-21 14:35:37 PDT
sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.
Comment 39 Tom Hundt 2007-08-01 11:01:31 PDT
Let's make this perfectly clear.

David Bienvenu in Comment 32 says,
>we would fetch message headers *and* message
>bodies at the same time, and store the message bodies in the offline store, and
>when all the headers and all the message bodies had been completely downloaded,
>then apply the filters.

>I'm not sure that would be acceptable.

YES IT WOULD BE ACCEPTABLE.  PLEASE MAKE IT WORK AGAIN.  Do you not see all the people clamoring for this functionality??

Understand:  People don't use IMAP because it only downloads the headers.  People use IMAP because they can have multiple e-mail clients on multiple machines, and they all stay nicely in sync, automatically.  Bandwidth is plentiful.  Users' TIME is not.
Comment 40 TT 2007-08-01 11:19:31 PDT
(In reply to comment #39)

> YES IT WOULD BE ACCEPTABLE.  PLEASE MAKE IT WORK AGAIN.  Do you not see all the
> people clamoring for this functionality??
> 
> Understand:  People don't use IMAP because it only downloads the headers. 
> People use IMAP because they can have multiple e-mail clients on multiple
> machines, and they all stay nicely in sync, automatically.  Bandwidth is
> plentiful.  Users' TIME is not.

I agree 100%.  However, even though bandwidth is plentiful, local disk access is faster.

I have "Make the messages in my Inbox available when I am working offline" specifically so that Seamonkey will always download all messages, including the bodies.  This makes it faster to read email, because then Seamonkey doesn't have to download a message every time I click on it.

Comment 41 David :Bienvenu 2007-08-01 12:21:57 PDT
It never worked before, so I don't understand the PLEASE MAKE IT WORK AGAIN comment.

And some people do use IMAP because it only downloads headers. YOU don't use it for that reason.

I can't do this right now but I'm happy to help anyone who wants to try to contribute a patch for it.
Comment 42 Tom Hundt 2007-08-01 13:25:20 PDT
David, I'm pretty sure this worked under some previous version -- like 1.5.  You had to add "body" as a custom message header.

See, for example,
http://kb.mozillazine.org/Message_Filters#Filtering_the_message_body

So from our point of view as users, we've lost that functionality.  It's possible it only ever worked by accident -- but it was a useful accident!
Comment 43 David :Bienvenu 2007-08-01 17:21:04 PDT
Hmm, I'd say less accident, and more miracle. I can see that doing that would make us download the body from the server, but I don't know why it would make the filter work - I don't know that we'd store the msg body in the offline store, for example, which would be required for making the filter work. I'll try to find time to test it.
Comment 44 Massimo Coppola 2007-08-05 07:10:48 PDT
Short version: Why not adding to Pre-filters (i.e. the standard ones) also POST-filter, to be run whenewer a message finally gets to the local mailbox copy? 

Alternatively pre-filters could also have the option to set a tag that says to a post filter to actually dowload the message if it gets past the junk detection...

Full version: I do not want to add too much noise, I understand both concerns: I want to be able to dowlonad headers only on a slow connection, but body-filters are actually useful when I'm on a broadband one.
I do not understand why filters have to be applied _before_ junk detection, which is worst. Since changing the implementation now seems to be too hard, we can just add another plug for filters to be run either
1) after message dowloading, to immediatly move the e-mail to its right place
2) after junk processing, when information about the new headers is available, and a filter may present the user a request window with supposedly "important" messages
Don't know the code, bu these options seem less likely to disrupt the existing implementation, they can probably reuse some existing API and impose, in their simples implementation, a small footprint addition to the UI which answers a long-time need.
Comment 45 Armin Fuerst 2007-08-09 05:50:59 PDT
If I see it correctly, the size of a message is known / described by the header. Is this right? If yes, what about a mandatory field when defining a search for the body limiting this condition to messages smaller than "x"kb? The messages I need this feature for have less than 1 kb, and it really wouldn't make sense scanning all large messages for a string in the body if I know from the size it will never be a match.
Additionally, this can always be the last filter of all required filters, so this should kick out a lot of messages anyway in case I have a quicker filter defined.
Comment 46 Jay 2007-10-01 12:28:28 PDT
I am new to the forum but we use TB extensively in the office. I can say that we hate spam and because of this we use to be able to filter on the body. Now we can not. What I would like to draw attention to is that the other options to filter on do not help much either. The reasoning is that a lot of spammers are just putting something like RE: in the subject line. We we can not go off this because RE can be in any subject and in legimitate emails that we need to look at. The other fields to filter on do no good because we do not know who on any given day is sending it. The only vital way that can differentiate the good emails from the bad are in the body because that is where the sender discloses the purpose of the email. Thank you if you do decide to put this in as soon as possible. I hope that is what is going to be done. 
Comment 47 Matthew Flaschen 2007-10-26 17:08:53 PDT
I am interested in using IMAP with Thunderbird, and I think this feature would be very valuable.  Currently, I use POP on my major accounts, and have many filtering rules set up.  Some of these include body filtering.  The obvious advantage to switching to IMAP is that if I move the sorted folders to the server, I can access the /sorted/ email from anywhere.

However, I also want complete offline access to all my email.  Thus, I intend to set the "Make the messages in my Inbox available when I'm working offline", and "When I create new folders select them for offline use" (as well as selecting this for existing folders).

Since I would already be downloading all email, I would like to be able to then filter the mail by body.  I think this is a minority use case, but not a very unusual one.
Comment 48 Eric A. Meyer 2008-02-11 14:09:32 PST
I too would like the ability to filter on the body in IMAP messages, even if that means having it only work when full messages are downloaded.  I only wish I knew enough to work on a patch.
Comment 49 SteffanVigano 2008-02-29 13:42:26 PST
Add me to the list of people that would LOVE this to work. 
Thanks
Comment 50 Walt Scheper 2008-03-06 06:12:26 PST
I agree with most of the comments. Provide the option and let the user determine if the overhead is too high. We already filter and download these messages at our site but we can't categorizer them correctly unless we can scan the body for information that isn't in the header. 

Mozilla provides solutions that generally exceed our needs and as an organization has been highly responsive to the user community. Perhaps this is another opportunity to exceed the average and go beyond the MS-esque response that you just don't need it.

Comment 51 Erik Andersson 2008-03-17 08:29:53 PDT
I want this too..
Comment 52 Serge Gautherie (:sgautherie) 2008-06-20 10:36:48 PDT
Filter on "Nobody_NScomTLD_20080620"
Comment 53 Carl Menezes 2008-09-01 13:56:11 PDT
I don't know the details of the IMAP protocol, so forgive me if this question is simplistic. Would it be possible to say, tell Thunderbird to retrieve only the message text (not the attachments) from the server and then treat that as a header in itself? That would then limit the time spent fetching the messages, plus there wouldn't be the need to store the entire message offline. Just an idea.
Comment 54 Shad Sterling 2008-09-01 14:38:17 PDT
I hadn't thought so, but a quick scan of RFC3501 gives me the impression it should be possible with BODYSTRUCTURE and BODY.PEEK.  It would be more complicated, but would be a better way to do it, if and when someone has the time.
Comment 55 Bill Davidson 2008-11-10 12:22:39 PST
I would find the ability to filter on the message body with IMAP extremely helpful.
Comment 56 Ludovic Hirlimann [:Usul] 2009-03-19 07:30:02 PDT
*** Bug 464825 has been marked as a duplicate of this bug. ***
Comment 57 Kent James (:rkent) 2009-07-07 09:28:38 PDT
*** Bug 502808 has been marked as a duplicate of this bug. ***
Comment 58 Ben Slade 2010-03-08 07:15:23 PST
I've noticed that *only* with IMAP used with SSL on Thunderbird 3.0, under server "View settings for this account", there's a new "Synchronization & Storage" option (*only* with SSL, non-SSL IMAP doesn't have this option).  If you go into that option, and enable "Keep messages for this account on this computer" *and* click on the "Advanced" button just below this option, and make sure the "Download" option is enabled for the folder(s) you want to body search, then the search body option *will* appear for that folder.

Additionally, if you enable the above option, you can include body searches in the filters (Tools->Message Filters).   I'm pretty sure this feature wasn't available before Thunderbird 3.0.

It's sort of bizarre that this new feature is only available for IMAP with SSL enabled.  Seems like it should be available for non-SSL IMAP too.
Comment 59 David :Bienvenu 2010-03-08 07:33:32 PST
It has nothing to do with SSL, afaik. 3.0 enabled body filters for imap messages; rkent would know the bug and details.
Comment 60 Kent James (:rkent) 2010-03-08 08:57:09 PST
The bug that was fixed that added the body search for imap was bug 127250.

I'm surprised to see I did not make that comment here earlier, since these bugs are closely related and I am well aware of this bug.

As David says, there is no reason for IMAP body filtering to be restricted to SSL, and I would be very surprised if it was.

I suppose though that while we are here, we should discuss the disposition of this bug. I did the work on bug 127250 rather than here, because there really is no practical way to filter on the IMAP body without also downloading that body. so you need to enable offline storage of messages, as Ben correctly notes in comment 58, for this to work. That as the point of bug 127250.

So is there any way or need to filter messages based on body content, without also downloading them for offline use? I suppose you could request 1) use server search or 2) download but not save the body, or 3) silently switch to offline storage if body search is enabled in an account.

Any interest in making this work without storing messages for offline use?
Comment 61 Brian Cartier 2010-03-08 13:32:06 PST
This is pathetic.  Years of di
Comment 62 Kent James (:rkent) 2010-03-08 13:36:12 PST
And what is pathetic? This feature now works in the only situation where I imagine it makes sense. I'm just asking if that is the correct interpretation of the need.
Comment 63 Chris Ilias [:cilias] 2010-03-08 21:50:04 PST
Here's my use-case:
I get a ton of bugmail and most of it goes into a sub-folder to read later. However, if someone mentions my name in a bug, I want to tag the message to read now.
Comment 64 Shad Sterling 2010-03-09 09:09:51 PST
Body filtering is much like body viewing: you do need to download the message, you don't need to keep it for later.  I'd prefer to have the UI give some notice when a body filter is created (and messages are not automatically downloaded anyway) that applying that filter will force a download of every message it applies to, and then have the downloaded message body stored or not by the same rules as if the message had been viewed.

It would also be good to have the UI give some notice if downloading/storing options change while a body filter is in effect, and to keep some statistics on the increase in downloads due to body filters, but I wouldn't let either of those delay allowing the filters.
Comment 65 Kent James (:rkent) 2010-03-12 10:17:59 PST
(In reply to comment #64)
> Body filtering is much like body viewing: you do need to download the message,
> you don't need to keep it for later.  I'd prefer to have the UI give some
> notice when a body filter is created (and messages are not automatically
> downloaded anyway) that applying that filter will force a download of every
> message it applies to, and then have the downloaded message body stored or not
> by the same rules as if the message had been viewed.
> 

These types of issues are the things that scared me away from supporting the case where people are not storing imap messages locally. But my main question was, is there a viable use case for it? Your replies and other make a case for that, so this bug should remain open to cover those.
Comment 66 Carl Menezes 2010-03-12 16:00:29 PST
I'd just like to add that it is possible to only download the text of the message and not any accompanying attachments. I have yet to come across an instance where I have needed to filter a message based on the content of an attachment. Therefore, forcing a download of only the text of the message is not such a bad thing, neither is it that slow. 

Here are a few use cases where I need to filter on the body of a message:

1) Bug reports that mention areas that I am interested in.
2) Tag emails with meeting notes that mention projects, keywords (like "show stopper", product names, etc)
3) Tag emails from family with words like "birthday", "anniversary", "death", etc (very few family members actually use calendars for that sort of thing)
Comment 67 Ben Slade 2010-03-17 20:48:57 PDT
Re: comments 58/59/60, requiring SSL with IMAP to search email bodies

Hmmm, I swear I saw that behavior, but I don't now.  Apologies for sharing my apparent hallucination with everyone here.

Re: comment 60, interaction of download email options and search body options

I see 2 or 3 questions here:
1) How much slower is it to download emails and save them to the client disk versus just having to download them (to search mail bodies).
My guess is, when you consider how long it takes to get an email from the server, and with modern buffered disk I/O, not all that much slower.

2) How should the GUI be restructured
Right now, the procedure to enable the body option for filters is pretty obscure.   The "search body" option just doesn't show up in the filter options unless you know to do a 2 stage enabling procedure (including using an "advanced" button) in the account settings screens.   If the "body" option could always available in the filters, and the body would be downloaded and discarded if the "Keep messages for this account on this computer" option and the per folder "download" option were not enabled for this account/folder, that would be great.  How hard would it be to do this?

Alternatively, you could automatically enable the "Keep messages for this account on this computer" option and the per folder download option, when someone sets up a body filter, but that just seems like it will have more side effects.   Would you unset it if they deleted the filter?  But if this was easier to do, it would seem acceptable to me.
Comment 68 Kent James (:rkent) 2010-03-17 22:53:06 PDT
(In reply to comment #67)

> 1) How much slower is it to download emails and save them to the client disk
> versus just having to download them (to search mail bodies).

Speed differences are minimal at best, the issue is simply preserving disk space.

> If the "body" option
> could always available in the filters, and the body would be downloaded and
> discarded if the "Keep messages for this account on this computer" option and
> the per folder "download" option were not enabled for this account/folder, that
> would be great.  How hard would it be to do this?

Prior to the release of TB 3, the prevailing direction of the product was for download and storage of IMAP messages to be enabled by default, so this whole discussion was pretty moot. But there has been considerable pushback on that. I think that if I did it again, after this current discussion, I would probably ask more questions about the non-offline storage case. I understand that there is a memory cache that holds messages, and can imagine that the difference is fairly trivial between downloading to message cache versus disk. But I actually don't have any experience with the message memory cache, so I am not sure.

Th hard reality though is that filtering is considered a second-order feature, and there is a viable workaround here, so I doubt if this is going to be fixed any time soon.

> Alternatively, you could automatically enable the "Keep messages for this
> account on this computer" option and the per folder download option, when
> someone sets up a body filter, but that just seems like it will have more side
> effects.   Would you unset it if they deleted the filter?  But if this was
> easier to do, it would seem acceptable to me.

It would be easy, but as you say has issues. Enabling a feature like message download as an unexpected side effect would not be desirable IMHO. But the current complication is also not desirable.
Comment 69 Ben Slade 2010-05-03 11:00:04 PDT
Note, I've added a help item for this under Thunderbird help (which is just a link to the "Thunderbird Knowledge Base" at http://support.mozillamessaging.com ) The URL for the help page is:

http://getsatisfaction.com/mozilla_messaging/topics/how_to_search_for_text_in_the_mail_body_content_of_multiple_emails_with_imap_mailboxes_with_thunderbird_3_x

A short URL is http://gsfn.us/t/10v3f (All feedback/corrections to this help page are appreciated)

But there's no hierarchical organization to the help web pages (ie. no specific help page for "Search Messages" window) so it's hit or miss if people will find this based on keywords.  Also, the help pages don't allow for quoted keywords so a search for "Search Messages" finds all matches for Search and all matches for Messages, blech.
Comment 70 Chris Ilias [:cilias] 2010-05-05 18:54:54 PDT
As I understand it, this bug is about filtering (Tools > Message Filters), not searching.
Comment 71 Ben Slade 2010-05-06 06:57:03 PDT
(In reply to comment #70)
> As I understand it, this bug is about filtering (Tools > Message Filters), not
> searching.

A fair point.  This bug is about filtering based on the email body, but it turns out that turning on the "keep messages on this computer" option enables the "body" option for both filtering *and* "Search Messages" (ie., searching across multiple emails)

I've updated the help item I created to reflect this (http://gsfn.us/t/10v3f)
Comment 72 grasswistle 2010-07-24 12:34:50 PDT
We're finally moving in the right direction, but all of these steps are really not user-friendly.  

1)  The Body filter-item should always be available in the filter creation dialogue.  
2) When the user decides to create a filter that reads bodies, 
a) Thunderbird should offer to make all the changes delineated in http://gsfn.us/t/10v3f for the folder in question
b) Thunderbird should warn the user of the disadvantages of making these changes.
3) If the user deletes all filters requiring that the body of the messages of the folder in question be downloaded and kept locally,
a) Thunderbird should offer to undo the chances it made in 2a (if the user agreed)
b) Thunderbird should present the advantages of undoing these changes.

No commonly adopted program should expect its users to manually make so many configurations changes for such a simple and fundamental feature.  

I think that such a design would provide all the flexibility that a large base of users needs in order to be happy and productive, given their differing professional and personal requirements.

Grasswistle
Comment 73 grasswistle 2010-07-24 12:39:12 PDT
We're finally moving in the right direction, but all of these steps are really not user-friendly.  

1)  The Body filter-item should always be available in the filter creation dialogue.  
2) When the user decides to create a filter that reads bodies, 
a) Thunderbird should offer to make all the changes delineated in http://gsfn.us/t/10v3f for the folder in question
b) Thunderbird should warn the user of the disadvantages of making these changes.
3) If the user deletes all filters requiring that the body of the messages of the folder in question be downloaded and kept locally,
a) Thunderbird should offer to undo the chances it made in 2a (if the user agreed)
b) Thunderbird should present the advantages of undoing these changes.

No commonly adopted program should expect its users to manually make so many configurations changes for such a simple and fundamental feature.  

I think that such a design would provide all the flexibility that a large base of users needs in order to be happy and productive, given their differing professional and personal requirements.

Grasswistle
Comment 74 peter 2010-08-12 00:36:17 PDT
*** Bug 580560 has been marked as a duplicate of this bug. ***
Comment 75 Eric A. Meyer 2010-12-02 08:09:42 PST
It is possible to filter body text in IMAP now, so shouldn't this bug be marked as RESOLVED?  I agree with Grasswhistle that it could be easier to do, but surely that's a different issue?
Comment 76 Kent James (:rkent) 2010-12-02 09:58:35 PST
The work to add IMAP body search in cases where offline storage is enabled was added in a different bug. As I said in comment 65, I would leave this bug open since the job is not complete, but needs to be added for the non-offline storage case as well.
Comment 77 Billy Newsom 2011-10-08 20:59:45 PDT
Recommend we close this bug. WFM.
Comment 78 Magnus Melin 2011-10-11 03:20:11 PDT
No, comment 76 still applies.
Comment 79 John Hopkins (:jhopkins) 2012-01-25 11:53:07 PST
(In reply to Magnus Melin from comment #78)
> No, comment 76 still applies.

I'd recommend creating a new bug and close this one to ease triage.
Comment 80 Magnus Melin 2012-01-25 22:41:56 PST
Well, this bug has all the details, so... (updating the summary)
Comment 81 Wayne Mery (:wsmwk, NI for questions) 2012-02-14 19:44:42 PST
*** Bug 674848 has been marked as a duplicate of this bug. ***
Comment 82 RadoQ 2012-05-08 04:48:55 PDT
(In reply to David :Bienvenu from comment #41)
> It never worked before, so I don't understand the PLEASE MAKE IT WORK AGAIN
> comment.

I don't know the timeline (given I reply to a comment from 2007), but I know in TB 2.0.0.x (x<18) it has worked (without a hack of source or "custom headers" tweak, it just worked out of the box).

So... whatever made it work back then has been lost somehow with the jump to 3.xxx.
Comment 83 Arne Berglund 2012-08-03 09:54:07 PDT
Just stumbled on this, and have some additional data to share. We can duplicate this on version 13 & 14, both Mac and PC. It appears to be related to local indexes getting messed up.

If you search (locally, not on server) a folder where everything is normal, Body is an option in the drop-down. Re-name the folder, and try again - Body does not appear. Closing and re-opening Thunderbird does not bring back the ability to do a Body search on this re-named folder, nor does a folder repair. Re-naming back to the original name will bring back the Body option. Or you can unsubscribe-resubscribe to the re-named folder, then Body is again available.
Comment 84 Kent James (:rkent) 2012-08-06 10:39:34 PDT
(In reply to Arne Berglund from comment #83)
> 
> If you search (locally, not on server) a folder where everything is normal,
> Body is an option in the drop-down. Re-name the folder, and try again - Body
> does not appear. Closing and re-opening Thunderbird does not bring back the
> ability to do a Body search on this re-named folder, nor does a folder
> repair. Re-naming back to the original name will bring back the Body option.
> Or you can unsubscribe-resubscribe to the re-named folder, then Body is
> again available.

What you are describing does not appear to be the main point of this bug. Could you file that as a new bug?
Comment 85 dougd 2013-03-05 14:03:03 PST
Ok I have a legitimate Body search problem (likely related)    I want to copy all calendar invites coming into my inbox into another place.  I did this by the search string  -> Content-Type: text/calendar; charset="utf-8"; method=REQUEST <- which only shows a few of them ..    I shortened it to  -> text calender REQUEST <- which shows them all in quick filter..   But in Tools->Message Filters "body" does not seem to be able to filter them at all, nothing shows up in the filter log

TB 17.0.3   IMAP server (exchange)  Theoretically all messages are being copied into the local inbox .. sync and storage are set up to keep messages on server, etc..
Comment 86 chris.rapson+bugzilla 2014-04-24 01:02:56 PDT
The webpage originally linked to in comment 69 seems to be dead, and the information isn't repeated here. So for posterity's sake, the workaround is:
Edit -> Account Settings --> Synchronization & Storage
Tick the box at the top "Keep messages for this account on this computer".

This is documented in the help: https://support.mozilla.org/en-US/kb/imap-synchronization (but not the fact that doing so makes body filters work).
Comment 87 Julien Pierre 2014-06-24 18:48:08 PDT
I set the checkbox for "Keep messages for this account on this computer".

Now, the Body option appears in the Search dialog.

However, Thunderbird fails to find anything when I search on this field.
It doesn't look like it's performing the search at all.
I am using IMAP SSL.

But "Run search on server" is off in the search dialog.
Comment 88 Armin Fuerst 2014-06-24 23:08:06 PDT
(In reply to Julien Pierre from comment #87)
> I set the checkbox for "Keep messages for this account on this computer".
> 
> Now, the Body option appears in the Search dialog.
> 
> However, Thunderbird fails to find anything when I search on this field.
> It doesn't look like it's performing the search at all.
> I am using IMAP SSL.
> 
> But "Run search on server" is off in the search dialog.

Even though this issue is about message filtering (and not searching), I cannot confirm this:

1. synchronized account
 - search body locally is working
 - search body on server is working

2. not synchronized account
 - search body locally is not available
 - search body on server is working
Comment 89 Julien Pierre 2014-06-25 16:37:39 PDT
Armin,

Well, maybe I'm running into a different bug. But for me, the only way to search body in messages is to copy messsages from my IMAP folder to another folder created under "Local folders", and then search that folder. 

Nothing else produces any search results even though the option to search on Body is available.

Any idea what might be the issue ?
Comment 90 slmingol 2014-09-02 07:30:40 PDT
I'm having the exact issue as Armin, where I can only search the body when I've selected a top level folder to do body searches. If I pick a nested folder that choice is simply missing from the list. I'm using IMAP and the IMAP is via GMail if that makes a difference.

Note You need to log in before you can comment on or make changes to this bug.