543416 - Search messages option to search in message body is not available for non-synchronised IMAP folders. `run search on server` option doesn't stick.

Status: NEW

(Thunderbird :: Search, defect, P3)

Description

[Daniel Essin]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

option to search in message body is no longer available

Reproducible: Always

Steps to Reproduce:
1. Try to select message body for search scope in toolbar or search window
2.
3.
Actual Results:  
The option is missing

Expected Results:  
It was in version 2 and it's essential.

Comment 1

[Nathan Tuggy (:tuggyne)]

I can't reproduce the problem [using Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 ThunderBrowse/3.2.8.1 ID:20100111101938]; when I drop down the search filter scope, I see Message Body Filter as the last item.

See screenshot about to be attached.

Comment 2

[Nathan Tuggy (:tuggyne)]

Attachment: Screenshot of body filter option showing up in search drop-down

Comment 3

[Daniel Essin]

Attachment: Search screen showing no bessage body option

Per the comment on the Wall Street Journal that thunderbird is still for techno-geeks, the word filter should be removed from all the search option descriptions. It confused me to the point that I misinterpreted "message body filter".

The search screen with detailed criteria does not offer a body option at all - see screen shot.

Comment 4

[rsx11m]

Do you have indexing enabled in Tools > Options > Advanced > General tab, and are you using offline synchronization? (that's in Tools > Account Settings.) Searching for the message body on IMAP accounts is possible locally only if synchronized copies are held (the indexing may be only relevant for the search bar "everything" option). If you check "Run search on server" then the "Body" item should reappear in the search dialog.

As for the use of "filter", I remember this being criticized in another bug as well, and I have to agree that "filter" in the sense of message filter versus search filter is rather ambiguous. Anyway, that's what was decided on...

Comment 5

[Daniel Essin]

I have indexing turned off and I am not using local synchronization. My computers are not fast enough to handle the indexing activity without a significant performance hit (since it is busy all the time, or so it seems) and, since I access the IMAP server from about 10 different machines local copies (which I was trying to escape when I abandoned POP) would be impossible to manage.

I see that if I enable "run search on server" that the options change. This was not obvious and there isn't much documentation. Perhaps turning off local synchronization should automatically enable searching on the server or there should be a nested checkbox in the options so that when you uncheck local sync you have the option to enable server searching?

I'm also not sure that the default should be to sync IMAP because it immediately starts downloading and, if you are short of disk space or have performance or bandwidth limitations, it can create a situation that can be difficult (or at least time consuming) to recover from. I would have preferred to be asked as part of the install. This would have had 2 benefits:
1 - It would have educated me about the possibilities
2 - It would have saved a lot of time and confusion because I would have turned off all performance degrading options before they afflicted me.

I would have never figured this out without your input.
Thank you

Comment 6

[rsx11m]

You are welcome, that's indeed is a hard to find solution for this problem.

I'm tempted to reopen this for the purpose of making the dependencies somehow more visible to the user, in either case it's a support/documentation issue.

Comment 7

[Nathan Tuggy (:tuggyne)]

(In reply to comment #6)
> I'm tempted to reopen this for the purpose of making the dependencies somehow
> more visible to the user, in either case it's a support/documentation issue.

Feel free to reopen this; I was a bit hasty in resolving it, sorry.

(In reply to comment #5)
> I would have preferred
> to be asked as part of the install. This would have had 2 benefits:
> 1 - It would have educated me about the possibilities
> 2 - It would have saved a lot of time and confusion because I would have 
> turned off all performance degrading options before they afflicted me.

Was the migration wizard shown on first startup unsuitable for this?

Comment 8

[rsx11m]

Ok, let's give it another try, there seems to be room for improvement.
Reopening and confirming to clarify the user's options.

Comment 9

[Daniel Essin]

The search does not remember the 'search on server' check. I think it needs to be a config option.

Comment 10

[rsx11m]

You are right, it flips back to unchecked when opening the dialog the next time, bummer. An intuitively simple solution would be to check that option by default when opening the search from an IMAP folder which is not synchronized (or, if this affects other searches, flip it on by default if the user selects "Body" as a search criterion).

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> Searching for the message body on IMAP accounts is possible locally only if
> synchronized copies are held (the indexing may be only relevant for the search
> bar "everything" option).

If search of single search target folder, current implementation(no body search option if IMAP folder offline use=off) is effective to reduce users confusion.
However, if virtual folder(saved search folder), and if multiple search target folders, and if search target folder is mixed, "body search can't find mail in offline use=off folder" still occurs.
    (a) IMAP folder of "offline use=off"
    + (b) Local mail folder + (c) IMAP folder of "offline use=on"
"Search online" is grayed out. Body search is enabled but body search can't search body of mails in (a).
To make body search possible for (a) in this case, next may be required.
 [x] Online search of offline-use=off folder(unchecked=body search is ignored)  
 [x] Online search of offline-use=on folder(unchecked=local body search)

rsx11m, should we open separate bug for this special case of Virtual folder?

Comment 12

[Daniel Essin]

This sounds like a reasonable approach.

Also - there really ought to be a toolbar button, or at least a menu option, to invoke the search dialog without having to go to the account home page and clicking on Advanced Features|Search Messages. This undoubtedly belongs in a different section but I'm not familiar enough to know which.

Comment 13

[Nathan Tuggy (:tuggyne)]

(In reply to comment #12)
> Also - there really ought to be a toolbar button, or at least a menu option, to
> invoke the search dialog without having to go to the account home page and
> clicking on Advanced Features|Search Messages. This undoubtedly belongs in a
> different section but I'm not familiar enough to know which.

Edit -> Find -> Search Messages… (Ctrl+Shift+F) should do what you want.

Comment 15

[Andy Stevens]

Regarding the "Run search on server" checkbox: I agree with Daniel Essin that this should be a config option.  

Also, I am seeing this buggy behavior: if I forget to check the "Run search on server" box, I get zero results (as expected because I don't use offline synchronization).  But if I subsequently check the box, then my search STILL returns zero results.

Comment 17

[Wayne Mery (:wsmwk)]

this has multiple reports in gsfn, http://getsatisfaction.com/mozilla_messaging/topics/unable_to_search_message_body is one such topic, and it also gives a workaround http://getsatisfaction.com/mozilla_messaging/topics/unable_to_search_message_body#reply_3456747

http://getsatisfaction.com/mozilla_messaging/searches?query=search+AND+body&x=0&y=0&style=topics has some some matches. Can someone tag one of the "better" topics canonical and mark it and this bug according to https://wiki.mozilla.org/Thunderbird/Support/Workflow#GS_Triage  ... and also identify the ones that are duplicate topics? Then someone can merge them to the canonical topic.

there may also be duplicates to this bug in https://bugzilla.mozilla.org/buglist.cgi?chfieldto=Now&chfield=[Bug%20creation]&query_format=advanced&chfieldfrom=2010-01-01&short_desc=search%20body&short_desc_type=allwordssubstr&type0-0-0=nowords&value0-0-0=count%20counts&resolution=---&product=MailNews%20Core&product=Thunderbird

regarding comment 11, Jeff's Bug 600513 also has an approach

Comment 18

[Jeff Bloomfield]

I believe that searching message body (IMAP) should always appear in the list
box whether or not search-on-server is checked.  To a mail guru I suppose the
current behavior is obvious, but to an ordinary user it is not.  That is, the
knowledge that search-on-server needs to be checked for "message body" to
appear on the search list box is implicit not explicit knowledge.

I think a reasonable behavior, assuming search-on-server is unchecked, would
simply to show an "ok/cancel" box with the message something like, "This option
will requiring searching on your mail server and will take more time than a
simple mail message header search.  Proceed?"  Or simply shut-up and do the
body search on the server.

I had to search on google to learn of this behavior.  The current behavior
makes sense from an engineer's point of view but not from the point of view of
an ordinary user, especially one used to the previous version.

Comment 19

[Dmitry]

I can't find the search drop-down (attachment#424548 [details]) anymore: in Thunderbird 3.1.5 there is only search icon, but no drop-down. However search via "Edit -> Find -> Search messages" (attachment#424622 [details]) works fine for me. A bit confusing that "Body" is hidden if "Run search on server" is disabled. I agree with Jeff Bloomfield that "Body" option should be always there, and if selected then Thunderbird should run search on server automatically, or complain that the search is not possible (e.g. because server does not support it).

Comment 21

[u331193]

Hi,

I can find "body" but only when I'm searching from "Incoming messages for xxx" (Courrier entrant sur xxx in my french version).
When I select a subfolder, body becomes grayed, and if I open the dropdown list, it is not in it anymore.
I must search from the whole account if I need to look for a part of the content of the mail body.

I'm usin IMAP account, with the following version : 
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7

Thanks.

Comment 22

[nixscripter]

Attachment: Shows workaround search with grey bar

Comment 23

[nixscripter]

In my version Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100920 Thunderbird/3.0.8, there is a workaround.

The bug occurs on an IMAP mailbox but not POP. If I search on the pop mailbox, select "Body", and then change the mailbox to the IMAP one in the search selection dialog, the criteria will grey out but stick. Then, I can search, and it will work. See attachment (which I accidentally put above this message, sorry.)

Comment 24

[Charles]

My comment is only tangential, but, if a config option is provided, I hope it will be made available on a per account basis (see below for why)...

One thing that no one has mentioned so far is there are at least two (not sure about gmail) very popular IMAP servers out there - Cyrus and Dovecot - that provide full server side indexing of all messages. With Dovecot, all headers and meta-data are indexed by default, and 'full text' indexing (includes message bodies) it is a plugin option, but most people use it. I believe Cyrus indexes everything by default.

So, I would also hope that this new config option would take this fact into account, and maybe a two-pronged option:

[ ] 'Perform search on server' always for this account

    [ ] Use 'Perform search on server' for Quickfilter Toolbar

This way, for servers that support it, we could enable Body searches on the Quickfilter toolbar without having to have all messages available OFFLINE, and take advantage of the server side indexes.

Comment 25

[Charles]

There is an interesting thread on the question of server side searches here:

http://groups.google.com/group/tb-enterprise/browse_thread/thread/4ee517c902c1075a/8380a26cef9619bf

Comment 26

[Charles]

In summary, the bottom line is, according to Timo (dovecot author), *all* IMAP
servers are *required* to support SEARCH extension, so no reason to query that,
but there is a new (a year or so old) RFC for SEARCH=FUZZY for a server to
announce that it supports server side *indexes*, but apparently because it is s
new, dovecot is the only one advertising it currently (but gmail and cyrus
(when it is enabled) both support server side indexed searches)...

Comment 27

[lotis666]

Have same problem

Comment 28

[ThomasH]

Confirming the same issue with TB 52.2.1
Option to search inside message body has vanished. 

Please repair expediently, this is really crucial.  I used UNIX-like tools to search through message files in order to find the text in question.

Comment 29

[ThomasH]

Please change IMPORTANCE to "CRITICAL".
Thanks in advance.

Comment 30

[Nathan Tuggy (:tuggyne)]

(In reply to ThomasH from comment #28)
> Confirming the same issue with TB 52.2.1
> Option to search inside message body has vanished.

Did you check the workarounds listed upthread? Make sure you're using server-side search. A quick check on shows that Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 still supports Body search under the right circumstances.

(In reply to ThomasH from comment #29)
> Please change IMPORTANCE to "CRITICAL".

Yeah, that's not happening. See the guidelines at https://wiki.mozilla.org/BMO/UserGuide/BugFields under Severity:

>critical	Crashes, loss of data, severe memory leak
>major		Major loss of function
>normal		Regular issue, some loss of functionality under specific circumstances
>minor		Minor loss of function, or other problem where easy workaround is present 

There is "some loss of functionality under specific circumstances". Those happen to be ones that you experience frequently, but many other users do not. "Major" would be for things like "can't sort messages by date". Since there's no easy workaround, it's arguably not minor. (Although since there *is* a workaround, it's at least arguable.)

Comment 32

[Worcester12345]

Following this bug because you can't do a normal search using body for Yahoo IMAP mail.

Please add the key words:
ux-affordance
ux-control

Comment 33

[Wayne Mery (:wsmwk)]

Related bugs https://mzl.la/3XliMEr

Quick filter using "body" can also be affected.

Comment 34

[Worcester12345]

(In reply to Worcester12345 from comment #32)

Following this bug because you can't do a normal search using body for Yahoo IMAP mail.

Please add the key words:
ux-affordance
ux-control

Comment 38

[chrizilla]

see screenshots in duplicate bug 1676722

Comment 39

[Worcester12345]

@chrizilla

(In reply to Worcester12345 from comment #34)

(In reply to Worcester12345 from comment #32)

Following this bug because you can't do a normal search using body for Yahoo IMAP mail.

Please add the key words:
ux-affordance

How about tracking on 128 & 128ESR?

Comment 40

[Jeff Bloomfield]

Why would I be getting a bug the was filed 16 years ago. What does this say about Thunderbird?

Comment 41

[Wayne Mery (:wsmwk)]

(In reply to Worcester12345 from comment #39)

@chrizilla

(In reply to Worcester12345 from comment #34)

(In reply to Worcester12345 from comment #32)

Please add the key words:
ux-affordance

How about tracking on 128 & 128ESR?

We tend not to mark very old bugs for tracking - it's more often used for regressions, which this bug is not. It's also not something chrizilla is authorized to do. But more generally, Thunderbird search will be getting an total overhaul in the coming year, so this bug is unlikely to get a near term fix unless a community volunteer steps forward to fix it.

Comment 42

[Francesco (no longer active, use PM for contact)]

Somehow the bug title is confusing:
advanced search messages option to search in message body is not available for local search
That's actually wrong, you can search bodies on local messages for local folders. Apparently it doesn't work for IMAP messages, if the folder isn't synchronised for offline use. That's not a surprise. The duplicates actually have better summaries.

I've changed the summary to match the complaints in the duplicates: 1) No body search unless you select "online search" (on server) and 2) this option doesn't stick.

Comment 43

[Worcester12345]

(In reply to Wayne Mery (:wsmwk) from comment #41)

(In reply to Worcester12345 from comment #39)

@chrizilla

(In reply to Worcester12345 from comment #34)

(In reply to Worcester12345 from comment #32)

Please add the key words:
ux-affordance

How about tracking on 128 & 128ESR?

We tend not to mark very old bugs for tracking - it's more often used for regressions, which this bug is not. It's also not something chrizilla is authorized to do. But more generally, Thunderbird search will be getting an total overhaul in the coming year, so this bug is unlikely to get a near term fix unless a community volunteer steps forward to fix it.

Then this should just be marked "WONTFIX" and be done with it.