546925 - custom header search is non-functional ('Run search on server' of Edit/Find/Search Messages doesn't work any more)

Status: RESOLVED FIXED

(Thunderbird :: Search, defect)

Description

[Stan Hoeppner]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

Custom header search is non-functional.

IMAP server: Dovecot 1.2.10 Debian Lenny, same results with 1.0.15
Folder format: mbox
I am _not_ synchronizing IMAP folders to local cache/copies

Right click imap folder and select search, or navigate edit menu to get to same search window.  Select "Customize" from the far left drop down.  Add "Received" to custom header list.  Select "Received" from list, uncheck search subfolders (there are none), check "Run search on server", select "contains" from middle drop down, type known to exist Received head string into far right box.  click "Search" button.

Nothing happens.  TB does not perform the search.  There is no error given.  Simply "No matches found."  My main concern is received headers, but it appears this occurs with _any_ custom header.  It doesn't work, with or without checking "Run search on server".  TB just doesn't perform the search.

Reproducible: Always

Steps to Reproduce:
1. Right click IMAP folder, select search, check "Run on search on server"
2. Create custom "Received" header, select it, select "contains" center drop down
3. Enter Received header string such as an IP address or domain known to exist in message Received: headers in that folder, hit enter or click "search" button.  Nothing happens.
Actual Results:  
Search did not execute.  Returns result "No matches found".

Expected Results:  
Emails with matching strings within Received: headers should appear in the results window.

Searching Received: headers is a critical feature for me.  It should be no different that searching From: headers, but for some reason TB ignores custom headers completely.  For me this is a major flaw.  For others that don't need to be able to search custom headers it obviously would not be.  It does make me wonder why there is even an option to create custom headers within the search window, considering they are then ignored when you try to use them.  Very irritating situation.  If there is any additional information you need please let me know.  I'd really like this bug fixed so I can use this feature.

Comment 1

[Alfred Peters [:infofrommozilla]]

I can confirm this for:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre)
Gecko/20100401 Lightning/1.1a1pre Shredder/3.2a1pre

It is not a question of custom or not custom header, but of 'Run search on server'. When activated TB should send an SEARCH-command to the server, but TB doesn't.
You got a result on non-custom Headers because TB uses the local storage on these.

'Search on server' works fine when saved as Serach Folder.

BTW: I have the same behavior with "Run search on server" on Newsgroups (XPAT).

Comment 2

[WADA:World Anti-bad-Duping Agency]

I could confirm with Tb 3.0.4 and Shredder/3.2a1pre.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre) Gecko/20100331 Shredder/3.2a1pre

If Edit/Find/Search Messages or Search of folder context menu, IMAP commands is vever issued.
If Saved Search folder, next command is issued.
> 13 uid SEARCH UNDELETED HEADER "received" "mail-gw.auone-net.jp" 
mail.server.default.autosync_offline_stores=true or false is irrelevant.

Confirming.

"auto-sync + Gloda" broke online search feature of IMAP, even if auto-sync & Gloda is disabled?

Comment 3

[WADA:World Anti-bad-Duping Agency]

FYI.
"Body" was removed from drop down selection list of Search(File/Edit Search Messages) and Quick Search for IMAP folder of offline-use=off, in order to avoid user's confusion caused by 'Run search on server dosn't work on Body'.
Bug 543416 is for this issue of "removed Body search option".
> Bug 543416 option to search in message body is no longer available (run server-side searches w/o IMAP synchronization not obvious, won't stick)

Comment 4

[WADA:World Anti-bad-Duping Agency]

setting dependency to bug 543416 for ease of tracking.

Comment 5

[Alfred Peters [:infofrommozilla]]

The broken Server-Search is caused by:
<http://hg.mozilla.org/comm-central/rev/346308260286>
from Bug 537820

Proofed by checkout.

News- and IMAP-search on server work fine without that patch.

Comment 6

[Stan Hoeppner]

Just to clarify due to some comments above, server side (IMAP) searches work for me but only for body searches.  TB is performing all header searches locally regardless of "run search on server" being checked.  This is apparently why custom header searches return no results, as custom headers aren't cached or indexed locally.

Also, I have no idea what a saved search folder is.  I've never used this and probably never will, if my guess is correct as to what function it provides.  I rarely if ever search for the same thing twice, so saved search folders would be useless.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #6)
> server side (IMAP) searches work for me but only for body searches.

"offline-use=on" folder case? (if offline-use=off, "Body" is not selectable)

> TB is performing all header searches locally regardless of "run search on server" being checked.
> This is apparently why custom header searches return no results, as custom headers aren't cached or indexed locally.

Thanks for making problem clear. Different but similar issue to bug 184490. 
> Bug 184490 Run Now for Run selected message filter / After-the-fact Filters
> on custom header (eg "User-Agent" or "Newsgroup") won't match for IMAP messages

We are better to open bug of "remove custom headers from selection list of Search until this bug will be fixed"?

Comment 8

[Stan Hoeppner]

(In reply to comment #7)
> (In reply to comment #6)
> > server side (IMAP) searches work for me but only for body searches.
> 
> "offline-use=on" folder case? (if offline-use=off, "Body" is not selectable)

I'm not familiar with the internal boolean variables of TBird and their explicit meanings and interdependencies.  What I can tell you is this:  In my setup, your parenthetical statement above is incorrect.  I only work online and "Body" is available to me.  The Body searches are always performed on my Dovecot IMAP server.  Body searches work very well.  Searching for headers that are not part of the standard default headers that TBird indexes locally fails, apparently because that data is not indexed locally.  TBird doesn't issue the IMAP search commands to the IMAP server for headers that are not cached locally.  At least, that what it looks like from here.

> > TB is performing all header searches locally regardless of "run search on server" being checked.
> > This is apparently why custom header searches return no results, as custom headers aren't cached or indexed locally.
> 
> Thanks for making problem clear. Different but similar issue to bug 184490. 
> > Bug 184490 Run Now for Run selected message filter / After-the-fact Filters
> > on custom header (eg "User-Agent" or "Newsgroup") won't match for IMAP messages

I've run into this 184490 bug as well.  I don't create filter rules that often so I just move messages to folders manually.  The filters work on newly arriving mail, just not on matching messages already in the inbox at the point in time the rule is created.  This 184490 bug wasn't/isn't a priority for me as it's easy to work around.  

There really isn't a workaround for this custom header search bug, except possibly sync'ing messages locally, which I'm not willing to do.  My IMAP mailbox is around 500MB.  The IMAP server is on local switched fast ethernet.  For remote access, on the rare occasions I need it, I use Roundcube webmail, so there is no legitimate reason to sync messages in TBird.

> We are better to open bug of "remove custom headers from selection list of
> Search until this bug will be fixed"?

That recommendation doesn't solve any problems, but only attempts to mask their existence.  This bug is many months old.  Why hide it now?  Also, would such a change not break this search functionality for users who DO sync messages locally, and currently successfully search custom headers locally?  You are recommending to just remove the "Customize" option from the drop down, correct?  I'm guessing that would break many users and you'd have more bug reports filed because of it.

The best course of action is to just fix this bug.  Another dev already pointed out the fix, which is to remove the patch that broke this in the first place.

Comment 9

[Stan Hoeppner]

(In reply to comment #8)
> (In reply to comment #7)
> > (In reply to comment #6)
 
> I've run into this 184490 bug as well.  I don't create filter rules that often
> so I just move messages to folders manually.  The filters work on newly
> arriving mail, just not on matching messages already in the inbox at the point
> in time the rule is created.  This 184490 bug wasn't/isn't a priority for me as
> it's easy to work around.  

I need to clarify this statement above.  When I do create rules, and am unable to run the rule against mail already in the inbox, at that point I move the existing messages manually.  I receive hundreds of mails per day, and rely heavily on my TB filters to move mail to the proper IMAP folders.  I believe my statement above made it sound as if I sort/move all of my mail by hand, which is _not_ the case.  That would really be a PITA given my volume of email.

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)

I meant Folder Properties/Synchronization, "Select this folder for offline use" by my "offline-use". (offline-use = on:the option is checked, off:unchecked)

"Body" was removed from Search(Edit/Find/Search Messages) for IMAP folder of offline-use=off. So, bug 543416 was opened.
You probably use the IMAP folder with offline-use=on currently.

> > We are better to open bug of "remove custom headers from selection list of
> > Search until this bug will be fixed"?
> That recommendation doesn't solve any problems, but only attempts to mask their existence.

However, many unwanted dup bugs of this bug by general users will be avoided by the change, if this bug will not be fixed for long time.

> Also, would such a change not break this search functionality for users
> who DO sync messages locally, and currently successfully search custom headers locally?
> You are recommending to just remove the "Customize" option from the drop down, correct?

Oh, sorry for lack of some words. I wanted to say;
  Remove custom headers from list of Search(Edit/Find/Search Messages)
  for IMAP folder of offline-use=off, as done on Body,
  if this bug will not be fixed for long time,
  to avoid user's confusion and unwanted bug open by general users.
Bugs opened for lack of functionality(online custom header search in this case) by general users are far easier to process than bugs for mis-behaviour or wrong-behavior or unwanted-behaviour like this bug.

Comment 11

[Alfred Peters [:infofrommozilla]]

(In reply to comment #10)
>   Remove custom headers from list of Search(Edit/Find/Search Messages)
>   for IMAP folder of offline-use=off, as done on Body,
>   if this bug will not be fixed for long time,

Look at that patch I mentioned above. It needs just another option for "Run search on server=ON". That should be possible - But I am no developer ;-)

In case of "Run search on server=OFF" *and* "offline-use=OFF" custom headers should be removed, yes.
But in case of "offline-use=OFF": "Run search on server=ON" should be the default.

>   to avoid user's confusion and unwanted bug open by general users.

"Search on server" is an essential functionality of IMAP.
Gloda is a nice feature, but as long as it needs to store the E-Mail local (offline) it is no option for me. I use IMAP, because I don't want to store my E-Mail local.

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
> Look at that patch I mentioned above.
> It needs just another option for "Run search on server=ON".

Can this bug's problem be called "regression produced by patch for bug 537820"?
> comment of patch for bug 537820. 
> - // If both scopes work, honor the onlineSearch request
> + // If both scopes work, honor the onlineSearch request, unless we're
> + // filtering (quick search and/or a view selected)
If comment is right, onlineSearch should be honored for custom header search at ordinal Search?
Or online/custom header search didn't work at ordinal Search far before the change? 

> "Search on server" is an essential functionality of IMAP.

I think so. I think normal use of IMAP with "Gloda=off/auto-sync=off" should be garanteed by IMAP mail client. "Gloda and auto-sync" shouldn't block normal use of IMAP by user.

Comment 13

[Stan Hoeppner]

(In reply to comment #11)
> (In reply to comment #10)
> >   Remove custom headers from list of Search(Edit/Find/Search Messages)
> >   for IMAP folder of offline-use=off, as done on Body,
> >   if this bug will not be fixed for long time,
> 
> Look at that patch I mentioned above. It needs just another option for "Run
> search on server=ON". That should be possible - But I am no developer ;-)
> 
> In case of "Run search on server=OFF" *and* "offline-use=OFF" custom headers
> should be removed, yes.
> But in case of "offline-use=OFF": "Run search on server=ON" should be the
> default.

+1  I can't understand why this is not already the default.  It's simple logic.  What would be even simpler and better is to let the user force TB to do "everything" (that's possible with IMAP protocols anyway) on the IMAP server, period.  I'd rather that TB didn't do any local indexing and relied on IMAP calls for virtually everything.  Dovecot has super fast message indexing already.  Why duplicate the indexing on the client?  TB indexing actually SLOWS my email experience dramatically because of this.  Since I use TB to sort inbox mail into various IMAP folders, due to all the TB local indexing, it can take up to a minute or more to sort 150 messages and move them to the correct IMAP folders.  This is insane.  I deal with this daily because of all the list mail I receive.  Most days I have 100 to 150 messages in the inbox when I fire up TB.  If TB didn't do this completely unnecessary indexing of IMAP messages, it should be able to move those 150 messages to the correct IMAP folders in less than 5 seconds.  Sorry for the rant.  This performance issue has been bugging me for a while.  I should probably file a separate bug report on that, but it appears that performance issue is tied directly into this one due to the way TB handles IMAP in general.  TB just doesn't seem to do IMAP correctly, or maybe a better description is that TB doesn't do IMAP optimally.  I'm trying not to be too harsh as I really do love and rely on most of TB's functionality.  I'm just really peeved with the way the general IMAP support is implemented.

> >   to avoid user's confusion and unwanted bug open by general users.
> 
> "Search on server" is an essential functionality of IMAP.
> Gloda is a nice feature, but as long as it needs to store the E-Mail local
> (offline) it is no option for me. I use IMAP, because I don't want to store my
> E-Mail local.

+1  Dovecot IMAP has fantastic and speedy search capabilities and they improve regularly.  Because I use IMAP search so heavily I uncovered a bug recently WRT mbox searches in Dovecot that Timo fixed in 1.2.11.  I would use TB IMAP search much more heavily if I could search full headers.  Like you I don't want to store copies of IMAP mail locally on any given PC.  That's why I implemented an IMAP server in the first place.

As things are now with TB, to search what Mozilla considers "custom headers" I have to log into my Roundcube webmail server, which has no problems searching anything.  It simply has a check box for "search entire message".  I wish TB's IMAP search options were so simple.  Most of my messages have larger headers than message bodies so there's really no performance hit compared to 'only' searching the headers.  And Dovecot has fantastic search indexes so the difference between searching headers and bodies is pretty much zero.  Roundcube isn't a suitable replacement yet as a daily mail client, but it's my only option currently for searching full headers.  That, or sync'ing the IMAP folders in TB, which I absolutely am not willing to do.  If I'd wanted to store all the mail locally I'd still use POP and local folders.

Comment 14

[Alfred Peters [:infofrommozilla]]

(In reply to comment #12)
> Can this bug's problem be called "regression produced by patch for bug 537820"?

I would say so. IMO dependency should be set to that Bug (I've not the rights to do so).

> > comment of patch for bug 537820. 
> > + // If both scopes work, honor the onlineSearch request, unless we're
> > + // filtering (quick search and/or a view selected)
> If comment is right, onlineSearch should be honored for custom header search at
> ordinal Search?
> Or online/custom header search didn't work at ordinal Search far before the
> change? 

IMO "Quick search" is that search field in the ToolBar. Possibly QS and "Search Message" use the same code.

I've done some debugging with Venkman:

| let filtering = this._userTerms != null || this._viewTerms != null;

"this._userTerms" is assigned an array with the search condition, so "filtering" is TRUE.

| scope = (!filtering && this.onlineSearch) ? serverScope : offlineScope;

"this.onlineSearch" is TRUE but "!filtering" is FALSE so scope is set to offlineScope.

This behavior maybe intended for quick search.

Comment 15

[WADA:World Anti-bad-Duping Agency]

CC-ing to David:Bienvenu who is owner of Bug 537820.
To David:Bienvenu, is this bug regession produced by your patch for Bug 537820?

Comment 16

[M]

The bug summary is misleading in that it gives the impression that it only affects searches with "Run search on server". In fact, it seems to me that the problem is "Cannot search on custom headers". Whether checking "Run on server" or not makes no difference. 

Or should there be a separate bug for "Cannot search IMAP on custom headers if not synching folders locally"?

A related problem is that the search dialog wrongly says "No matches found", giving the false impression that it did perform the requested search. It should say something like "Error: cannot search custom headers". Maybe that should be a different bug, but much better would be to just fix the "search custom headers" bug.

Comment 17

[Stan Hoeppner]

(In reply to comment #16)
> The bug summary is misleading in that it gives the impression that it only
> affects searches with "Run search on server". In fact, it seems to me that the
> problem is "Cannot search on custom headers". Whether checking "Run on server"
> or not makes no difference. 
> 
> Or should there be a separate bug for "Cannot search IMAP on custom headers if
> not synching folders locally"?

If folders are sync'd, you're not performing an IMAP search of custom headers, you're performing a local search of custom headers.  So, no, there shouldn't be a separate bug report.  If you are sync'd, but check "Run search on server" it's the exact same problem as if you're not sync'd.  The point being:  whether you're sync'd is irrelevant.  Searching of custom headers on IMAP server message data doesn't work.
 
> A related problem is that the search dialog wrongly says "No matches found",
> giving the false impression that it did perform the requested search. It should
> say something like "Error: cannot search custom headers". Maybe that should be
> a different bug, but much better would be to just fix the "search custom
> headers" bug.

I thought this was already discussed above, though I didn't re-read the comments.  I created this bug report many months ago and it seems nobody is working on it.  Low priority to the dev team I guess given all the other bugs.  I was very pleased to see the last report I filed just a week or two ago was picked up and solved within days.  It was in mail-news core though, which is much higher priority than this search bug.  Apparently very few people perform custom header searches against IMAP.  I wish many more did bumping the priority.  It sucks having to ssh into the server and grep the mbox files directly, then copy/paste.  I'd be really irritated if I didn't run my own mail servers.

Comment 18

[Bill Keegan]

(In reply to comment #17)
> Apparently very few people perform custom header searches against IMAP.
> I wish many more did bumping the priority.

The number is one more then before, vote early - vote often.

> It sucks having to ssh into the server and grep the mbox files directly,
> then copy/paste.  I'd be really irritated if I didn't run my own mail 
> servers.

Having my own mail server doesn't help me when I'm running Cyrus, solution uninstall TB 3 for now. Hello version 2, very quick actually.

Comment 19

[tektonaut]

Server-Side search isn't engaged at all, neither on Custom Headers nor in Body, To or CC. I also don't have my IMAP-Account synced, copying 6GB of Mailing-List archives to any lokal client also don't make any sense to me. Actually, server-site-searching is the reason why I use an IMAP-Server.

Server-side search now only works in search-folder. 

Searching and Filtering is by far the most useful application of a MUA an Thunderbird now breaks it down to unusability - at least for me. The Bug was filed in February and for now it is ot even assigned to anyone. Sry guys, srsly? :-/

Comment 21

[Emre]

I'm also affected by this bug as I have 15gb+ emails on IMAP server (dovecot) that I need to search without downloading them to Thunderbird. It works only with CTRL_SHIFT_F search window but it's cumbersome to use it.

Comment 22

[Stan Hoeppner]

(In reply to Emre from comment #21)
> I'm also affected by this bug as I have 15gb+ emails on IMAP server
> (dovecot) that I need to search without downloading them to Thunderbird. It
> works only with CTRL_SHIFT_F search window but it's cumbersome to use it.

You're confusing the problem described in this bug with a different issue.  Custom header search doe not work, period, unless you sync locally.  Please re-test and confirm.

Comment 23

[Emre]

I'm sorry for the confusion Stan, I understand that the quick search or other "gloda" search does not do server side search on all folders by default. So my comment is invalid.

Comment 24

[Stan Hoeppner]

(In reply to Emre from comment #23)
> I'm sorry for the confusion Stan, I understand that the quick search or
> other "gloda" search does not do server side search on all folders by
> default. So my comment is invalid.

You still don't grasp the scope of this bug report.  Note "custom header" in the title?  In SMTP there is no such thing as a "custom header".  Headers are headers are headers.  Mozilla uses the word "custom" to describe those headers which it does not copy to the local cache.  The bug here is that searchiing an IMAP folder for a "custom" header fails because the search is performed only on the local cache.  TB simply doesn't send the commands to the IMAP server.  This, is the bug.

Comment 25

[mitch deoudes]

I know that this is a crazy old bug, but I can confirm that it's still a problem in the latest T-bird.  I have a large IMAP folder in my Gmail account, and I am unable to search custom headers at all unless I sync it locally.  

Which for me sort of invalidates the point of accessing via IMAP in the first place...  It's a spam folder, so I don't want all 90 gajillion bytes of it on my laptop, and custom header searching is the only kind of searching I'm likely to do on it (for spamassassin-tuning purposes).

"Run on server" produces no results.  I can't say if that is a t-bird thing (c.f. comment #24), or a gmail thing.  Gmail doesn't allow arbitrary headers searches via the web interface, so no workaround other than downloading it all.

Comment 26

[Martin Fernau]

Still exists in TB 24.6.0

Comment 27

[Mario Splivalo]

Still exists in TB 31 (Running on Linux).

Comment 28

[Wayne Mery (:wsmwk)]

(In reply to Mario Splivalo from comment #27)
> Still exists in TB 31 (Running on Linux).

Yes. This is currently by design (well, a result of a previous patch).  When the design changes then a developer will update this bug report.


There is also Bug 564168 - RFE: Option to run all searches on the server when using an IMAP server

(In reply to WADA from comment #15)
> CC-ing to David:Bienvenu who is owner of Bug 537820.
> ... is this bug regession produced by your patch for Bug 537820?

It's unclear to me whether this question was ever resolved. If the problem described by this bug exists in 3.0 [1], then the likely cause is bug 511131.  If however the problem first exists in 3.0.1 [2], then the likely cause is Bug 537820

[1] http://download.cdn.mozilla.net/pub/mozilla.org/thunderbird/releases/3.0/
[2] http://download.cdn.mozilla.net/pub/mozilla.org/thunderbird/releases/3.0.1/

Comment 30

[Hungerburg]

Intermediate workaround: with the keyconfig addon installed one can create a custem shortcut (I use ctrl-b) to open a search window preset to "body". See http://forums.mozillazine.org/viewtopic.php?p=13510433#p13510433 - On the server side I made body search include other headers too. In the prefs.js file add:

user_pref("keyconfig.main.xxx_key1_Body_Search", "control][B][][openDialog(\
"chrome://messenger/content/SearchDialog.xul\", \"_blank\", \"chrome,resizable,status,centerscreen,dialog=no\",{ folder: gFolderDisplay.displayedFolder }).addEventListener(\"pageshow\", function tempFunction(event){ this.removeEventListener(event.type, tempFunction, false);\nvar searchAttr0 = this.document.getElementById(\"searchAttr0\");\nsearchAttr0.value = searchAttr0.valueIds[searchAttr0.valueStrings.indexOf(\"Inhalt\")];\nthis.document.getAnonymousElementByAttribute(this.document.getElementById(\"searchVal0\"), \"class\", \"search-value-textbox\").focus(); }, false); ][chrome://messenger/content/messenger.xul");

Comment 31

[WADA:World Anti-bad-Duping Agency]

FYI.
Customized View(View/Customize..., ,View/Custom Views) has similar issue.
  IMAP Offline-use=Off folder. No option of Online Search or Offline Search if View.
  If Body is not used, Custom header search doesn't work.
  If Body is used, Custom header search works.

If Search folder or Custom View, conditions can be pre-defined and can be modified, so "Body doesn't contain !!!???!!!" can be set in conditions always,.

Comment 32

[WADA:World Anti-bad-Duping Agency]

FYI.
Bug 861046 is RFE for "View as an alternative of Search wih better result presentation".
  Bug 861046 - Utilize View/Custom View for better "Advanced Search"

Comment 33

[WADA:World Anti-bad-Duping Agency]

(In reply to Hungerburg from comment #30)
> Intermediate workaround: with the keyconfig addon installed (snip)

 "Custom Buttons" addon is perhaps better because it's visible.
    Create my own Toolbar button, add it to Toolbar via Customize Toolbar, paste script code in button.
    Script: 1. Find Search Window via Window Enumerator, 2. Change requred field of Search Window as you want.

Comment 34

[WADA:World Anti-bad-Duping Agency]

FYI.
Another circumvention: Put required header name in customDBHeaders too.
   mailnews.customHeaders = X-XXX: Received: Resent-From: Content-Type: Sender: MIME-Version: Message-ID: List-ID
   mailnews.customDBHeaders = X-XXX: Resent-From: 
Headers in customHeaders is fetched upon header fetch of new mail, and is used by message filter, but is not saved in msgDBHdr of the mail.
Headers in customDBHeaders is fetched upon header fetch of new mail, and is saved in msgDBHdr of the mail.
This is a needed action for "After the Fact message filter on Custom header", because message filter doesn't have "Online Search" capability.

Comment 35

[WADA:World Anti-bad-Duping Agency]

Quick summary of currently known workaround.
(1) Put "Body doesn't contain !!!???!!!" in Search conditions of Advanced Search, with "Search Online" enabled.
(2) Put "Body doesn't contain !!!???!!!" in Search conditions of Customized View, and use Customized View for Search.
(3) Use Search Folder with "Search Online" enabled.
(4) Put header in mailnews.customDBHeaders, and do "Loclal Search".

Comment 36

[WADA:World Anti-bad-Duping Agency]

(In reply to mitch deoudes from comment #25)
> It's a spam folder, so I don't want all 90 gajillion bytes of it on my laptop,
> and custom header searching is the only kind of searching I'm likely to do on it (for spamassassin-tuning purposes).

If pre-defined condition is usable, and if "Search Subfolders" is not mandatory, I believe "Customized View" is best workaround or alternative for you, because search result display is far better than advanced Search.

I think current "base design of search" is "Search of View" :
  If Body is used, do Online search, else do Local search. "Single Search Target Folder" only is supported.
- Advanced Search :
  If "Online search" is prohibited, Online search is not executed even when Body is used. "Search Subfolders" is supported.
- Search Folder :
  If "Online search" is requested, do Online search, and if "Local search" is requested, do "Local search".
  "Search Subfolders" is not supported, but any folder of any account can be included in Search Target Folder List.

Let's stop adding comment of merely a complaint.
Which part of current implementation(code) is inconsistent for majority of Thunderbird users?
What is best spec of each search for majority of Thunderbird users?

Comment 37

[Alfred Peters [:infofrommozilla]]

Summary: 'Run search on server' of Edit/Find/Search Messages doesn't work any more, unless Body is also used

This bug has become a bit confused. As far as I can see, the main problem is that TB does not respect the Run search on server option when searching.
As I already wrote in comment #5 this is caused by bug 537820. Adding a "body" search term is only a workaround.

Bug 537820 prefers an offline search for quick search if offline and online search is available. But the bug overdoes it.

TB respects Run search on server for saved search folders, so it should work the same way for the search dialog.

Comment 38

[Alfred Peters [:infofrommozilla]]

In order not to break bug 537820 again, we have to distinguish from where the search was started.

If anyone sees an elegant way to do so: Patches are welcome.

Comment 39

[Alfred Peters [:infofrommozilla]]

Attachment: Take in account 'Run search on server' of Edit/Find/Search Messages

(In reply to Alfred Peters from comment #38)

In order not to break bug 537820 again, we have to distinguish from where the search was started.

This patch is a bit ugly. It checks the existence of this.owner._enteredFolder.
It is only available if the search is connected to an folder but not with the dialogue.

Comment 40

[Magnus Melin [:mkmelin]]

Comment on attachment 9183661 [details] [diff] [review]
Take in account 'Run search on server' of Edit/Find/Search Messages

Review of attachment 9183661 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/base/modules/SearchSpec.jsm
@@ +467,5 @@
>          // If both scopes work, honor the onlineSearch request, unless we're
>          // filtering (quick search and/or a view selected).
>          // If only one works, use it. Otherwise, default to offline
>          if (onlineAvailable && offlineAvailable) {
> +          scope = (!filtering || !this.owner.hasOwnProperty('_enteredFolder')) && this.onlineSearch ? serverScope : offlineScope;

Should add some comments above to explain.
But, should this be !this.displayedFolder instead of checking _enteredFolder? Or this.isVirtual?

Comment 41

[Alfred Peters [:infofrommozilla]]

(In reply to Magnus Melin [:mkmelin] from comment #40)

But, should this be !this.displayedFolder instead of checking _enteredFolder? Or this.isVirtual?

_	this.owner.isVirtual	this.owner.displayedFolder	!this.owner.displayedFolder
QuickSerach:	false	[xpconnect wrapped (nsISupports, nsIMsgFolder, nsIMsgImapMailFolder)]	false
SavedSearch:	true	[xpconnect wrapped (nsISupports, nsIMsgFolder, nsISupportsWeakReference)]	false
SearchDialog:	false	null	true

OK - !this.owner.displayedFolder will do.

Comment 42

[Alfred Peters [:infofrommozilla]]

(In reply to Magnus Melin [:mkmelin] from comment #40)

     // If both scopes work, honor the onlineSearch request, unless we're
     // filtering (quick search and/or a view selected).

Should add some comments above to explain.

Isn't that old comment wrong anyway?
'selected view' is a saved search right?
But we do (and should) honor the 'onlineSearch request' there.

Comment 43

[Magnus Melin [:mkmelin]]

I think selected view may refer to e.g. what we have in View | Folders | Unified

Comment 44

[Alfred Peters [:infofrommozilla]]

Attachment: Take in account 'Run search on server' of Edit/Find/Search Messages

I changed the condition to !this.owner.displayedFolder and adapted the Comment.

Comment 46

[Magnus Melin [:mkmelin]]

Comment on attachment 9184968 [details] [diff] [review]
Take in account 'Run search on server' of Edit/Find/Search Messages

Review of attachment 9184968 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks, looks good, but needs linting. r=mkmelin

Comment 47

[Alfred Peters [:infofrommozilla]]

Attachment: Take in account 'Run search on server' of Edit/Find/Search Messages

(In reply to Magnus Melin [:mkmelin] from comment #46)

Comment on attachment 9184968 [details] [diff] [review]

Thanks, looks good, but needs linting. r=mkmelin

done

Comment 48

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/2410a7cea917
Take in account 'Run search on server' of Edit/Find/Search Messages r=mkmelin

Comment 49

[pgnet.dev]

@mkmelin

fyi, after upgrade to

Name Thunderbird
Version 78.4.1
Build ID 20201104231226

'body' searches exec'd 'on server' DO appear to exec on the server. at least, at 1st check, I see the queries in my backend Solr logs.

otoh, 'subject' searches -- or any other header for that matter -- similarly exec'd 'on server' do NOT hit the backend at all; there's NO traffic from TB to the Solr instance.

Comment 50

[Magnus Melin [:mkmelin]]

This bug has been fixed for tb 84+. I don't know what you're reporting, but if needed, file a new bug about it.

Comment 51

[pgnet.dev]

I don't know what you're reporting,

A bug was filed. Previously. Against v78.

It's here:

https://bugzilla.mozilla.org/show_bug.cgi?id=1673928

You're CC'd on it.

It was closed, with comment:

"A fix for this is upcoming."

and marked as a DUPLICATE of this bug, 6 days ago ... immediately above your comment,

https://bugzilla.mozilla.org/show_bug.cgi?id=546925#c46

Comment 52

[Wayne Mery (:wsmwk)]

(In reply to pgnet.dev from comment #51)

I don't know what you're reporting,

A bug was filed. Previously. Against v78.

Because you reporting the same issue against an unfixed version, we don't need bug 1673928.
The patch next needs to go through beta, which will be beta 84.
And after that into version 78, perhaps 78.6.0 (it's not likely to make 78.5.0)

Comment 53

[Magnus Melin [:mkmelin]]

Comment on attachment 9185766 [details] [diff] [review]
Take in account 'Run search on server' of Edit/Find/Search Messages

[Approval Request Comment]
User impact if declined: per bug description
Testing completed (on c-c, etc.): cc and beta
Risk to taking this patch (and alternatives if risky): not too much risk, fairly confined

Comment 54

[Wayne Mery (:wsmwk)]

Comment on attachment 9185766 [details] [diff] [review]
Take in account 'Run search on server' of Edit/Find/Search Messages

[Triage Comment]
Approved for esr78

Comment 55

[Rob Lemley [:rjl]]

Thunderbird 78.5.1:
https://hg.mozilla.org/releases/comm-esr78/rev/c536b88adc03