Closed Bug 395100 Opened 17 years ago Closed 13 years ago

search messages, body, fails to find text in part of mail (IMAP, Online Search, Non-Gmail)

Categories

(MailNews Core :: Networking: IMAP, defect)

x86
Windows XP
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: wsmwk, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Attached file email for search body
version 3.0a1pre (2007082704)

search messages, body, fails to find text in part of mail.
doesn't find text "282930" for example.
message parts are separated as defined by by 
 boundary="FMT20502.1187913600/mx1.sinectis.com.ar"

message attached
version 3.0a2pre (2008071003)
Status: UNCONFIRMED → NEW
Ever confirmed: true
With Name=Thunderbird,Version=3.0a2pre,BuildID=2008071003 (on MS Win-XP SP3).
(a) Mail is held in local mail folder(dummy POP3 account) : WORKSFORME
(b) Mail is copied to Gmail IMAP folder                   : Confirmed
    (Usual folder. Other than [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam)

Gmail IMAP returned nothing for search of "Body contains 282930".
> 788[3ee2658]: 40c08d0:imap.gmail.com:S-Label-C/X1:SendData: 9 uid SEARCH UNDELETED BODY "282930"
> 788[3ee2658]: ReadNextLine [stream=41ae2a0 nb=10 needmore=0]
> 788[3ee2658]: 40c08d0:imap.gmail.com:S-Label-C/X1:CreateNewLineFromSocket: * SEARCH
> 788[3ee2658]: ReadNextLine [stream=41ae2a0 nb=33 needmore=0]
788[3ee2658]: 40c08d0:imap.gmail.com:S-Label-C/X1:CreateNewLineFromSocket: 9 OK SEARCH completed (Success)
Definition/meaning of "Body of a mail" is possibly different between IMAP server and Thunderbird.

To Wayne Mery(bug opener):
Local mail folder?  IMAP mail folder? If IMAP, Gmail IMAP?
(In reply to comment #2)
> To Wayne Mery(bug opener):
> Local mail folder?  IMAP mail folder? If IMAP, Gmail IMAP?

imap.  no gmail
david, see comment 2
Component: Mail Window Front End → Networking: IMAP
Product: Thunderbird → Core
QA Contact: front-end → networking.imap
Product: Core → MailNews Core
Summary: search messages, body, fails to find text in part of mail → search messages, body, fails to find text in part of mail (IMAP, Online Search, Non-Gmail)
Body search of recent Gmail IMAP looks to work well, as far as charset is appropriately specified in SEARCH command.

Gmail IMAP, Online search of "Body" for Japanese text in a message body(utf-8/base64 encoded):
(a) Folder Properties/General: Default character encoding: windows-1252
 => many of false positives(all of existent mails looks to be shown by Tb 7/trunk)
(b) Folder Properties/General: Default character encoding: utf-8
 => Gmail IMAP's online search returns mail correctly

Note: For this kind of issue, Bug 404255 is already opened.
> Bug 404255 Consider using UTF-8 when searching inside message body (IMAP online search) to avoid search failure

To Wayne(bug opener):

Because this bug was opened in 2008, comment #0 may be next;
  Body search of IMAP folder doesn't work, because no mail data is held locally
  if IMAP but body search searches locally.
  To search as expected, "online search" should be requested explicitly. 
Do you still see your problem in newest Tb release, with -safe-mode?
  If recent Tb, "Filter these messages: in Quick Filter Bar" is available,
  and this search looks to execute next;
    local search  if IMAP folder is offline-use=on folder
    online search if IMAP folder is offline-use=off folder
  And confusing but, there is another search of "Search All Messages" in Toolbar.
  When you check online search, use Search of folder context menu or Edit/Find/
  Search Messages, and explicitly request "online search", please.

What kind of characters do you search for? ascii only? non-ascii is involved?
If non-ascii is involved, can "Default character encoding: utf-8 in Folder Property/General" be a workaround of your problem?

I believe you already know IMAP logging of Tb, and I believe you know "IMAP log is almost mandatory for IMAP related problem analysis".
Can you attach IMAP log?

If local search, Bug 667854 can happens if mail contains URL(on both plain text mail and base64/quoted-printable encoded mail, because Bug 132340 is already fixed in 2009).
Please rule out "false negative in local body search" cases from your bug report, because this bug is for "false negative in online body search", if you check with recent Tb.
bugger, I've since deleted the message from the folder.

I was of course using the old quick filter at that time. Straight ascii search term. 

If you found no trouble, I'm happy to close it WFM.
Thanks for testing it.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → WORKSFORME
Wayne, I recently found bug 697021(false posive, body search searches header of next mail), and I've analyzed bug 667854(false negative in body search, if url with search keyword is contained in message text). Even though plain text mail, "Search All Messegaes in Toolbar"(Search by Gloda) doesn't look work if url with search keyword is contained in message text. Needless to say, bug 37031 already has many dupes. It doesn't look that "message/rfc822 part version of bug 37031" is opened, although some bug reports seem report of problem when message/rfc822 is contained in message. And I already know two problems in online search with Gmail IMAP(enhancement of bug 404255 is needed, and bug 500272). 

Further, I recently knew some bugs were wrongly duped to bug 379988 very quickly without analysis of both duping bug and duped bug.

So, I tried to rule out known causes of "false negative and false positive in body search" from bug reports of "body search fails to find" or "body search returns wrong result". And, I tried to rule out known causes in your bug too.

It seems that we must utilize meta bug 519202 opened by you clevery.

Anyway, we look successful in closing of a hard-to-analyze old bug listed in dependency tree for meta bug 519202 :-)
(In reply to Wayne Mery (:wsmwk) from comment #0)
> Created attachment 279814 [details]
> email for search body

Content-Type: multipart/report, so it's same as multipart/mixed, from view poit of mail strucure.

Checked with Tb 7.0.1, using local mail folder.

Next line is contained in message/rfc822 part.
> Message-ID: <46CDE629.7090200@lehigh.edu>
So, because body search searches message/rfc822 part, Body Search finds "Message-ID: <46CDE629.7090200@lehigh.edu>" of this mail.
This is an example of complaint of "header is searched".

Because even header line in message/rfc822 part is succesufully searched currently, message body line in message/rfc822 part is also searched, then problem like original problem of comment #0 is already worksforme, if local mail folder. 

However, "Search all messages... in Toolbar" can't find this mail.
This is an example of "different search result between Gloda search and ordinal body search".

Next line is also contained in message/rfc822 part.
> [V] Version, build string pasted from  help|about  tells us:
"Body search of Quick Search in Quick Filter Bar" can find next string.
> [V] Version, build string pasted from help|about tells us:
But "Search all messages... in Toolbar" can't find this. 

Next line is also seen in message body text of message/rfc822 part.
> more: https://bugzilla.mozilla.org/page.cgi?id=fields.html#resolution
Above problem is perhaps same as one I saw on test mails for bug 667854 with "Search all messages... in Toolbar". It occurs on test mail of bug 667854 even when message body data is plain text(not base64/quoted-printable encoded).

Wayne, thanks for your actual sample mail which is useful for problem anlysis of other many stil-unclear bug reports.

I never want to morph original problem of comment #0, so I keep WORKSFORME.
I've opened bug 700541 for issue of "body search of message/rfc822 part", for ease in problem analysis of many bug reports which may be relevant to body saerch and message/rfc822 part.
My observation on "Search all messages... in Toolbar" case in comment #9 was absolutely wrong.

In my test cases, text string in message body was {FINDSTRING} and [SEARCHTEXT].
If serch for FINDSTRING or SEARCHTEXT, "Search all messages... in Toolbar" found all mails, regrdless of content-transfer-encoding.
I forgot charcteristics of tokenizer used for global-messages-db.sqlite.
As the search is by Gloda's search, serch is done for full TERM(continuous non-space/non-special ascii characters if ascii, minimum of two or three chars?). So, search fails for substring of FINDSTRING and substring of SEARCHTEXT.
To improve this, search such as "term start with", "term end with", and "term contains" is mandatory. IIRC, it's still difficult with current extention of SQLite, although it's not impossible. I forgot this kind of current restriction.

Sorry for my confusion.
Bug I forgot was Bug 598605 to which I posted comments last year.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: