293823 - "Search messages" doesn't work with unicode (even though "find in this message..." does)

Status: RESOLVED WORKSFORME

(MailNews Core :: Search, defect)

Description

[Andrey Beresnyak]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312
Build Identifier: Mozilla Thunderbird 1.0.2 (Macintosh/20050317)

I tried to search folder (message bodies) for some russian text, which I knew existed in some messages 
and failed. When I tried to search for separate letters the search returned some pages that were entirely 
in latin. 

I used unicode keyboard for entering text, which works anywhere else in Thunderbird. Since the option
"Fing in this message" worked just fine. I suspect that "Search Messages" do not recode its query 
according to the charset of every single message. This makes it possible to search only for english
text.





Reproducible: Always

Steps to Reproduce:
1. Go to "Edit -> Find -> Search Messages"
2. Search for any non-english text
3.

Actual Results:  
Incorrect or missing search results

Expected Results:  
search messages taking into account charset

Comment 1

[Stéphane Démurget]

I can also reproduce it: searching for japanese text works with "search in
message" but not with "search in folder"

I am using Thunderbird version 1.0.6 (20050727) included in Ubuntu Hoary.

Comment 2

[Stéphane Démurget]

btw, sorry to have forgotten that, but I'm using Ubuntu Linux on x86, so it's
not a MacOs-specific bug...

Comment 3

[Balázs Csapó]

I have a same result, even if I try to search for an English text:

My last try: 
I have had some messages contained the word "pixum". I would to know, how many such messages I have.
I searched for "pixum", I got 13 messages. I looked a messages source this (8 bit, text/plain, charset 8859-15) messages, but they didn't contained the word "pixum". The messages containing the word were not between the founded messages.

I am using Thunderbird, Version 1.5.0.8 (20061025), Hungarian.
OS is MS XP, version 5.1.2600

Comment 4

[Martin Fleurke]

I cannot find emails in my 'entire-message' search (which i should find) that have encoding quoted-printable.

Before, emails with the same content were sent without encoding header, and I can find those emails in my search. 

Looks like the encoding header has something to do with it, and not specific UTF-8 

An example email that can't be found when searching (e.g. for 'the'):
---------------------------------------------------------
From - Tue Sep 02 17:07:08 2008
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>
Delivered-To: [e-mail]
Received: (qmail 13710 invoked by uid 510); 2 Sep 2008 17:06:56 +0200
Received: from 192.168.1.217 by mail2 (envelope-from <[e-mail]>, uid 507) with qmail-scanner-1.25-st-qms 
 (-=PORTASCAN=- 
 Clear:RC:1(192.168.1.217):. 
 Processed in 0.02506 secs); 02 Sep 2008 15:06:56 -0000
X-Antivirus-XXXXX-Mail-From: [e-mail] via mail2
X-Antivirus-XXXXX: 1.25-st-qms (Clear:RC:1(192.168.1.217):. Processed in 0.02506 secs Process 13705)
Received: from unknown (HELO xxxxxx.nl) (192.168.1.217)
  by mail.xxxx.nl with AES256-SHA encrypted SMTP; 2 Sep 2008 17:06:56 +0200
Received: from xxx (localhost.localdomain [127.0.0.1])
	by xxx.nl (8.13.1/8.13.1) with SMTP id m82F6trb000791
	for <[e-mail]>; Tue, 2 Sep 2008 17:06:55 +0200
Date: Tue, 2 Sep 2008 17:06:55 +0200
Message-Id: <[e-mail]>
From: =?WINDOWS-1252?Q?"sendername"=20?=
 <[e-mail]>
To: =?WINDOWS-1252?Q?recipient=20?=
 <[e-mail]>
Subject: =?WINDOWS-1252?Q?The=20subjectline?=
X-Priority: 3
Content-Type: text/plain; charset=WINDOWS-1252
Content-Transfer-Encoding: quoted-printable
X-smtpproxy-01: xxxxx
X-Antivirus-XXXXX-1.25-st-qms: added fake MIME-Version header
MIME-Version: 1.0

the body
---------------------------------------------------

Using TB Version 2.0.0.16 (20080708)

Comment 5

[Ludovic Hirlimann [:Usul]]

Stephane, andrey are you still confornted to this issue ?

Martin I think your issue is currently being fixed in Thunderbird 3.0b3 ( see bug 249841)

Comment 6

[WADA:World Anti-bad-Duping Agency]

Local mail folder(POP3, "Local Folders")? IMAP folder?
If IMAP, IMAP folder with offline-use=On? Or IMAP folder with offline-use=Off?
If IMAP folder && offline-use=Off && non-quoted-printable / non-base64, see Bug 404255.
Sorry but I don't know about online search of IMAP folder && quoted-printable / base64 case.

Comment 7

[Martin Fleurke]

Hmm, I don't seem to be able to reproduce the failed search (quoted printable) with v 2.0.0.22, so it might have been fixed already?!

Comment 8

[Martin Fleurke]

I have reproduced the bug now; I have some mails that can't be found whatever I type as seach, and I have kind of identical mails that can be found. It does not matter if I copy the mail to a different IMAP folder; even if I copy it as single mail in an empty folder, I can search whatever I want, the mail cannot be found. 
The mails are error mails from a website, and for the same error I get mails that can be found and that cannot. I don't see the difference.

It seems to be fixed in Thunderbird 3.0b3, I can find the mails there when searching. Im not sure if bug 249841 was the cause, but I'm happy :)

Comment 9

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

-> WFM per comment #8.