68630 - POP mail search by body: search in a child folder should use its own folder charset

Status: VERIFIED INVALID

(MailNews Core :: Internationalization, defect)

Description

[ji]

*****observed with win32 and linux 2001-02-12-06-mtrunk build.******

Now local mail search is using the folder charset. But the search in a child
folder is looking at its
parent's charset instead of its own. This problem only exists in POP mail
search. It doesn't happen to
IMAP mail search.

Steps to reproduce:
1. Setup a POP mail account.
2. Create a folder containing an ISO-2022-JP mail. Have some Japanese string in
the mail body, like ???
3. Create a child folder under the folder and have the child folder containing a
mail with a Ja string partially matched the one in its parent folder, like ??.
4. Set the parent's folder charset as ISO-2022-JP.
5. Set the child's folder charset as Western (ISO-8859-1)
6. Go to Search | Search Mail/News Messages, set the search criteria as "Body
contains <??>" and make sure you have checkmark on Search subfolders. Click on
Search button, you'll get two mails listed,
one is from the parent folder and the other is from the child folder. Since the
child folder's charset
is not set to ISO-2022-JP, it shouldn't get its mail listed.
7. Do the same testing for an IMAP account, only the mail in the parent folder
is listed.

Comment 1

[ji]

Sorry, the Ja strings in the report are not displayed. I'll attach a testing
folder later.

Comment 2

[nhottanscp]

>7. Do the same testing for an IMAP account, only the mail in the parent folder
>is listed.
So no search for sub-folder? Taka, is this a spec of IMAP server?

Anyway, we need to decide the desired behavior for sub folder search. I think
the current behavior is not so bad. Use sub-folder charsets would be more
complex for the user.

Comment 3

[ji]

Attachment: A zipped testing folder.

Comment 4

[ji]

Unzipped the attached testing folder and put it in your pop mail account. You'll
have two folders showing up: parent and child.
Create a folder under parent and copy the message from "child"  folder
to the new created subfolder under parents.
Search mail body by Japanese string "nihon"

Comment 5

[ji]

It would be easier for users if we can make the behaviour be consistent between
IMAP and POP account.

Comment 6

[nhottanscp]

Currently, they are consistent both use parent charset.
Imap does not search sub-folder, please test if that's also true for ASCII mails.

Comment 7

[marina]

POP and IMAP always had different specs: same applies to non-ascii search. Pop
will return you the exact match whereas IMAP will return ascii and non-ascii strings

Comment 8

[ji]

Naoki, I checked with ASCII mails. IMAP searches subfolder.

Comment 9

[nhottanscp]

Thanks, so IMAP does search sub-folders. It uses a sub-folder charsets to
generate a query string, sorry for the confusion.

Taka, sorry we did not talk about the detail before the implementation. Could
you tell us the current implementation both IMAP and local search about folder
charset?

Comment 10

[ji]

Marina, are you talking about the search by latin-1 string?

Comment 11

[Takayuki Tei]

The current implemenation relies on folder charset only when 'Content-Type'
header doesn't have 'charset' sub type. In the attached case, both message
have 'charset' sub type, thus, it's expected behavior.  Try having multipart
message instead as test case.

Comment 12

[ji]

That's right. After I removed the charset subtype, search uses folder charset.
But why IMAP search doesn't look at the charset subtype? 

Comment 13

[nhottanscp]

I think the IMAP also uses the 'Content-Type' charset, but the quesry string
generated by the client is encoded with the folder charset.
We could send a query string as UTF-8 but not every server can understand UTF-8
query. So there was a plan to do it as a fallback but I don't think it's
implemented currently.

Comment 14

[nhottanscp]

So the folder charset is used when the message is not labeled.
Charset override for search is filed separately as bug 68344.

Comment 15

[ji]

Marked it as verified.