Steps to reproduce:
get your Messenger window;
click on the Inbox of the local mail folder;
click on the Subject to get it sorted (one click ascending, another click
//note: all high ascii are added at the end
for ex :
Yahoofr firstname.lastname@example.org 5/8/99 read
ç'est la vie email@example.com 6/1/99 read
Marina, could you zip your inbox and attach?
There is a field "Attachments: Create a new attachment (proposed patch,
testcase, etc.)", you can click and attach a file.
Created attachment 615 [details]
a zip file of a mailbox for sorting test
The problem is because collation key is generated from MIME encoded string. It
needs to be decoded before the key generation.
This doesn't seem to be a problem newly created after the sort was once broken
I checked with Marina but she has not seen it (i18n sort) working before.
Reassigning to firstname.lastname@example.org.
I'm sorry; I didn't understand the last comment. Are you saying I need to decode
the mime encoded string and then generate a collation key from the decoded
string? I think I just plugged in the patch someone sent me so I'm a little out
of my depth here.
Yes, it needs to be decoded. In 4.5, i18n provided a function to MIME decode
also create a collation key. But 5.0, they are separate interfaces.
I looked into sort today and after clicking sort by Subject my system runs out
of virtual memory and all processes after it taking several minutes to be
Not going to happen in m8
Should be fixed when the mailnews Necko branch lands. I'm now doing the decode
David, is it the right time to verify the fix (because i don't see it working
yet)or wait for Necko/M9 build?
Yes, wait for the m9/Necko build. We haven't landed necko yet, though it's
supposed to happen today.
** Checked with 8/6/99 Win32 build **
Now that we decode MIME-encoded headers first before sorting
them using locale-specific APIs, we see dramatic improvement in
I looked at 2 platforms with both IMPAP and POP3 : WinNT-US and
WinNT-Japanese. Here are the results:
1. On NT-US, Latin 1 and ASCII headers are sorted in such
a way as to put the base character like "a" right before
an accented "a" rather than placing accented ones at the end.
If non-Latin characters exist in the same folder, e.g.
Japanese headers, they are sorted according to Uniocde point
2. On Japanese NT4, the sort is done according to JISX208 mostly
but Kana characters are done according to pronunciation.
This is also a desired result.
Locale-sensitive sort was already working but now the input into
the sort routine is also correct as the strings are first
Based on these results, I'm going to mark this fix verified.
It is working for subjects but not addresses.
For example, the below address is sorted at the top (seems like not MIME
From: =?iso-8859-1?Q?N=E1oki?= Hotta <email@example.com>
IQA, please file a separate bug or reopen this, thanks.
Naoki, I think we should deal with this additional issue when Bug 8405 is
fixed. At present, I only see Japanese characters as "dots" in the Sender field
display of the thread pane. It is very difficult to test the Sender field sort without the
I thought this bug was about whether not the MIME-decode itself was being done.
And for that, my testing indicates that it is indeed occurring.
Created attachment 1263 [details]
i18n message sort data
Using my local build (pulled 8/16) and the attached data, Náoki Hotta is sorted
before Bob Jung.
I'm a bit confused about this.
In Bug 8405, nhotta mentions that the MIME-decode is done in the Sender
field. But in this bug, MIME-decode is not done.
Is this a matter of timing as to where the sorting key is created
prior to MIME-decoding or post-decoding?
Since Latin 1 data nhotta provided provides justification,
I'll re-open this for the Sender field sort.
>Is this a matter of timing as to where the sorting key is created
>prior to MIME-decoding or post-decoding?
MIME decode for display and sort key creation may be done separately. They were
done separately in 4.x.
I don't believe I can fix this with the current api's that exist. Sorting needs
to be by the pretty name (not the e-mail address). To get the pretty name, I
need to use nsIMsgHeaderParser::ExtractHeaderAddressName, which takes a char *
string, and returns a char *string, after converting to utf-8, I believe. So I
can't decode before calling that routine, and it's too late to decode after. Any
suggestions? I don't believe there's anything I can do here.
nsIMsgHeaderParser::ExtractHeaderAddressName uses utf-8 for internal processing
I think it is possible to call the decoder,
nsMimeConverter::DecodeMimePartIIStr(const char *header,
then call nsIMsgHeaderParser::ExtractHeaderAddressName with returned charset and
If I understand you correctly, this would be a change to libmime, so I'm
reassigning to Rich, on the off-chance he can get to it before sabbatical.
>If I understand you correctly
I don't thik so... I said the caller (not libmime) could decode before getting
the pretty name.
Should this be target milestone M9? I realize Rich is going on sabbatical
From Naoki's comments, it looks like the caller needs to change the way in
which DecodeMimePartIIStr() is called. David, if I'm mistaken, let me know.
Not m9, for sure!
OK, I have a fix for this - it's quite involved, unfortunately, but I'll check
it in tonight if the tree opens...
fixed more or less as nhotta suggested, though I had to jump through a lot of
hoops to get the data in the right form.
MIME-decoding does not seem to be taking place for the final
M9 build (8/24/99) but seems to be working for the latest M10
build. I assume that this did not get into M9, and will verify
this later for M10.
If the target fix milestone says m10, that means it's fixed in m10 - our QA
group is very picky about that. So it's important to pay attention to the target
fix version when verifying a "resolved fixed" bug. It will save you time.
** Checked with 9/17/99 Win32 build **
Locale-sensitive sorting is working on Win-NT4 US and WinNT-J
well for Latin 1 and Japanese Subject and Sender fields with the
above build. On Win98 and Win95, locale-sensitive part is not
working yet but we get Unicode sort which is decent for Japanese.
The Sender field sort is working OK now for Latin 1 showing
that the MIME-decoder is engaged before sort keys are generated.
For Japanese names in the sender field, a display bug prevents us
from verifying this feature. This should be looked at when Bug 8405