257511 - Showing date failed with UTF-8 locales that includes non-ASCII characters in date strings

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Takanori MATSUURA]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a3) Gecko/20040831 Firefox/0.9.1+
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a3) Gecko/20040831 Firefox/0.9.1+

When I access any FTP sites, mozilla shows the directories and files list except
for day, month, and year.

Reproducible: Always
Steps to Reproduce:
1. access to the FTP sites (eg. ftp://ftp.mozilla.org/)
2.
3.

Actual Results:  
shows the directories and files list WITHOUT day, month, and year.

Expected Results:  
shows the directories and files list WITH day, month, and year.

I tested with the browsers
  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a3) Gecko/20040831 Firefox/0.9.1+
and
  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a3) Gecko/20040831

I also tested with
  Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8a3)
Gecko/20040830 Firefox/0.9.1+
and this seems to be worked fine.

Comment 1

[Takanori MATSUURA]

Attachment: Source that mozilla got.

I compare both Linux and MacOS X source and no difference found.

Comment 2

[Takanori MATSUURA]

Attachment: HTML-parsed source. Linux

resulting source that mozilla treats.
Day, month, and year fields are empty.
Time shows 24-hour style.

Comment 3

[Takanori MATSUURA]

Attachment: HTML-parsed source. Mac OS X

Resulting source that mozilla on MacOS X treats.
Day, month, and year fields are filled with "yy.yy.dd" style.
Time shows 12-hour style with AM/PM.

Comment 4

[Takanori MATSUURA]

(In reply to comment #3)
> Day, month, and year fields are filled with "yy.yy.dd" style.

"yy.mm.dd" style. Sorry.

Comment 5

[Takanori MATSUURA]

firefox-1.0 works fine. But trunk
Mozilla/5.0 (X11; U; Linux i686; ja-JP; rv:1.8a6) Gecko/20050111
does not work properly even now.

       +--mozilla1.7.5 (good)
       |
       |  +--(20040515?)--firefox1.0 (good)
       |  |
trunk--+--+--20040831_trunk (no_good)

Comment 6

[Takanori MATSUURA]

I tried more tests.

       +--mozilla1.7.5 (good)
       |
       |  +--(20040515?)--firefox1.0 (good)
       |  |
trunk--+--+--mozilla1.8a1 (good)--mozilla1.8a2 (no_good)

Comment 7

[Takanori MATSUURA]

Date is not displayed with the locale ja_JP.UTF-8 and displayed with ja_JP.eucJP.

Date is also not displayed
  1. "Modified" and "Expires in View -> PageInfo
  2. Date header in Mozilla Mail

Was the date detection mechanism changed between mozilla-1.8a1 and mozilla-1.8a2?

Comment 8

[Hidehiro Kozawa]

In Japanese bugzilla, Some poeple says that bug 153586 may cause this.

>jshin
All CJK have same problem.

Comment 9

[Jungshik Shin]

You're right. This has regressed as a reult of the fix for bug 153586, which
also broke Date/Time in Calendar/Sunbird when run under ll_CC.UTF-8. The reason
is that in the patch '#PLATFORM' was rmoved and there's no more distinction
between 'll_CC' locale and 'll_CC.UTF-8' locale. To be fair, bug 153586 fixed
one thing, but broke another.   It's  my fault as well because I never manage to
point out what exactly would break with the patch although I knew it would break
something.  (bug 153586 comment #67, bug 153586 comment #71). The file in
question was shuffled around (so that Lxr didn't help) and I lost track of bugs
in which '#PLATFORM' was added to work around this kind of problem.

Under 'll_CC.UTF-8' locale, |strftime| is called with the locale set to 'll_CC'
(e.g. ja_JP), but the platform character encoding is 'UTF-8' so that down the
road the result of strftime (which is in EUC-JP) is interpreted as UTF-8, which
would obviously fail. 

Just adding back '#PLATFORM' may solve the original issue reported here (FTP
date/time), but doesn't quite solve JS date/time (Sunbird/Calendar cases) for
some reason. I looked at this a week ago, but didn't figure out why, then.

Comment 10

[Jungshik Shin]

Rolling back all the changes in intl/locale/* in attachment 146210 [details] [diff] [review] fixes this
problem and doesn't break |Date.prototype.toLocaleString| in javascript in both
ko_KR.EUC-KR locale and ko_KR.UTF-8 locale on Linux. However, repeated 'trips'
to | nsPlatformCharset::GetDefaultCharsetForLocale| are bothering me.

BTW, I guess I have to deal with  Date/time problem in sunderbird/calendar in
another bug. With the same change, I'm getting 'C' locale date/time while
without it, I get an empty string as is the case for ftp file listing.

  

Comment 11

[Jungshik Shin]

Attachment: patch

This is basiscally reverting to pre-bug 153586 as far as intl/* is concerned.
The strange thing I observed was due to that I forgot to change
intl/uconv/src/nsUNIXCharset.cpp. The reason for the roll-back is that on Unix,
we need to keep the distinction between ll_CC and ll_CC.UTF-8 locales. [1]  In
the current code, we lose the distinction while going back and forth between
xpLocale designation 'll-CC' and posix locale designation. 

An alternative to this is to modify the converter between Posix locale names
and xpLocale names. We may attach and detach the character encoding name
returned by nl_langinfo(CODESET) (in addition to going back and forth between
'-' and '_'). However, this is rather tricky because there's no guarantee that
the codeset name returned by nl_langinfo() is also used in the locale name. On
Linux (glibc 2.3.x), it does, but I wouldn't believe that it's always the case
on other Unix. 

Btw, LocaleToUnicode in dom/src/base/nsJSEnvironment.cpp had better use
NSI_LOCALETIME##PLATFORM for XP_UNIX although it would work (as it is) in most
cases.


[1] I found that locales like hi_IN, ta_IN and hi_IN.UTF-8 and ta_IN.UTF-8 are
equivalent in glibc 2.3.x because glibc never supported hi-IN and ta-IN before
it began to support UTF-8 locales.

Comment 12

[Jungshik Shin]

This is definitely a blocker for 1.8beta and TB 1.1. Date field is blank for
mail (in both mail thread panel and individual message view) when run under
UTF-8 locales where date/time have non-ASCII characters.
 

Comment 13

[Jungshik Shin]

thansk for r/sr. fix checked in