239535 - [Mac] nsDateTimeFormatMac is broken in Japanese locale (Page Info fails to load / Media Tab doesn't work / Cookie Manager doesn't show "Expires")

Status: RESOLVED FIXED

(Core :: Internationalization, defect, major)

Description

[HARUNAGA Hirotoshi]

Page Info never finish loading at a certain page on Mac.

Steps:
1. Go to the URL above.
2. Open View -> Page Info.

JavaScript Console shows as follows:
Error: uncaught exception: [Exception... "Component returned failure code:
0x8000ffff (NS_ERROR_UNEXPECTED) [nsIScriptableDateFormat.FormatDateTime]" 
nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame ::
chrome://navigator/content/pageInfo.js :: formatDate :: line 1017"  data: no]

Confirmed with 2004040105/MacOS 10.3.3.
2004031605/Mac has no problem.
2004031705/Mac has this problem.
regression.

Comment 1

[HARUNAGA Hirotoshi]

On this page (Bug 239535), or http://www.mozillazine.org/,
loading can be done, but if you click Media tab in Page Info,
same JavaScript error occurs, and we can't use Media tab on Mac.

Comment 2

[HARUNAGA Hirotoshi]

Is this caused by Bug 233631?

Comment 3

[Constantine A. Murenin]

(In reply to comment #2)
> Is this caused by Bug 233631?

I'll try to do some testing. Apparently, I just changed
nsScriptableDateFormat::FormatDateTime to do the same thing as
nsIDateTimeFormat::FormatTime does, so the bug must be elsewhere. 

Comment 4

[Constantine A. Murenin]

I cannot reproduce the problem. (I only have access to OS X 10.2.8.) 

Could you provide some more details? Does this problem occur in the Cookie
Manager or in any other instances where the date is shown? Are there any
assertions? 

Comment 5

[HARUNAGA Hirotoshi]

Is this Mac OS 10.3.x only?

I can still reproduce comment 0 and comment 1
with 2004040805/Mac OS 10.3.3, even if I create a new profile.

Cookie Manager is also strange as expected by Constantine.
In "Stored Cookie", "Expires" are not displayed at all
although JavaScript Console shows nothing.

Comment 6

[Hiro]

It reproduced.

Mac OS X 10.3.3 + New Profile
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7b) Gecko/20040408
Javascript Console error message:
Error: uncaught exception: [Exception... "Component returned failure code:
0x8000ffff (NS_ERROR_UNEXPECTED) [nsIScriptableDateFormat.FormatDateTime]" 
nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame ::
chrome://navigator/content/pageInfo.js :: formatDate :: line 1017"  data: no]

Comment 7

[Constantine A. Murenin]

Moving to intl land.

Could someone provide ASSERTIONs that occur when this bug happens? 

Comment 8

[HARUNAGA Hirotoshi]

Attachment: console.log with debug build on Mac OS 10.3.3

Comment 9

[HARUNAGA Hirotoshi]

If I set "English" as most preferred language in Mac OS "System Preferences"
-> Personal -> International -> Language, log out and log in,
then this problem disappears.
Loading is done, and Type, Source, Size, Expires, Dimensions, image
are displayed in Media tab.
And Cookie Manager also works fine.

In "Japanese" language, this problem occurs.

Comment 10

[HARUNAGA Hirotoshi]

requesting blocking 1.7 because Page Info on Firefox 1.0/Mac will not
work for Japanese locale users.

Comment 11

[Constantine A. Murenin]

Attachment: commands for JavaScript Debugger

I think we need to change the summary somehow to reflect the problem (like
"nsIScriptableDateFormat is broken in Japanese locale"). 

We need to narrow this problem. I do not have Japanese locale on the Macs I
have access to, so I cannot reproduce this problem at all. 

I am not sure that the following gives any help, but since no one else says
anything, I'll take the word. 

Reporters: try to open JavaScript Debugger and execute the lines from this
attachment. I am not sure that it will do any help, but it may give us some
idea of what the problem is. Try to use some other locales, combinations, etc. 


Also, please report if the time shows in MailNews and bookmarks manager.

Comment 12

[HARUNAGA Hirotoshi]

It seems to me that MailNews and Bookmark Manager don't have this
problem.
JavaScript Debugger shows as follows on Japanese locale.
This error doesn't occur on English locale.

0001: z =
Components.classes["@mozilla.org/intl/scriptabledateformat;1"].createInstance();
[xpconnect wrapped nsISupports]
0002: for (a in Components.interfaces) z instanceof Components.interfaces[a];
false
0003: var date = new Date();
undefined
0004: z.FormatDateTime("", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
[Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED)
[nsIScriptableDateFormat.FormatDateTime]" nsresult: "0x8000ffff
(NS_ERROR_UNEXPECTED)" location: "JS frame ::
chrome://venkman/content/venkman-eval.js :: con_eval :: line 50" data: no]
0005: z.FormatDateTime("en-GB", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
Thu, Apr 29, 2004 9:23:00 am
0006: z.FormatDateTime("en-US", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
Thu, Apr 29, 2004 9:23:00 AM
0007: z.FormatDateTime("ru", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
Чт
0008: z.FormatDateTime("ja-JP", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
2004年 4月 29日 (木) 9:23:00 AM
0009: z.FormatDateTime("ko-KR", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
Thu, Apr 29, 2004 9:23:00 AM
00010: z.FormatDateTime("zh-CN", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
2004年4月29日週四, 9:23:00 AM
00011: z.FormatDateTime("", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
04.4.29 9:23:00 AM
00012: z.FormatDateTime("en-GB", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
29/4/04 9:23:00 am
00013: z.FormatDateTime("en-US", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
4/29/04 9:23:00 AM
00014: z.FormatDateTime("ru", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
29.04.2004 09:23:00
00015: z.FormatDateTime("ja-JP", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
04.4.29 9:23:00 AM
00016: z.FormatDateTime("ko-KR", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
4/29/04 9:23:00 AM
00017: z.FormatDateTime("zh-CN", z.dateFormatShort, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
2004/4/29 9:23:00 AM
00018: z.FormatDateTime("", z.dateFormatYearMonth, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
04/04 9:23:00 AM
00019: z.FormatDateTime("", z.dateFormatWeekday, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
木 9:23:00 AM

Comment 13

[HARUNAGA Hirotoshi]

JavaScript Debugger on 1.6/Mac showed as follows in Japanese locale.

0004: z.FormatDateTime("", z.dateFormatLong, z.timeFormatSeconds,
date.getFullYear(), date.getMonth()+1, date.getDate(), date.getHours(),
date.getMinutes(), date.getSeconds());
2004年 5月 1日 (土) 10:14:12 PM

So default(?) dateFormatLong is broken in Japanese locale.

Comment 14

[Constantine A. Murenin]

(In reply to comment #13)
> So default(?) dateFormatLong is broken in Japanese locale.

No, nsDateTimeFormatMac is broken for the long date format, iff the locale is
Japanese and we do not pass the locale to nsDateTimeFormatMac explicitly (as I
can see from your comment 12). It just happens this way that no other place of
C++ Mozilla uses long date format, so it only shows up in JavaScript, i.e. in
nsIScriptableDateFormat. 

Here is the link for the code in question:
http://lxr.mozilla.org/mozilla/source/intl/locale/src/mac/nsDateTimeFormatMac.cpp

I will not have any time to look at it before next Tuesday, 2004-05-04. 

Comment 15

[Asa Dotzler [:asa]]

not going to block the release for this but we'd consider taking a fully
reviewed patch if someone can come up with one before 1.7 ships.

Comment 16

[HARUNAGA Hirotoshi]

*** Bug 240150 has been marked as a duplicate of this bug. ***

Comment 17

[Constantine A. Murenin]

removing duplicate from dependencies

Comment 18

[HARUNAGA Hirotoshi]

When opening Page Info at http://www.mozilla.org/ in Japanese locale
on Mac 10.3.4,  I can see the followings which are different from those
in English locale.

^C
Program received signal SIGINT, Interrupt.
0x900074c8 in mach_msg_trap ()
(gdb) b nsDateTimeFormatMac::FormatTime
Breakpoint 1 at 0x48201a8: file nsDateTimeFormatMac.cpp, line 330.
(gdb) c
Continuing.
Reading symbols for shared libraries . done
Reading symbols for shared libraries . done
WARNING: the property eo already exists
, file nsPersistentProperties.cpp, line 282
Document http://www.mozilla.org/ loaded successfully
++WEBSHELL == 4
++DOMWINDOW == 4
XP Popups: This is a nag to indicate that an inconsistent hack is being done on
the Mac for popups.
WARNING: trying to find screen for sizeless window, using primary monitor, file
nsScreenManagerMac.cpp, line 119

Breakpoint 1, nsDateTimeFormatMac::FormatTime(nsILocale*, int, int, long,
nsString&) (this=0x10a89090, locale=0x0, dateFormatSelector=1,
timeFormatSelector=1, timetTime=1085094152, stringOut=@0xffcd43c) at
nsDateTimeFormatMac.cpp:330
330       return FormatTMTime(locale, dateFormatSelector, timeFormatSelector,
localtime(&timetTime), stringOut);
(gdb) n
331     }
(gdb) n
nsScriptableDateFormat::FormatDateTime(unsigned short const*, int, int, int,
int, int, int, int, int, unsigned short**) (this=0xffcd430, aLocale=0x1117a90,
dateFormatSelector=1, timeFormatSelector=1, year=2004, month=5, day=21, hour=8,
minute=2, second=32, dateTimeString=0x10a89080) at nsScriptableDateFormat.cpp:153
153       if (NS_SUCCEEDED(rv))
(gdb) n
156       return rv;
(gdb) p rv
$2 = 2147549183
(gdb) n
157     }
(gdb) n
0x01c44a84 in _XPTC_InvokeByIndex () at nsTObsoleteAStringThunk.cpp:51
51            static const void* get_vptr()
(gdb) c
Continuing.
JavaScript error: , line 0: uncaught exception: [Exception... "Component
returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED)
[nsIScriptableDateFormat.FormatDateTime]"  nsresult: "0x8000ffff
(NS_ERROR_UNEXPECTED)"  location: "JS frame ::
chrome://navigator/content/pageInfo.js :: formatDate :: line 1013"  data: no]

Comment 19

[Constantine A. Murenin]

(In reply to comment #18)
> (gdb) b nsDateTimeFormatMac::FormatTime

Could you try to break it on FormatTMTime instead? That is the function that the
error occurs. Thanks. 

Comment 20

[HARUNAGA Hirotoshi]

(In reply to comment #19)
> Could you try to break it on FormatTMTime instead?

^C
Program received signal SIGINT, Interrupt.
0x900074c8 in mach_msg_trap ()
(gdb) b nsDateTimeFormatMac::FormatTMTime
Breakpoint 1 at 0x483cabc: file nsDateTimeFormatMac.cpp, line 355.
(gdb) c
Continuing.
Reading symbols for shared libraries . done
Reading symbols for shared libraries . done
WARNING: the property eo already exists
, file nsPersistentProperties.cpp, line 282
Document http://www.mozilla.org/ loaded successfully
++WEBSHELL == 4
++DOMWINDOW == 4
XP Popups: This is a nag to indicate that an inconsistent hack is being done on
the Mac for popups.
WARNING: trying to find screen for sizeless window, using primary monitor, file
nsScreenManagerMac.cpp, line 119

Breakpoint 1, nsDateTimeFormatMac::FormatTMTime(nsILocale*, int, int, tm const*,
nsString&) (this=0x10d851d0, locale=0x0, dateFormatSelector=1,
timeFormatSelector=1, tmTime=0x115bed0, stringOut=@0x102c90bc) at
nsDateTimeFormatMac.cpp:355
355       (void) Initialize(locale);
(gdb) n
358       if (dateFormatSelector == kDateFormatNone && timeFormatSelector ==
kTimeFormatNone) {
(gdb) n
363       stringOut.AssignWithConversion(asctime(tmTime));      // set the
default string, in case for API/conversion errors
(gdb) n
366       NS_ASSERTION(tmTime->tm_mon >= 0, "tm is not set correctly");
(gdb) n
367       NS_ASSERTION(tmTime->tm_mday >= 1, "tm is not set correctly");
(gdb) n
368       NS_ASSERTION(tmTime->tm_hour >= 0, "tm is not set correctly");
(gdb) n
369       NS_ASSERTION(tmTime->tm_min >= 0, "tm is not set correctly");
(gdb) n
370       NS_ASSERTION(tmTime->tm_sec >= 0, "tm is not set correctly");
(gdb) n
371       NS_ASSERTION(tmTime->tm_wday >= 0, "tm is not set correctly");
(gdb) n
373       macDateTime.year = tmTime->tm_year + 1900;
(gdb) n
377       macDateTime.month = tmTime->tm_mon + 1;
(gdb) n
378       macDateTime.day = tmTime->tm_mday;
(gdb) n
379       macDateTime.hour = tmTime->tm_hour;
(gdb) n
380       macDateTime.minute = tmTime->tm_min;
(gdb) n
381       macDateTime.second = tmTime->tm_sec;
(gdb) n
385       macDateTime.dayOfWeek = tmTime->tm_wday +1 ; 
(gdb) n
387       ::DateToSeconds( &macDateTime, (unsigned long *) &dateTime);
(gdb) n
390       Handle itl1Handle = mUseDefaultLocale ? nil : (Handle)
GetItl1Resource(mScriptcode, mRegioncode);
(gdb) n
391       Handle itl0Handle = mUseDefaultLocale ? nil : (Handle)
GetItl0Resource(mScriptcode, mRegioncode);
(gdb) n
394       if (timeFormatSelector != kTimeFormatNone) {
(gdb) n
396         if ( itl0Handle &&
(gdb) n
406           ::TimeString(dateTime, (timeFormatSelector == kTimeFormatSeconds),
timeString, itl0Handle);
(gdb) n
411       switch (dateFormatSelector) {
(gdb) n
413           ::DateString(dateTime, abbrevDate, dateString, itl1Handle);
(gdb) n
414           break;
(gdb) n
433       if (dateFormatSelector != kDateFormatNone && timeFormatSelector !=
kTimeFormatNone) {
(gdb) n
434         CopyPascalStringToC(dateString, localBuffer);
(gdb) n
435         localBuffer[dateString[0]] = ' ';
(gdb) n
436         CopyPascalStringToC(timeString, &(localBuffer[dateString[0] + 1]));
(gdb) n
446       if (mDecoder) {
(gdb) n
447         PRInt32 srcLength = (PRInt32) PL_strlen(localBuffer);
(gdb) n
448         PRInt32 unicharLength = sizeof(Str255)*2;
(gdb) n
451         res = mDecoder->Convert(localBuffer, &srcLength, unichars,
&unicharLength);
(gdb) n
452         if (NS_SUCCEEDED(res)) {
(gdb) n
457       return res;
(gdb) p res
$1 = 2147549183
(gdb) n
458     }
(gdb) n
nsDateTimeFormatMac::FormatTime(nsILocale*, int, int, long, nsString&)
(this=0x10d851d0, locale=0x0, dateFormatSelector=1, timeFormatSelector=1,
timetTime=1086801448, stringOut=@0x102c90bc) at nsDateTimeFormatMac.cpp:331
331     }
(gdb) n
nsScriptableDateFormat::FormatDateTime(unsigned short const*, int, int, int,
int, int, int, int, int, unsigned short**) (this=0x102c90b0, aLocale=0x1117a80,
dateFormatSelector=1, timeFormatSelector=1, year=2004, month=6, day=10, hour=2,
minute=17, second=28, dateTimeString=0x10d851c0) at nsScriptableDateFormat.cpp:153
153       if (NS_SUCCEEDED(rv))
(gdb) c
Continuing.
JavaScript error: , line 0: uncaught exception: [Exception... "Component
returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED)
[nsIScriptableDateFormat.FormatDateTime]"  nsresult: "0x8000ffff
(NS_ERROR_UNEXPECTED)"  location: "JS frame ::
chrome://navigator/content/pageInfo.js :: formatDate :: line 1013"  data: no]

Comment 21

[HARUNAGA Hirotoshi]

additional test in Japanese locale:

Comment 22

[HARUNAGA Hirotoshi]

Attachment: debug log of additional test in Japanese locale

Oops, I can't send the log as commment.  Making an attachment instead.

Comment 23

[HARUNAGA Hirotoshi]

Firefox 0.9 has been released.
Workaround by hard coding for Japanese locale users:

1. copy browser.jar to somewhere and extract it.
2. modify FormatDateTime("" to FormatDateTime("ja-JP"
   in browser/pageInfo.js and browser/cookieviewer/CookieViewer.js
3. archive back content/ as browser.jar.
4. replace the original browser.jar with this.

Comment 24

[HARUNAGA Hirotoshi]

requesting blocking 1.7.1

Comment 25

[Constantine A. Murenin]

The minimum testcase must be:

z = Components.classes["@mozilla.org/intl/scriptabledateformat;1"].createInstance();

for (a in Components.interfaces) z instanceof Components.interfaces[a];

z.FormatDateTime("ja-JP", z.dateFormatLong, z.timeFormatSeconds, 2004, 1, 1, 0,
0, 0);
z.FormatDateTime("", z.dateFormatLong, z.timeFormatSeconds, 2004, 1, 1, 0, 0, 0);

Probably, breaking the code both times at nsDateTimeFormatMac::FormatTMTime and
at nsIUnicodeDecoder::Convert, and comparing the stacks could reveal the source
of the problem. 

Comment 26

[Mano (::mano, needinfo? for any questions; not reading general bugmail)]

Can anybody check the status of this bug with the patch for bug 252475

Comment 27

[HARUNAGA Hirotoshi]

(In reply to comment #25)
> The minimum testcase must be:

Yes.

0001: z =
Components.classes["@mozilla.org/intl/scriptabledateformat;1"].createInstance();
[xpconnect wrapped nsISupports]
0002: for (a in Components.interfaces) z instanceof Components.interfaces[a];
false
0003: z.FormatDateTime("ja-JP", z.dateFormatLong, z.timeFormatSeconds, 2004, 1,
1, 0, 0, 0);
2004年 1月 1日 (木) 0:00:00 AM
0004: z.FormatDateTime("", z.dateFormatLong, z.timeFormatSeconds, 2004, 1, 1, 0,
0, 0);
[Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED)
[nsIScriptableDateFormat.FormatDateTime]" nsresult: "0x8000ffff
(NS_ERROR_UNEXPECTED)" location: "JS frame ::
chrome://venkman/content/venkman-eval.js :: con_eval :: line 50" data: no]

> Probably, breaking the code both times at nsDateTimeFormatMac::FormatTMTime and
> at nsIUnicodeDecoder::Convert, and comparing the stacks could reveal the source
> of the problem. 

(gdb) b nsIUnicodeDecoder::Convert
the class nsIUnicodeDecoder does not have any method named Convert

What should I do?

Re: comment 26:

JavaScript code in bug 252475 comment 0 returns "ja-JP" in Japanese
Language/Format.
So I think that bug 252475 is probably different from this...

Comment 28

[jhp (no longer active)]

Attachment: patch

The 0x8000ffff error comes from UnicodeDecoder->Convert failing to convert the
date/time string to the user's language.  The reason is that the unicode
decoder is created using mSystemCharset, which is the filesystem's encoding; 
this is always "UTF-8" on MacOSX.  We should always be using mCharset to create
the unicode decoder.

This is the quick patch.  For 10.3, we need to switch to the new APIs. I'll try
to create a patch for that also.

Comment 29

[Hiro]

Attachment: PageInfo of a SSL page

https://register.www.infoseek.co.jp/Login?appRedirect=http://email.www.infoseek.co.jp/


The certificate cannot be seen in PageInfo of a SSL page.
And tt does not react, even if it pushes the View button.
(When applying the patch of comment#23, it was able to see normally.)

This may be a very serious problem.
It desires to be repaired in the next release.
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; ja-JP; rv:1.7) Gecko/20040802
Firefox/0.9.1+

Comment 30

[Constantine A. Murenin]

Comment on attachment 154694 [details] [diff] [review]
patch

This patch looks fine to me. 

harunaga, could you confir that it solves the problem? 

jhpedemonte, could you request reviews? Let's fix this for now, and switch to
the new APIs later (in a different bug).

Comment 31

[HARUNAGA Hirotoshi]

(In reply to comment #30)
> harunaga, could you confir that it solves the problem? 

The patch works fine for me in the Japanese Language environment
on Mac OS 10.3.5.

Comment 32

[Hiro]

With this patch, it is working also by Firefox of aviary branch.
The problem of comment#29 is also Fixed!!

Mac OS X 10.3.5

Comment 33

[Jungshik Shin]

Comment on attachment 154694 [details] [diff] [review]
patch

r=jshin

After getting sr, let's get this in for branches. Then, on trunk, we should
make a new patch that uses new APIs.

Comment 34

[Constantine A. Murenin]

Comment on attachment 154694 [details] [diff] [review]
patch

Requesting approvals for aviari and 1.7.x. 

jshin: could you commit the patch to the trunk in the meantime?

Comment 35

[Asa Dotzler [:asa]]

Comment on attachment 154694 [details] [diff] [review]
patch

a=asa for branch checkins.

Comment 36

[jhp (no longer active)]

Patch checked in everywhere.  Is there already a bug open for moving to the new
APIs on the trunk?

Comment 37

[Mano (::mano, needinfo? for any questions; not reading general bugmail)]

removing blcoking? flags.

Comment 38

[jhp (no longer active)]

Bug for new APIs is bug 105045.  Marking fixed.