374039 - Some dates not displayed when LC_TIME=hi_IN.UTF-8

Status: RESOLVED INCOMPLETE

(Core :: Internationalization, defect)

Description

[Aniruddha Shankar]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20060601 Firefox/2.0.0.2 (Ubuntu-edgy)
Build Identifier: version 1.5.0.10 (20070306)

I'm using mozilla thunderbird under kubuntu dapper drake 6.10. . My locale settings are as follows:

LANG=hi_IN
LC_CTYPE=hi_IN.UTF-8
LC_NUMERIC=hi_IN.UTF-8
LC_TIME=hi_IN.UTF-8
LC_COLLATE=hi_IN.UTF-8
LC_MONETARY=hi_IN.UTF-8
LC_MESSAGES=hi_IN.UTF-8
LC_PAPER=hi_IN.UTF-8
LC_NAME=hi_IN.UTF-8
LC_ADDRESS=hi_IN.UTF-8
LC_TELEPHONE=hi_IN.UTF-8
LC_MEASUREMENT=hi_IN.UTF-8
LC_IDENTIFICATION=hi_IN.UTF-8
LC_ALL=

locales available on my machine are :
bn_BD
bn_IN
C
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_IN.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8
hi_IN
hi_IN.utf8
POSIX

Unless I change LC_TIME to en_US.UTF-8 some dates are not displayed by thunderbird, as the screenshot shows. Some dates are displayed properly, however.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Aniruddha Shankar]

The screenshot is at http://191a.net/nodate.png . And  I mistakenly stated that I'm using dapper drake when I actually meant edgy eft.

Comment 2

[Aniruddha Shankar]

I believe the results should be reproducible on other machines by installing a hindi font (there should be an indic fonts package in your repository somewhere), exporting the variable and starting up your copy of thunderbird.

Comment 3

[Magnus Melin [:mkmelin]]

Could an extension cause the problem? Have you tried tb 2.0 (not final yet, but soon)? <http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-mozilla1.8/>

Comment 4

[Aniruddha Shankar]

Magnus, Hi. 
Could you tell me how to run this particular file ? I downloaded the tarball and extracted it, but when I try and run thunderbird, I get the following error:

karim@bhim:~/thunderbird-2.0pre.en-US.linux-i686/thunderbird$ pwd
/home/karim/thunderbird-2.0pre.en-US.linux-i686/thunderbird
karim@bhim:~/thunderbird-2.0pre.en-US.linux-i686/thunderbird$ ./thunderbird
Segmentation fault
karim@bhim:~/thunderbird-2.0pre.en-US.linux-i686/thunderbird$ 

Is this merely a problem with this particular nightly?        

Comment 5

[Aniruddha Shankar]

Since I couldn't get the nightly to run, I went home and built it on my Gentoo box. I was running tbird version 2 beta 2 build 20070316. I also disabled all extensions and made sure that LC_ALL and LC_CTIME were set to hi_IN.UTF-8 . The problem persisted and I have uploaded a flash video that illustrates the issue to http://191a.net/baddates.html . 

Comment 6

[Magnus Melin [:mkmelin]]

Don't know why the nightly didn't run for you...

Do you get any errors in the Error Console (JavaScript Console in 1.5)? 

Is there something particular with the dates of the mails that don't display? Can you give the Date: header of a few of the ones that doesn't work?

Comment 7

[Aniruddha Shankar]

On the unstable build (running on Gentoo), i get this message (I think it's not relevant but pasting anyway)


Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIIOService2.manageOfflineStatus]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///usr/lib/mozilla-thunderbird/components/offlineStartup.js :: anonymous :: line 95"  data: no]
Source File: file:///usr/lib/mozilla-thunderbird/components/offlineStartup.js
Line: 95

Comment 8

[Aniruddha Shankar]

Attachment: Email that does not display the date field.

This is an email (.eml file) that does not display the date field.

Comment 9

[Aniruddha Shankar]

Attachment: Email that DOES display the date

Thunderbird DOES show the date of this email (.eml file)

Comment 10

[Simon Montagu :smontagu]

Is it possible that dates with times PM are being displayed and times AM are not?

Comment 11

[Simon Montagu :smontagu]

More exactly, I think some dates need more space than the 80 bytes that we allow in nsDateTimeFormatUnix::FormatTMTime().

Comment 12

[Simon Montagu :smontagu]

Attachment: Patch

I think that this was probably the original intention, considering that NSDATETIME_FORMAT_BUFFER_LEN*2 is the size of strOut.

Comment 13

[Jungshik Shin]

Comment on attachment 258982 [details] [diff] [review]
Patch

r=jshin

It appears that 'x2' factor was used originally for the inflation of a single byte in ASCII/ISO-8859-x to 2 bytes. Probably because UTF-8 locales were not widely used.

Comment 14

[Simon Montagu :smontagu]

Moving back to the original component to request blocking-thunderbird2. This makes the main mail display seriously hard to use and affects all the Indian languages. Risk is minimal: the patch doesn't even change the size of the buffer used to receive the formatted date string.

Comment 15

[Jungshik Shin]

Comment on attachment 258982 [details] [diff] [review]
Patch

oops. forgot '+' it

Comment 16

[jag (Peter Annema)]

Attachment: The not-so-quick fix

This compiles (on my Mac) but is untested. Maybe something we should consider for trunk?

Comment 17

[jag (Peter Annema)]

Comment on attachment 258982 [details] [diff] [review]
Patch

sr=jag for a minimalistic branch fix.

Comment 18

[Scott MacGregor]

Comment on attachment 258982 [details] [diff] [review]
Patch

We could be tagging for Thunderbird 2 as early as tomorrow morning, I'll ping the 1.8.1.4 triage team to see if they'll approve this for the branch. It looks very low risk and fixes a problem for intl mail users on linux.

Comment 19

[Daniel Veditz [:dveditz]]

Comment on attachment 258982 [details] [diff] [review]
Patch

Approved for 1.8/1.8.0 branches, a=dveditz

Comment 20

[Scott MacGregor]

Simon, I'll land the branch patch for 1.8 later tonight if you don't beat me to it to ensure that's its checked in before we tag tomorrow morning. Thanks again for working on this.

Comment 21

[Scott MacGregor]

Simon I landed this on the 1.8 branch tonight so it would make it into tb2. I'll let you land it on the trunk and the 1.8.0 branch. Thanks again for the patch.

Comment 22

[Simon Montagu :smontagu]

Comment on attachment 259086 [details] [diff] [review]
The not-so-quick fix

This has to be the most generous response to a sr request ever :) I wanted to  do something like this for trunk after getting the quick fix in for TB2. Thanks for beating me to it.

Comment 23

[Simon Montagu :smontagu]

Landed on 1.8.0 branch. Thanks for the 1.8 checkin, Scott.

Comment 24

[Simon Montagu :smontagu]

It would be good to have a test for this, but do we have a way to force a locale when running tests?

Comment 25

[Christian :Biesinger (don't email me, ping me on IRC)]

Comment on attachment 259086 [details] [diff] [review]
The not-so-quick fix

The function here has a locale argument, doesn't it? But the system may not have the hi_IN locale installed, so testing this may not be possible...

nsDateTimeFormatUnix.cpp
+    stringOut.AssignLiteral("");

I'd make that stringOut.Truncate();

+      if (NS_SUCCEEDED(rv))
+        stringOut.Assign(unichars, unicharLength);

I think you could use stringOut.SetLength/BeginWriting to write directly to that string.

Comment 26

[Aniruddha Shankar]

(In reply to comment #23)
> Landed on 1.8.0 branch. Thanks for the 1.8 checkin, Scott.
> 

I just upgraded to 2.0.0.0_rc1 and the problem still remains. Is the 1.8.0 branch  not going to be used for thunderbird 2.0.0.0 ?

Comment 27

[Scott MacGregor]

thunderbird 2 was shipped from 1.8.1.3. So this fix will show up in the first security update after the release. Sorry for the confusion.

Comment 28

[Simon Montagu :smontagu]

jag, can you go ahead and check this in?

Comment 29

[Makoto Kato [:m_kato]]

nsDateTimeFormatUnix is gone, so if there is new issue that uses ICU, we should file a new bug.