Setting date locale no longer works in Thunderbird 60 on linux (LC_TIME=en_DK.utf8 behaves differently than it used to)

UNCONFIRMED
Unassigned

Status

defect
UNCONFIRMED
a year ago
14 hours ago

People

(Reporter: mail, Unassigned)

Tracking

(Blocks 1 bug)

58 Branch
Unspecified
Linux

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [workaround: use LC_TIME=sv_SE.UTF-8 instead])

(Reporter)

Description

a year ago
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20171221100108

Steps to reproduce:

In Thunderbird 52 on Ubuntu 16.04 I can change the date format to `YYYY-MM-DD` by running `env LC_TIME=en_DK.utf8 thunderbird`.

In Thunderbird 58b2 this no longer works.


Actual results:

Thunderbird shows the Date column as DD/MM/YYYY.


Expected results:

Thunderbird shows the Date column as YYYY-MM-YY, as TB 52 did.

Comment 2

a year ago
We had a felt million bugs regarding the date/time display format changes that have occurred on Mozilla core, Firefox and Thunderbird during 2017, actually, all this started to the day one year ago on Christmas Eve 2016.

In Thunderbird you have a preference you can set under Tools > Options, Advanced, General (might we elsewhere on the Window on Linux).

If this doesn't work please, browse some of the other bugs, starting with bug 1344594 (and duplicates) then moving to bug 1389972 and then bug 1389369. You will find suggestions that will fix your issue.
Status: UNCONFIRMED → RESOLVED
Last Resolved: a year ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1344594
(Reporter)

Comment 3

a year ago
(In reply to Jorg K (GMT+1, mostly AFK 22-27 Dec., bustage-fix only) from comment #2)
> In Thunderbird you have a preference you can set under Tools > Options,
> Advanced, General (might we elsewhere on the Window on Linux).

I do have an option there to choose between "Application locale: English (United States)" and "Regional settings locale: English (Denmark)" on Linux, but neither has any effect on the dates being displayed.

> If this doesn't work please, browse some of the other bugs, starting with
> bug 1344594 (and duplicates) then moving to bug 1389972 and then bug
> 1389369. You will find suggestions that will fix your issue.

@jorgk: All of the issues linked are closed.

Bug 1366134 is open but there and in all other places it says that switching from _US to _DK should work as long as both locales start with en_ and Thunderbird is an en_ build.

Also, on that issue, Zibi recommended for a user who reported that this doesn't work as advertised that they should open a new bug with their specific issue.

So I'm reopening this for now, unless we can identify an appropriate open issue to set this as a duplicate for.

Zibi, could you confirm whether this is known to be currently broken in TB 58 beta and if so, which issue should this be a duplicate for?
Status: RESOLVED → UNCONFIRMED
Flags: needinfo?(gandalf)
Resolution: DUPLICATE → ---

Comment 4

a year ago
Regional settings locale: English (Denmark) should have worked after a restart of TB. Bug 1389369 comment #7 doesn't help?
(Reporter)

Comment 5

a year ago
For me it doesn't work, no matter how often I restart TB.

How should https://bugzilla.mozilla.org/show_bug.cgi?id=1389369#c7 help, isn't that only about 12h vs 24h? My issue is about the entire date format, e.g. changing to YYYY-MM-DD format.
I tested the LC_TIME on Linux/Gnome and it seems to me that if I switch the `Formats` setting in `Region & Language` in Gnome Settings, Thunderbird (Nightly) follows along nicely.

My suspicious is that multiprocess somehow makes us read the session LC_TIME rather than command line LC_TIME and that's why your command doesn't work.

Can you try to change your LC_TIME for the whole session?
Flags: needinfo?(gandalf)
(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #6)
> ...
> My suspicious is that multiprocess somehow makes us read the session LC_TIME
> rather than command line LC_TIME and that's why your command doesn't work.
> 
> Can you try to change your LC_TIME for the whole session?

Niklas?
Flags: needinfo?(mail)
Blocks: tb60found
(Reporter)

Comment 8

11 months ago
I've tried again with Thunderbird Daily (thunderbird-62.0a1.en-US.linux-x86_64.tar.bz), and the Beta 60.

I still cannot get Thunderbird to show the '2018-05-23 15:57:07' format.

I believe I've tried any combination of

* Changing the system locale via `unity-control-center` in the "Regional Formats" tab of gnome's "Language Support" settings and clicking "Apply system-wide" (`locale` shows `LC_TIME=en_DK.UTF-8` right after login)
* Setting LC_TIME=en_DK.UTF-8 manually in my .xinitrc so that my Window manager (i3) reads it on startup
* Thunderbird's "Date and Time Formatting" setting

Thunderbird keeps showing the '23/05/2018, 10.32' format.

Meanwhile `date +'%x %X'` shows the format I desire as expected.
Flags: needinfo?(mail)
(Reporter)

Comment 9

11 months ago
Just for clarity, I can also see LC_TIME=en_DK.UTF-8 in `tr '\0' '\n' < /proc/$(pidof thunderbird)/environ | grep LC`.

So at least in theory, Thunderbird should be able to see it.

Comment 10

11 months ago
Zibi, can you help? Even Magnus was happy in the end, see bug 1389972 comment #5. Nothing ever got implemented in bug 1337069, right?

Niklas, which locales are shown in TB's Advanced/General settings under "Date and Time" Formatting?
Flags: needinfo?(gandalf)
Is there an equivalent of firefox about:support? I'd like Niklas to dump the output of the Intl/L10n section of it.

Also, Niklas - does it work if you change other LC_*? We may not recognize the difference between LC_ALL, LC_DATE and LC_TIME yet.

Comment 12

11 months ago
(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #11)
> Is there an equivalent of firefox about:support? I'd like Niklas to dump the
> output of the Intl/L10n section of it.
Yes, "Help > Troubleshooting Information". It's 99% of what FF offers, at least from TB 60 up and trunk.

Personally I see:
Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["de"]
App Locales 	["en-US"]
Regional Preferences 	["en-GB"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-GB"]

I have a German language pack installed (but not using it), hence the "de". Otherwise I'm using "en-GB". Anyway, I don't have a problem, the reporter has.
Flags: needinfo?(gandalf)

Updated

8 months ago
Component: Untriaged → General
(Reporter)

Comment 13

8 months ago
Thanks. I get this in "Help > Troubleshooting Information", when using LC_TIME=en_DK.utf-8:

 Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US"]
Regional Preferences 	["en-DK"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-GB"]
Regional Preferences 	["en-DK"]

And when also setting LC_ALL=en_DK.utf-8, I get no differences in the UI and these settings:

 Application Settings
Requested Locales   ["en-US"]
Available Locales   ["en-US"]
App Locales   ["en-US"]
Regional Preferences  ["en-DK"]
Default Locale  "en-US"
Operating System
System Locales  ["en-DK"]
Regional Preferences  ["en-DK"]

@Zibi: Is LC_DATE really a thing? In any case, if I set it, nothing changes.

> which locales are shown in TB's Advanced/General settings under "Date and Time" Formatting?

"English (United States)" and "English (Denmark)" are the two options with the above last setting.

I don't even understand what format the '23/05/2018, 10.32' is. As far as I know, neither en-US nor en-DK format times with a '.'.
Flags: needinfo?(gandalf)
Hi Niklas,

The reason you see "23/05/2018" is because Unicode specifies that format as the default for dates in `en-DK`.

You can try it in any modern browser:

```
(new Date()).toLocaleString("en-DK");
```

It'll format your date using `en-DK` locale format specified in Unicode CLDR database.
Flags: needinfo?(gandalf)

Comment 15

8 months ago
Hello Zibi,

You're right. toLocaleString("en-DK") uses a different format than LC_TIME="en_GB.utf8" used to do for Thunderbird.

In Gnome's Region & Language settings, I have Language set to "English (United Kingdom)" and Formats to "Denmark" and Gnome gives me:

- Dates: 2018-08-14
- Times: 11:14:58
- Dates & Times: 2018-08-14T11:14:58 CEST

And I cannot get that anymore in Thunderbird. So, I don't know what Thunderbird is doing, but neither of these two options do what I want:

- Application locale: und
- Regional settings locale: English (Denmark)

You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

If you know some other way to use ISO formatting in Thunderbird, please let us know! ;-)
I understand the confusion.

It seems to me that Gnome/POSIX is presenting different date format for en-DK than ICU/CLDR is. 
CLDR stores the default en-DK formats here: https://github.com/unicode-cldr/cldr-dates-modern/blob/master/main/en-DK/ca-generic.json#L321 - as you can see they use `dd/MM/y GGGGG` which translates to `23/05/2018` that you see.

My guess is that POSIX just doesn't know what to do with en-DK and it throws ISO format instead.

> You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

Well, it does. It just does it differently than you would expect. Unfortunately, I don't see an easy solution here, since we do not handle a local overlay on CLDR data, we do not have an easy way to add a custom overlay for en-DK.

What you could do is file an CLDR ticket - https://unicode.org/cldr/trac/ - and request en-DK locale date time format to be changed to what you believe to be the good default for en-DK.

Since no CLDR locale, as far as I know, uses ISO format, I can't even recommend setting any locale manually to Regional Preferences Locales in Gecko, because none will result in the ISO format. CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.

What I might recommend if you want your localization to be in English, you can pick one of other en-* variants that suits you better. You can see the list here: https://github.com/unicode-cldr/cldr-dates-modern/tree/master/main

Comment 17

8 months ago
Hello Zibi, Thank you for your response.

>> You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

> Well, it does. It just does it differently than you would expect.

What I meant is that there is no way to tell Thunderbird to use the system settings anymore; I consider that a major issue. The option "Regional settings locale" does not do on my Gnome system what it suggests it does, namely using my (system) regional setting.

> ... request en-DK locale date time format to be changed to what you believe to be the good default for en-DK.

Hmmm. You are aware that English is not spoken in Denmark (as a first language) and that en-DK is only used to get ISO format while using English?

> CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.

This is not a discussion about taste. If it were, there would be no need for localisation at all, just one format for everyone! ;-)

As far as I understand, there is a fair amount of people that were using en-DK to get the ISO format, and Thunderbird does not offer that in the latest version. I have no information about other issues that people may have found with this latest change, but if ISO format is the one that people miss, I suggest to add a third option in Thunderbird for ISO formats. Especially since there is no CLDR option to get ISO format, as you were aware of. And even if there were one, it is probably not one that works the same in POSIX.
> What I meant is that there is no way to tell Thunderbird to use the system settings anymore; I consider that a major issue. 

That's a major change introduced post Gecko 52 - we do not use OS intl APIs anymore. There are many important reasons for a multiplatform engine to not do that.

> The option "Regional settings locale" does not do on my Gnome system what it suggests it does, namely using my (system) regional setting.

That's not true. It does use your regional settings *locale*. It just resolves different values for that regional settings locale. It uses values from Unicode CLDR which is a really good database used by Windows, MacOS, Android, Chrome and others. That means that the values you will get are the same (or very similar) to ones you will likely encounter in most other modern software.

> Hmmm. You are aware that English is not spoken in Denmark (as a first language) and that en-DK is only used to get ISO format while using English?

That's not true. `en-DK` is an established partial locale which overlays root locale `en` in Unicode with custom internationalization settings. If you believe that `en-DK`'s date format should be ISO, I recommend you report it to CLDR under the link I provided.

That decision will be made with involvement from intl experts from all organizations participating in the Unicode project and naturally is going to be opinionated in scenarios where multiple defaults are possible.

> As far as I understand, there is a fair amount of people that were using en-DK to get the ISO format, and Thunderbird does not offer that in the latest version.

Yes, that's correct. Thunderbird is using Gecko platform which moved away from using OS APIs for retrieving intl information. That has many major positive consequences, but also may in edge cases like this, lead to unwanted changes as well.

> I suggest to add a third option in Thunderbird for ISO formats. 

That's perfectly possible and I would likely accept a patch to mozIntl/OSPreferences to allow for that, but I will not be able to prioritize that anytime soon to work on it myself.

Comment 19

8 months ago
> That's not true. It does use your regional settings *locale*. It just resolves different values for that regional settings locale. It uses values from Unicode CLDR which is a really good database used by Windows, MacOS, Android, Chrome and others. That means that the values you will get are the same (or very similar) to ones you will likely encounter in most other modern software.
I would agree if this behaviour was consistent for all platforms supported by Thunderbird. However this is not true: on Windows, you can change the date and time format (using Control Panel>Clock, Language, and Region>Change date, time, or numbers formats) e.g. to use ISO 8601, and this setting will be honoured by Thunderbird 60! I am not completely sure, but I guess the code responsible for that is https://dxr.mozilla.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#132. On Linux on the other hand, Thunderbird seems to only read the locale *name* from LC_TIME and then uses the CLDR database to construct the date format, ignoring the format specified in the system locale (https://dxr.mozilla.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#148). This seems to be an inconsistency across platforms that is likely to cause confusion.

> Since no CLDR locale, as far as I know, uses ISO format, I can't even recommend setting any locale manually to Regional Preferences Locales in Gecko, because none will result in the ISO format. CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.
Well, there is en-SE (https://github.com/unicode-cldr/cldr-dates-modern/blob/master/main/en-SE/ca-generic.json#L321), which is probably the closest you can get to ISO 8601 with Unicode CLDR: in contrast to ISO 8601, the separator between date and time is a comma, but other than that it follows the specification. Unfortunately, this is not commonly included in Linux distributions as it is not part of the standard GLIBC set of locales, but you can create a symlink from /usr/share/i18n/locales/en_DK to /usr/share/i18n/locales/en_SE, run locale-gen and set LC_TIME=en_SE.UTF-8 to achieve the desired effect. (Just setting LC_TIME to use this locale without actually generating it does not seem to work as Thunderbird only issues the GTK warning "Locale not supported by C library" and ignores the missing locale.)
> However this is not true: on Windows, you can change the date and time format (using Control Panel>Clock, Language, and Region>Change date, time, or numbers formats) e.g. to use ISO 8601, and this setting will be honoured by Thunderbird 60! I am not completely sure, but I guess the code responsible for that is https://dxr.mozilla.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#132. On Linux on the other hand, Thunderbird seems to only read the locale *name* from LC_TIME and then uses the CLDR database to construct the date format, ignoring the format specified in the system locale (https://dxr.mozilla.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#148). This seems to be an inconsistency across platforms that is likely to cause confusion.

That's a great read of the situation!

Yes, we do more on Windows, than we do on Linux. The reason being that Windows provides UI and API for us to read customized date/time patterns for styles short/medium/long.

Which means that we can fetch the *pattern* from the OS, and *localize* it using our own CLDR resources. This means, that if on Windows you'll tell us you want a date "{day} {month} {year}", and in Gecko you'll specify you want Polish language we'll present you a date "25 sierpnia 2018", which will fit properly into your UI localization.

On the other hand on Linux, we do not have a way to retrieve a pattern, that we could then localize.

This method could not be used for our public Intl API (the ECMA402 one exposed to the Web), and since our platform is converging around WebAPIs, we switched our toolkit's Intl API to use the same logic as ECMA402 API, which requires us to remain consistent across platforms.

We do extend it tho, to some degree, by, among others, allowing for the OS to provide us a pattern that the user customized to. If Linux will one day (or even Gnome itself :)), start exposing similar Regional Preferences UI to allow users to customize their patterns, we'll do the same thing as we do on Windows/MacOS.

Comment 21

8 months ago
> Yes, we do more on Windows, than we do on Linux. The reason being that Windows provides UI and API for us to read customized date/time patterns for styles short/medium/long.
> 
> Which means that we can fetch the *pattern* from the OS, and *localize* it using our own CLDR resources. This means, that if on Windows you'll tell us you want a date "{day} {month} {year}", and in Gecko you'll specify you want Polish language we'll present you a date "25 sierpnia 2018", which will fit properly into your UI localization.
> 
> On the other hand on Linux, we do not have a way to retrieve a pattern, that we could then localize.
You can retrieve the date and time patterns for the current locale using the POSIX function nl_langinfo() (http://pubs.opengroup.org/onlinepubs/9699919799/functions/nl_langinfo.html): a call like "nl_langinfo(D_FMT)" (or "locale date_fmt" in a shell) returns a format string for strftime like "%a %b %e %H:%M:%S %Z %Y". The supported locale elements are described in http://man7.org/linux/man-pages/man3/nl_langinfo.3.html and http://man7.org/linux/man-pages/man5/locale.5.html.

Unfortunately, I can see that this might not be a nice fit for the CLDR format you require: instead of providing multiple versions of the date and time patterns (short/medium/long/full), there is only one version for date (D_FMT), time (T_FMT) and date and time combined (D_T_FMT). I think the only (ugly) solution here would be to map the single pattern provided by the operating system to all four CLDR versions: there simply seems to be no standard way to specify locale information on Linux other than what is defined by the POSIX specification, which offers much less flexibility than the more recent CLDR specifications.
> You can retrieve the date and time patterns for the current locale using the POSIX function nl_langinfo() 

This returns the default value, not user-customized. The only reason we allow fetching into the OS is to respect user manual customizations in OSes that allow users to customize.

Retrieving a default pattern for a given locale that is different from a default pattern for the same locale that we have in CLDR would introduce internal inconsistency for much larger population (currently, there still is an inconsistency risk between UI that uses Intl API and mozIntl API which extends to fetch into the OS, but it affects only users who manually customized their format).

The further limitations of the POSIX model you listed is just another vector for which I'd prefer not to do that. It's much easier to take the pattern from Windows/MacOS which at worst use different CLDR version, than from POSIX which uses completely different database.

Comment 23

8 months ago
> This returns the default value, not user-customized. The only reason we allow fetching into the OS is to respect user manual customizations in OSes that allow users to customize.
For Linux, customisation of the locale means either modifying the file of the current locale or creating a new one (e.g. en_XX for Arch Linux: https://xyne.archlinux.ca/projects/locale-en_xx/). I completely understand the point of not supporting the first of these approaches to avoid inconsistencies. Not supporting the second approach (i.e. using the OS date format if the locale name cannot be matched to a valid CLDR locale) is a bit inconvenient with regard to customisations.

BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting? I would have thought "true" means "use the date format of the default Thunderbird locale", while "false" means "use the date format of the system locale [as specified by CLDR]". However Thunderbird seems to always use the date format from the locale in LC_TIME, regardless of that setting. Am I misunderstanding its function, or does it not apply to dates?

> The further limitations of the POSIX model you listed is just another vector for which I'd prefer not to do that. It's much easier to take the pattern from Windows/MacOS which at worst use different CLDR version, than from POSIX which uses completely different database.
FWIW, the people from GLIBC, current maintainers of the POSIX locales, are working on consolidating the POSIX and CLDR locale information: https://sourceware.org/ml/libc-locales/2016-q2/msg00246.html. There might be still some way to go though ;)
> Not supporting the second approach (i.e. using the OS date format if the locale name cannot be matched to a valid CLDR locale) is a bit inconvenient with regard to customisations.

That would be quite hard to fit. If you create your own locale `en-XY`, and we're trying to negotiate the best locale for the app that has plural rule categories, localization resources in this locale, date/time and relative time format patterns and so on, then your custom locale `en-XY` would only work for one aspect of the whole process.

It also likely doesn't help that POSIX locales and BCP47 locales which we use extend differently :/

> BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting?

https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences

tl;dr - we will match the *locale* that your OS returns, but we'll take the values from CLDR since we know we have all the tables for each locale that we need.

> FWIW, the people from GLIBC, current maintainers of the POSIX locales, are working on consolidating the POSIX and CLDR locale information: https://sourceware.org/ml/libc-locales/2016-q2/msg00246.html.

That's an awesome news!

I expect *a lot* of problems glibc is facing to go away once they converge around CLDR. Mainly because CLDR is well maintained by localization and linguistic experts from all major IT companies in the World. It is a Wikipedia of intl data. glibc, well, glibc is a good effort from the 80ties.
With the recent move of Microsoft to CLDR, I hope that the move of glibc will complete the unification.

Comment 25

8 months ago
> That would be quite hard to fit. If you create your own locale `en-XY`, and we're trying to negotiate the best locale for the app that has plural rule categories, localization resources in this locale, date/time and relative  time format patterns and so on, then your custom locale `en-XY` would only work for one aspect of the whole process.
I understand. Well, luckily in the case of ISO dates, using the en-SE locale for LC_TIME works great, even though it requires some manual work to create the POSIX locale first.

> > BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting?
> 
> https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences
> 
> tl;dr - we will match the *locale* that your OS returns, but we'll take the
> values from CLDR since we know we have all the tables for each locale that
> we need.
Ah, so the setting only applies if the language of Thunderbird and the system locale (from LC_TIME) differ. That explains why I did not see any difference, as both were the same in my case (en-US for Thunderbird and en-DK in LC_TIME). Thank you for the link, that was very helpful!

Comment 26

8 months ago
Thanks for the en_SE workaround! However, I still have a problem: after creating the en_SE locale and setting LC_TIME, I get the ISO date but I still have my time in 12h, f.ex 2018-08-23, 8:19 PM. Is there any way to change this?
You're in a particular luck, I think! One thing we do handle, because Gnome handles it is hour-cycle!

If you have Gnome Preferences, just go to Date&Time and set your hour cycle to the preferred one. If not, you can use any dconf-editor to get to org.gnome.desktop.interface.clock-format and set it to 12|24.

Comment 28

8 months ago
Thanks a lot! That worked a treat

Updated

8 months ago
OS: Unspecified → Linux
Summary: Setting date locale no longer works in Thunderbird 58 Beta → Setting date locale no longer works in Thunderbird 58 Beta on linux
(Reporter)

Comment 29

8 months ago
I tried the symlink workaround but couldn't get it working.

I ran (on Ubuntu 16.04):

    sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/en_SE
    sudo locale-gen

but it didn't print out a `en_SE.UTF-8...` and correspondingly I got `warning: setlocale: LC_TIME: cannot change locale (en_SE.utf-8): No such file or directory` at startup.

Did I miss a step?

Comment 30

8 months ago
You also have to enable the new locale in /etc/locale.gen by adding "en_SE.UTF-8 UTF-8" as a new line (which I missed in my original description), so the complete command sequence would be something like:

sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/en_SE
echo 'en_SE.UTF-8 UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen
(Reporter)

Comment 31

8 months ago
Thank you very much, at last some relief :)

> I suggest to add a third option in Thunderbird for ISO formats. 

That sounds like a really good idea.

Comment 32

7 months ago
I ran into this bug with Thunderbird 60.

Using en_DK is a very long-standing recommendation for getting ISO 8601 dates:
http://kb.mozillazine.org/Date_display_format

It is disappointing to find this broken now. Using en_SE as described above is at least an improvement, but I would definitely like a definitive option in the preferences to use ISO date format.

Thanks,
Corey

Comment 33

7 months ago
Any update on this?

With the TB 60 update and 50% of my add-ons not working I also lost the Super Date Format one which I was using to work around this [https://addons.thunderbird.net/en-US/thunderbird/addon/super-date-format/]

My information is as follows (Thunderbird 60.0 packaged from Manjaro):

Internationalization & Localization
Application Settings
Requested Locales 	["en-GB","en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["en-US","und"]
Default Locale 	"und"
Operating System
System Locales 	["en-GB"]
Regional Preferences 	["it-IT"]

Yet my system locale environment variables are:


LC_TIME=it_IT.UTF-8
LANG=en_GB.utf8

Thanks
Summary: Setting date locale no longer works in Thunderbird 58 Beta on linux → Setting date locale no longer works in Thunderbird 60 on linux (LC_TIME=en_DK.utf8 behaves differently than it used to)
Whiteboard: [workaround: use LC_TIME=en_SE.utf-8 instead]

Comment 34

6 months ago
There is no way to change date/time into ISO format after update to Thunderbird 60.2.1 on Linux Mint OS.

Advanced > General > Date and Time Formating don't affect anything.

Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["en-US"]
Default Locale 	"und"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-US"]


Also add-on "Super Date Formata" is now incompatible, and add-on "ConfigDate" settings don't affect anything.
Please see whiteboard and previous comments in this bug.

Comment 36

6 months ago
LC_TIME is set to 'nl_NL.UTF-8' for me.

Calendar: works after setting 'Date and Time formatting' in Advanced Preferences.
E-mail: no luck. LC_TIME is not respected.

org.gnome.desktop.interface.clock-format is set to '24h'


Internationalization & Localization

Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["nl-NL"]
Default Locale 	"und"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["nl-NL"]

Comment 37

6 months ago
I can confirm this bug on Thunderbird 60.2.1 (64-bit) on a Linux Mint 18.3 machine.

My system clock is 24 hours mode, but Thunderbird now has a confusing date that starts with the year and ends in am or pm - whether I prefer to use the application locale or the OS locale. And I can't find how to change the application locale in Thunderbird.

Comment 38

6 months ago
for those experiencing issues on gnu/linux

test the mozilla release (instead of the one from your distro):

https://ftp.mozilla.org/pub/thunderbird/releases/60.0/linux-x86_64/

(or wget https://ftp.mozilla.org/pub/thunderbird/releases/latest/README.txt)

tar xvf thunderbird-60.0.tar.bz2
cd thunderbird/
./thunderbird-bin

to instead start it from your applications' list:

## to run do   bash this-script.sh
## assuming thunderbird/ is extracted in ~/Downloads/
  for i in 16x16 22x22 24x24 32x32 48x48 64x64 128x128 256x256; do
    mkdir -pv ~/.icons/hicolor/$i/apps/
    cp -v ~/Downloads/thunderbird/chrome/icons/default/default${i/x*}.png \
              ~/.icons/hicolor/$i/apps/thunderbird.png
  done

Comment 39

6 months ago
I am using Thunderbird 60 with Debian Stretch & KDE5, and for me any locale settings for emails does not work anymore - I tried several workarounds without success.

My whole system is set to de_CH, the the Calendar works fine with date formats, but not emails, which is a pain in the neck.

This should be a MAJOR issue as it reduces functionality for anybody non-US immensely. At least there should be some way to "overwrite" the date format in the settings if Thunderbird is not able to correctly ascertain the correct system locale and format. Because the current option (using "Regional settings locale") does absolutely nothing.
Can you paste the `Internationalization & Localization` results from about:support ?

The date/time formatting should follow OS date/time if the language between OS and Thunderbird matches.

> This should be a MAJOR issue as it reduces functionality for anybody non-US immensely.

I'm not sure if you're correct. Majority of the users do use date/time format that matches their app's locale. I'm not trying to shy away from identifying the cause and fixing it, just putting things in perspective.

Comment 41

6 months ago
These are the results:

Application Settings
Requested Locales 	["de-CH","en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["de-CH"]
Default Locale 	"und"
Operating System
System Locales 	["de-CH"]
Regional Preferences 	["de-CH"]
Thank you!

The entry "Application Settings > Regional Preferences 	["de-CH"]" indicates to me that Gecko uses `de-CH` to format date and time in your app.

If that's not the case, we have a bug. Can you provide a screenshot of a place in the UI where the date/time is formatted differently than when you do `new Date(2018, 10, 22, 15, 23).toLocaleString("de-CH")` (which, in my case shows "22.11.2018, 15:23:00")? 

What OS are you on?

Comment 43

6 months ago
I am on Debian Stretch with KDE 5. Didn't find a way to include a screenshot here, but you can look at it here:

https://dl84.panaxis.ch/screenshot_thunderbird.png

Basically, it is everywhere, except for the Calendar - there the date is formatted correctly.

Comment 44

6 months ago
> > This should be a MAJOR issue as it reduces functionality for anybody non-US immensely.
> 
> I'm not sure if you're correct. Majority of the users do use date/time
> format that matches their app's locale. 

They do so because users have no option - hence this bug report :)

Unfortunately Linux chooses to use rfc3066[1] for Locales. This represents two completely different things as one value. IE: What LANGUAGE you speak, combined with a field separator of underscore, and then what REGION you are in. 

The first part has 487 languages, is defined by ISO 639[2], and is maintained by the International Information Centre for Terminology (Infoterm). 

The second part has 193 countries, is defined by ISO_3166-1[3], and is maintained by either United Nations Terminology Bulletin Country Names, or Country and Region Codes for Statistical Use of the UN Statistics Division.

Treating them as the same value would result in needing 93,991 entries to be complete, and the addition of one new region brings in an additional 487 entries. 

There are lots of reasons why people would want to use values not provided by the default Locales. As an example, the country of Ireland only gives the option of English and Irish languages, while the 2011 national census shows a total of 182 different languages in use. It continues to say "514,068 residents spoke a language other than Irish or English at home in 2011." Given ROI has a population of 4,792,500 this represents 10% of the install base that may wish to select a language other than Irish or English.[4]

The same extends to the Netherlands where we are restricted to Dutch, Limburgish, Low Saxon, and West Frisian. Despite 22.61% of the residents been described as non Dutch. Even then there is a sizable population of Dutch speakers that prefer to install English operating systems.

When you jump a flight to somewhere you are by default keeping the first part and changing the second. Something as simple as been able to switch country to get the correct paper format in the printer would be great.

So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.

Please point me to anywhere else I can open a bug on this.
> They do so because users have no option - hence this bug report

Umm, I don't think I can agree that all, or even majority, of users want a different date/time locale format than their selected product localization.

> So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.

We do. In particular, we separate our the language portion of the locale when we compare. And we normalize rfc3066 into bcp46. If you're interested in reading more, please take a look at https://firefox-source-docs.mozilla.org/intl/locale.html and the following chapter.

> https://dl84.panaxis.ch/screenshot_thunderbird.png

Jorg - can you take a look at what API call is used for the localization of the column in this screenshot? Based on the users support information, we should use `mozIntl.DateTimeFormat` with default locale which should be `de-CH` which should result in the date being formatted to `22.11.2018` while the screenshot shows `10/24/2018`. I suspect that we either use the generic `Intl.DateTimeFormat` or somehow mess up locale selection (by passing it to the constructor rather than leaving as undefined maybe?)
Flags: needinfo?(jorgk)

Comment 46

6 months ago
(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #45)
> > They do so because users have no option - hence this bug report
> 
> Umm, I don't think I can agree that all, or even majority, of users want a
> different date/time locale format than their selected product localization.
> 
I also can't agree but is it possible to get a breakdown of how may Windows/MAC users have non default Locales ? I would love to see some actual data for regions outside of the US.

> > So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.
> 
> We do. In particular, we separate our the language portion of the locale
> when we compare.

Unfortunately even if you do, I can't give them to you as I am limited by the installed Locales.

Comment 47

6 months ago
(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #45)
> Jorg - can you take a look at what API call is used for the localization of
> the column in this screenshot?
Sure, the C++ API, code here:
https://searchfox.org/comm-central/search?q=FormatPRTime&case=false&regexp=false&path=mailnews%2F
First hit, mailnews/base/src/nsMsgDBView.cpp
672 rv = mozilla::DateTimeFormat::FormatPRTime(dateFormat,
Flags: needinfo?(jorgk)
Thanks!

So that would indicate that for some reason this code https://searchfox.org/comm-central/source/mozilla/intl/locale/DateTimeFormat.cpp#94-97 does not do its job.

What it should do is hit: https://searchfox.org/comm-central/source/mozilla/intl/locale/gtk/OSPreferences_gtk.cpp#148 which in turn should hit https://searchfox.org/comm-central/source/mozilla/intl/locale/OSPreferences.cpp#171 and then modify the hourCycle based on data from GTK (talking about linux path).

Jorg - can you reproduce it and sprinkle printfs to see where it fails?

I'm wondering if the `mLocale` somehow doesn't end up being the `LocaleService::GetRegionalPreferencesLocale` here.
Flags: needinfo?(jorgk)

Comment 49

6 months ago
I'm afraid not since I'm on Bill's own OS Windows ;-) - Maybe you can get Aceman to help.
Flags: needinfo?(jorgk) → needinfo?(acelists)

Comment 50

6 months ago
The newest 60 version (60.2.1) which just became available to me on Debian resolved the issue, I guess there was a generic issue already which also fixed my problem - thanks for your help!
Interesting. Anyone can give it a try and confirm if it's fixed for them? Maybe we're chasing ghosts.

Comment 52

6 months ago
Hi,

I can confirm that in 60.2.1 x64 Linux (through Manjaro), the setting in 'Preferences > Advanced > Date and Time Formatting" is now honoured for emails (requires restart to take effect). The possible options are either English US or (I assume) the LC_TIME. In fact I have tested both with my system LC_TIME which is it_IT.UTF-8 as well as running from a terminal after changing LC_TIME to en_GB.UTF-8. Indeed this is also reported in the 'Help > Troubleshooting information' as the "Regional Preferences".

Hope this helps.
PS: I still believe that more customizability (as offered by the add-on cited above) would be nice, e.g. something similar to the date +FORMAT command in linux ;)

Comment 53

6 months ago
This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.
> This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.

Interesting, because we read LC_TIME - https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#44

Comment 55

6 months ago
(In reply to sbastig from comment #53)
> This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.
That is not a bug, LC_ALL is supposed to work like this (http://man7.org/linux/man-pages/man3/setlocale.3.html):
> [...] first (regardless of category), the environment variable LC_ALL is inspected, next the environment variable with the same name as the category [...], and finally the environment variable LANG.  The first existing environment variable is used.

Generally, LC_ALL should not be used at all, it is more of a "last resort" solution.

Comment 56

6 months ago
I did not actively set LC_ALL, it seems that some other program has set it. Possibly it was set by `dpkg-reconfigure locales`, but I am not certain. Anyway, I have removed it now and then LC_TIME started to take effect.

Comment 57

6 months ago
I've been using the aforementioned en_SE.utf8 workaround via a modified desktop file. After creating the en_SE locale, I essentially copied /usr/share/applications/thunderbird.desktop to ~/.local/share/applications/thunderbird.desktop and then made the following replacement:

s/Exec=thunderbird/Exec=env LC_TIME=en_SE.utf8 thunderbird/

This has lead to a somewhat usable interim fix, but the root of the problem appears to be caused by Unicode CLDR treating en_DK as if it were intended literally, rather acknowledging its origin as a workaround to bring ISO-8601 time formatting to English locales.

At the end of the day, I don't care how it happens, but I need ISO-8601 time formatting (YYYY-MM-DD dates and 24 hour time) in conjunction with the English language for the United States.

I also find it strange that my Thunderbird Preferences/Advanced/Date and Time Formatting show the Application locale as English (United Kingdom) rather than English (United States), but that is not really relevant to the issue of the en_DK ISO-8601 workaround breaking after the upgrade to Thunderbird 60.
Just a small addition to the workaround using en_SE:

There is also the `root' locale in the CLDR, which does not insert a comma between date and time.  Like `en-SE', it is not part of the glibc locale set, hence requires the same fiddling.

Updated

6 months ago
See Also: → 1502659

Comment 59

6 months ago
Thanks Einhard, that works perfectly. To clarify, the full workaround is:

sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/root
echo 'root.UTF-8 UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen

And then set LC_TIME=root.UTF-8

Comment 60

6 months ago
Any help here for a Fedora user?  I can't use the custom locale as described -- I don't have a locale.gen file, at minimum.  I'm using KDE/Plasma and have most all locale settings there set to United States - American English (en_US), except for Time which I set to Sweden - English (en_SE) (where they do time right apparently!).  The samples that Plasma's System Settings shows for the various formats look perfect.  In bash, I see:

$ locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_SE.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

That looks good.  In TB, however, date/times are still shown like 10/31/18, 9:38 PM.  I've switched Edit > Preferences > Advanced > General > Date & Time Formatting to both "Application locale: English (UK)" and "Regional Settings: English (US)" and in both cases I see the exact same results (like 10/31/18, 9:38 PM).
@jflorian, I had a problem with en_SE in Fedora as "not supported by C library" and I ended up with "lt_LT" which has a proper short date format. As days and others are not English I set it only for Thunderbird in the launcher file (as mentioned above in the comment #57):
> Exec=env LC_TIME=lt_LT.UTF-8 thunderbird %u

The short date is printed as with en_DE before: 2018-11-04 01:57. The day names in Lightning don't seem to be affected and I haven't encountered any visible side effects until now.

Comment 62

6 months ago
I can confirm the bug in TB 60.2.1 on Kubuntu 18.04.1 with KDE 5.12.6.

$ locale                                                                                                                                                                                                                       
locale: Cannot set LC_CTYPE to default locale: No such file or directory                                                                                                                                                                     
locale: Cannot set LC_MESSAGES to default locale: No such file or directory                                                                                                                                                                  
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_DE.UTF-8
LANGUAGE=en_DE.UTF-8
LC_CTYPE="en_DE.UTF-8"
LC_NUMERIC="en_DE.UTF-8"
LC_TIME="en_DE.UTF-8"
LC_COLLATE="en_DE.UTF-8"
LC_MONETARY="en_DE.UTF-8"
LC_MESSAGES="en_DE.UTF-8"
LC_PAPER="en_DE.UTF-8"
LC_NAME="en_DE.UTF-8"
LC_ADDRESS="en_DE.UTF-8"
LC_TELEPHONE="en_DE.UTF-8"
LC_MEASUREMENT="en_DE.UTF-8"
LC_IDENTIFICATION="en_DE.UTF-8"
LC_ALL=en_DE.UTF-8

Edit > Preferences > Advanced > General shows both the "Application locale" and the "Regional settings locale" as "English (United States)".


I consider this a high-priority issue; TB and the Lightning calendar are the cornerstone of my job and the date and time formats are very confusing to me. I would also suggest that, as a fallback, at least rely on ISO-8601 time formatting. (I'm assuming en_US is the standard fallback as nothing on my system should indicate en_US to TB.)
Looks to me you haven't set up the locale. Probably need something like

  locale-gen en_DE.UTF-8
  sudo pkg-reconfigure locales

Comment 64

6 months ago
(In reply to Marcin Zajaczkowski from comment #61)

> The short date is printed as with en_DE before: 2018-11-04 01:57.

Confirmed.

> The day names in Lightning don't seem to be affected and I haven't encountered any
> visible side effects until now.

My calendar definitely has the day names affected, not at the top of the calendar grid, but in the "Events in the Next 14 days" area shown at the top of my calendar tab along with the Events sidebar on my mail tab.  There I see things like "2018 m. lapkricio 8d., ketvirtadienis 13:00" (more or less; I couldn't copy/paste and there are lots of diacritics I'm not attempting).  This solution doesn't look like it'll work for me.  Thanks for the response though.
(In reply to jflorian from comment #64)
> My calendar definitely has the day names affected, not at the top of the
> calendar grid, but in the "Events in the Next 14 days" area shown at the top
> of my calendar tab along with the Events sidebar on my mail tab.  There I
> see things like "2018 m. lapkricio 8d., ketvirtadienis 13:00" (more or less;

I'm not affected probably thanks to:
> Thunderbird preferences -> General -> Date Text Format -> Short: 2018-11-05
(which I was set by default in my 10 years+ profile or I changed it long time ago and I forgot about that)

It's just an another workaround. I would be happy having the old en_DK behavior back in Thunder 60+.

Comment 66

5 months ago
(In reply to Marcin Zajaczkowski from comment #65)
> I'm not affected probably thanks to:
> > Thunderbird preferences -> General -> Date Text Format -> Short: 2018-11-05
> (which I was set by default in my 10 years+ profile or I changed it long
> time ago and I forgot about that)
> 
> It's just an another workaround. I would be happy having the old en_DK
> behavior back in Thunder 60+.

My Thunderbird (thunderbird:amd64/xenial-security 1:60.2.1+build1-0ubuntu0.16.04.4 uptodate) has no Date Text Format settings under Preferences/General.
I think he means the (lightning) preference on Preferences | Calendar | General
Yes, my bad.

> Thunderbird preferences -> Calendar -> General -> Date Text Format -> Short: 2018-11-05

Comment 69

5 months ago
I've just upgraded TB 52.9.1 -> 60.3.0 and was hit with this issue. I would like to put some weight on providing ISO date option, for the sake of sanity, I guess. 
I can not find anything related to CLDR/en_DK/ISO date situation, other than one alternative opinion (https://bugreports.qt.io/browse/QTBUG-59382?focusedCommentId=350361&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-350361). Does anyone know is CLDR was ever bugreported about this? searching '"en_dk" site:unicode.org/mail-arch' yields no results.

Comment 70

5 months ago
(In reply to Magnus Melin [:mkmelin] from comment #63)
> Looks to me you haven't set up the locale.

Thank you, you were right about this. Fixing that made another potential issue visible: Dates are now shown as DD.MM.YY although explicitly defined as DD.MM.YYYY in the new glibc locale (with: d_fmt   "%d.%m.%Y" ). But that's the same across KDE (date in taskbar, Dolphin etc.). Yet it seems inconsistent that TB would look for the glibc locale but then rely on another setting buried elsewhere in the system (and I don't know where KDE enforces DD.MM.YY).

Comment 71

5 months ago
For me setting LC_TIME by itself also does nothing (not shown in TB prefs).
Only when I set LC_ALL then the en_DK is picked up and used (e.g. in message date display) and shown as English (Denmark) in TB preferences.

Unsetting LANG and LC_ALL also makes it work in TB, however then the 'locale' command shows strange output:
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME=en_DK.utf8
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

So that does not seem as safe, as one would want to have the others set to some locale. Maybe you need to set all of them explicitly?

Comment 72

5 months ago
Can someone help me understand if bug 1505435 is a duplicate of this one or a different issue?

In a nutshell, what I reported there is that on Ubuntu, `Intl.DateTimeFormat().resolvedOptions().locale` is insisting on using en-GB date formatting no matter what I do. I've set locale environment variables, about:config preferences, the "Date and Time Formatting" preference in the UI (which is set to "Regional settings locale: English (United States)"), none of it seems to help.

I have both en-GB and en-US locales installed because Ubuntu requires the en-GB package as a dependency of the en-US package, so you can't install en-US without also installing en-GB.

I have tried reading the code in DateTimeFormat.cpp to understand what is supposed to be happening, and frankly it is so convoluted that I found it impenetrable.

If someone here could perhaps explain to me what _should_ be happening, I'd be happy to run Thunderbird under a debugger and try to understand what _is_ happening and how it's broken. Am I gathering correctly that no one has yet been able to do that?
> Can someone help me understand if bug 1505435 is a duplicate of this one or a different issue?

Different.

We have two "scenarios" in which we format Intl data (in this case, date/time) - we either do this as part of "privileged" code in the UI of the Gecko app, or as part of the "unprivileged" website.

Bug 1505435 is about what we do for websites. That's what the `Intl.DateTimeFormat().resolvedOptions().locale` do.

This bug is about what we do in the privileged code.

The difference between those two paths is not huge, but can be meaningful in edge cases. We use the same API, but we feed it with different data. For "chrome privileged" UI we use `mozIntl.DateTimeFormat()` which looks into several operating system environment variables and in result *should* format your date/time closer to your OS preferences.

For the web-exposed API we *do not* provide that information because it would mean leaking it and increasing fingerprinting.

I still don't understand why this bug exists as indicated in comment 48 and comment 54. As for bug 1505435 - for UI we should always use mozIntl, not Intl, if we can.
I can confirm this behavior in TB 60.2.1 on Linux Mint 18 Cinnamon 3.0.7 64-bit Linux Kernel 4.4.0-134-generic.

user agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1
buildid: 20181010142946

In the prior version I was using, I had implemented the en_DK.UTF-8 workaround from the above mentioned Mozillazine article to have the date/time formated like the ISO date/time format.

I have congigured 24hr time in the desktop environment.

I have LC_ALL defined but set to null value.

I have LC_TIME=en_DK.UTF-8

All other LC_ variables are en_US.UTF-8.

Date format was ISO like in the prior version, it changed to the format as reported by the OP.

After reading this thread, I have tested both the en_SE.UTF-8 and root.UTF-8 locales for LC_TIME and can confirm both produce an ISO like date/time format en_SE.UTF-8 'YYYY-MM-DD, HH:mm' and root.UTF-8 'YYYY-MM-DD HH:mm'

I have chosen to use the LC_TIME=root.UTF-8 workaround since 1) it does not include the comma and 2) seems to be more generic/universal of a locale definition less tied to language or region. That better fits the intent of the ISO date/time format.

My view of the locale treatment in Linux vs. Windows & Mac; Linux does have different support for customization than either Windows or Mac, but it does support customization. I can chose a different locale for the LC_TIME than all the others, my expectation is that the format specified in the generated locale file is what I should see. I do not know about Macs, but I know Windows implemented this through the registry and presented a UI and API for the custom format. I could edit the registry to change it. In Linux I can find and load a locale definition file in /usr/share/i18n/locales, create my own, or modify an existing one. I could modify the en_US file to achieve the same goal. The format expected for date/time is the format specified in the file specified by LC_TIME. If I so chose I think I could define a custom date/time format not used anywhere else in the world and set LC_TIME to it, I would expect that format to be used. I have experimented in Windows before to do just that. I do not expect the format to come from some other reference that the system itself does not use. my 2 cents.

Comment 75

5 months ago
I am also seeing this problem, on Red Hat Enterprise Linux 7.6, which updated to thunderbird-60.3.0-1.el7_5.x86_64. (RHEL 7.5 had thunderbird-52.9.1-1.el7_5.x86_64.)

For the longest time, I have used the trick of setting LC_TIME to en_DK.UTF-8 in order to get ISO-8601 -style date and time specifications:

$ cat ~/.i18n
LC_TIME="en_DK.UTF-8"

$ env -u LC_TIME date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: Mon 12 Nov 2018 08:14:15 PM EST
locale date: 11/12/2018
locale time: 08:14:15 PM

$ env LC_TIME=en_DK.UTF-8 date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: 2018-11-12T20:14:39 EST
locale date: 2018-11-12
locale time: 20:14:39

If I instead set LC_TIME to "lt_LT.UTF-8" (per Martin's comment 61), in Thunderbird, the date/time is displayed in ISO-8601 format, even though lt_LT uses YYYY.MM.DD instead of YYYY-MM-DD:

$ env LC_TIME=lt_LT.UTF-8 date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: 2018 m. lapkričio 12 d. 20:15:19
locale date: 2018.11.12
locale time: 20:15:19

Other than ensuring that Preferences→Advanced→General→Date and Time Formatting is set to "Regional settings locale", the only thing I have to do is launch Thunderbird with the lt_LT.UTF-8 LC_TIME setting. E.g., I copied the Thunderbird launcher out of the GNOME menu and modified it to be:

env LC_TIME=lt_LT.UTF-8 thunderbird %u

Otherwise, /etc/locale.conf (which is sourced by /etc/profile.d/lang.sh) sets LANG to en_US.UTF-8.

I wish this weren't so complicated (and/or buggy), but hopefully the lt_LT.UTF-8 trick works for others.

Comment 76

5 months ago
(In reply to gshollingsworth+mozilla from comment #74)
> ... My view of the locale treatment in Linux vs. Windows & Mac; Linux does have
> different support for customization than either Windows or Mac, but it does
> support customization. I can chose a different locale for the LC_TIME than
> all the others, my expectation is that the format specified in the generated
> locale file is what I should see. ... In Linux I can find and load a locale
> definition file in /usr/share/i18n/locales, create my own, or modify an existing
> one. ... If I so chose I think I could define a custom date/time format not used
> anywhere else in the world and set LC_TIME to it, I would expect that format to
> be used. ... I do not expect the format to come from some other reference that
> the system itself does not use. my 2 cents.

Completely agree that is basically why I reported the bug 1502659 (the usable workarounds are listed there maybe more clearly as there is a lot less comments:).
ISO-8601 format dates are the preference of quite a few people across languages and cultures for many reasons including those that lead to the standard being created in the first place.  As it is an international standard, globally accepted, for formatting date and time and embraced by most technical and many non-technical users alike, I believe it would be justifiable to make a special case in the date/time formatting dialog to add the option, right in the advanced/general UI preferences, to select this international, global standard for date/time formatting regardless of other I10N choices.

The extremely user-unfriendly hoops one has to jump through to present the date in a format that is an international standard are utterly absurd.  That there's a massive comment flood of highly technical people struggling to figure out how to do something so simple and basic is a clear proof that the UI model needs a correction.

ISO-8601 is /the/ global standard for date-time.  There is not another one.  There's no competition.  Adding an ISO-8601 option will not open the door to requests to add radio buttons for other standards because they're aren't any other global standards for the unambiguous representation of date and time.  Enabling easy selection of ISO-8601 simply makes a standards compliant experience something normal users can achieve, rather than an absurdly complicated, fragile exercise in root-level config file hacking.

Comment 78

5 months ago
(In reply to gessel@blackrosetech.com from comment #77)

> ISO-8601 is /the/ global standard for date-time.  There is not another one. 
> There's no competition.  Adding an ISO-8601 option will not open the door to
> requests to add radio buttons for other standards because they're aren't any
> other global standards for the unambiguous representation of date and time. 
> Enabling easy selection of ISO-8601 simply makes a standards compliant
> experience something normal users can achieve, rather than an absurdly
> complicated, fragile exercise in root-level config file hacking.

Well said.  I just want to add the MSF should take the lead here and not just with TB.  Please make this simple, if not the startup default.  It's time for the world to end the absurdities with the ludicrous number of time formats and the ridiculous complexities they impose on software and its users.  ISO-8601 is the global standard.  Let's embrace it and let the other formats fail for what they really are, relics of an un-unified world.  Those who wish to cling to the past should be the ones making herculean efforts, not those who want to improve the situation.

Comment 79

5 months ago
This issue is not about ISO 8601 date formatting. It's about the fact that the behavior of Thunderbird changed to no longer allow the user to set the time/date format as per the locale. Please file a separate bug (enhancement request, really) if you want Thunderbird to natively support the ISO 8601 date format, and stop commenting about it here.

Comment 80

5 months ago
An explicit option to enable ISO-8601 time formatting in the Thunderbird UI is definitely something that belongs in it's own enhancement request.

The assertion that this bug has nothing to do with ISO-8601 couldn't be further off the mark, though. While the string "ISO-8601" isn't in the bug title, the parenthetical description "(LC_TIME=en_DK.utf8 behaves differently than it used to)" makes it very much about ISO-8601 support. Setting LC_TIME to en_DK.utf8 has been a long standing documented solution to the lack of sensible date formatting options for English speaking users.

It's the entire reason that the en_DK locale exists.

Comment 81

5 months ago
To end the argument about applicability of this bug report to ISO 8601 date issue, I've just opened new unambiguously related one:
https://bugzilla.mozilla.org/show_bug.cgi?id=1509096

Comment 82

5 months ago
As Jonathan Kamens said, this is not about ISO 8601.  It is about the fact that you can create new locales: And people do!  Proof above is "localebug" who uses en_DE.

I use en_LU, and it has worked perfectly over the years.  Now?  I get the C locale, which is basically en_US. The log entry I see is:

(thunderbird:5925): Gtk-WARNING **: 09:23:34.665: Locale not supported by C library.
	Using the fallback 'C' locale.


It is not even hard to make a new locale: you copy the one you want and make a few changes, then you add it to the locale-gen configuration file (location depends on distro) and then you run locale-gen, and modify your /etc/defaults/locale.  You do that once, and you're fine.

That's why this system exists.

On Windows I do the same: I take en_GB and modify the settings to use our local date/time and currency settings.

On Linux, I usually take /usr/share/i18n/locales/fr_LU, copy it to /usr/share/i18n/locales/en_LU, and modify the field "language" under LC_IDENTIFICATION.  Why do this?  Because if I use fr_LU, I get my dates formatted like this: "23/11/2018, à 9h34" and the interface will use French weekdays.  I don't want that.

This post is that I fully disagree with the choice to just support only "well known locales".  Yes, you can do this with using LC_ environment variables and mix-and-matching, but the flexibility is so much worse than making your own.

Remember, Linux is supposed to give you Freedom.

Comment 83

5 months ago
> 
> (thunderbird:5925): Gtk-WARNING **: 09:23:34.665: Locale not supported by C
> library.
> 	Using the fallback 'C' locale.
> 

Sorry, this is when you don't have the locale installed. (Quick test on a machine that wasn't configured like this, my mistake)  However, if you do have a custom locale, the warning disappears but, the dates remain in C locale.

Comment 84

5 months ago
This is infuriating. I observe that thunderbird is in fact responding to changes in the LC_TIME environment variable, including LC_TIME=en_DK.UTF-8, but not the way it's supposed to. Specifically. I have tried C, ru_RU.UTF-8, en_GB.UTF-8, en_DK.UTF-8, and da_DK.UTF-8. These are the results:

en_GB is supposed to be 26/11/18, but thunderbird does 26/11/2018. It gets the time right, though.

en_DK is supposed to be 2018-11-26 and 17:07, but thunderbird does 26/11/2018 and 17.07. Note the . instead of a :. Really weird.

da_DK is supposed to be 26-11-2018 and 17:07, but thunderbird does 26.11.2018 and 17.07.

C is supposed to be 17:07, but thunderbird does 5:07 PM. It gets the date right, though.

ru_RU actually works like it's supposed to, but given the above results, I think this might be a coincidence.

There is no way this is intended behaviour. To me, this looks like some kind of memory corruption, or a broken linked list, or something. If over the coming weeks it **** me off enough, I'll step through the date rendering code with a debugger and maybe figure out what in the hell's going on.

Under Preferences>Advanced>General, Date and Time Formatting was set to Regional settings locale the whole time.

When I switch this preference to Application locale, everything basically still remains broken, in the same way. I don't feel like testing this in great detail. Oddly enough, though, even here, ru_RU is the only one that works like it's supposed to, causing English (US) dates to be displayed.
>  but not the way it's supposed to.

It would be useful to understand what do you reference to as what's "supposed to be".

We follow Unicode CLDR which provides date/time patterns for locales. You can review them for example here:

http://www.unicode.org/cldr/charts/34/summary/ru.html#2060
http://www.unicode.org/cldr/charts/34/summary/fr.html#2106

and so on. If you see an example where Gecko does not follow CLDR locale, please provide the output of the Intl section of about:support (bottom of the page) and STR. Thank you!

Comment 86

5 months ago
(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #85)
> >  but not the way it's supposed to.
> 
> It would be useful to understand what do you reference to as what's
> "supposed to be".
> 
> We follow Unicode CLDR which provides date/time patterns for locales. You
> can review them for example here:
> 
> http://www.unicode.org/cldr/charts/34/summary/ru.html#2060
> http://www.unicode.org/cldr/charts/34/summary/fr.html#2106
> 
> and so on. If you see an example where Gecko does not follow CLDR locale,
> please provide the output of the Intl section of about:support (bottom of
> the page) and STR. Thank you!

Thank you for replying so quickly, Zibi. This explains everything. I am comparing to the output of glibc's strftime(). The CLDR is obviously at odds with glibc as to the correct way of displaying the dates in all of these locales. Except for Russia. Apparently, Russia is much more assertive and decisive than the other countries I tested when it comes to communicating with these two organisations ;)

So, not-a-bug, sort of?

Is there a value of LC_TIME that the CLDR will interpret as ISO? I can set LC_TIME to that just for thunderbird. That would be a pretty good workaround. I'll start rummaging around the CLDR documentation and see what I find.
See whiteboard. At least pretty close, LC_TIME=en_SE.utf-8

Comment 88

5 months ago
I just read the previous posts. I really should have actually read this thread more thoroughly before. Looks like everything was already addressed. And I could have saved an enormous amount of time debugging if I had known to break on mozilla::DateTimeFormat::FormatPRTime. (whoops...)

I think it does make sense to have a value of LC_TIME that causes ISO dates to be rendered, but the en_DK hack frankly never made sense to me. What we need is a locale like LC_TIME=iso8601. There is a precedent for a non-localised locale: LC_TIME=C. Unfortunately, C is a complete locale usable for anything and corresponding roughly to en_US. There is no precedent for a partial locale AFAIK. I think it will be really really hard to sell the CLDR maintainers on this one.

It might still be possible to sell the CLDR maintainers on the en_DK hack, though. Failing that, I see two alternatives:

0. a new preference in thunderbird for ISO dates, as was suggested earlier in the thread.

1. a new preference for the local C standard library's strftime(). It is a part of the ISO C standard, therefore it is virtually guaranteed to be present on all platforms in some form or other.

Incidentally, it is icu that thunderbird uses to do the date formatting. Specifically, udat_open() is used to translate the locale, e.g. en-DK, into a format string.


A GOOD WORKAROUND
=================
I observe that sv_SE displays all dates and times exactly to ISO 8601 everywhere I've seen them so far. These are the steps to switch when using archlinux. I'm assuming it's basically the same in other distros.

0. Generate sv_SE. (Do this only once.)
$ sudo vim /etc/locale.gen
***uncomment sv_SE.UTF-8***
$ sudo locale-gen

1. Tell thunderbird to use sv_SE.UTF-8 from now on.
$ LC_TIME=sv_SE.UTF-8 thunderbird

Since en_DK is probably not working as expected in every application that uses icu to do its date formatting, this should fix them too. I already observe this workaround fixing the dates in qbittorrent! Yay! :D Today has been a good day.

You might have to learn the days of the week in Swedish. Whatever. It's basically just English with a funny accent.

Comment 89

5 months ago
(In reply to Magnus Melin [:mkmelin] from comment #87)
> See whiteboard. At least pretty close, LC_TIME=en_SE.utf-8

I observe there is no such locale as en_SE in glibc. I looked in /etc/locale.gen. I am using glibc version 2.28.
You just have to generate it

  locale-gen en_SV.UTF-8
  sudo pkg-reconfigure locales

FWIW, this is my global setup (my mother tongue happens to be Swedish, but I only want the time/date formatted locally - application texts and such in English).

Comment 91

5 months ago
Is it possible for a Thunderbird extension to override the used date format string?

Comment 92

5 months ago
(In reply to Magnus Melin [:mkmelin] from comment #90)
> You just have to generate it
> 
>   locale-gen en_SV.UTF-8
I'm assuming you meant _SE.
>   sudo pkg-reconfigure locales
> 
> FWIW, this is my global setup (my mother tongue happens to be Swedish, but I
> only want the time/date formatted locally - application texts and such in
> English).

locale-gen ignores its argument. Adding en_SE.UTF-8 to locale-gen won't do anything. Your thunderbird is probably just failing over to sv_SE.UTF-8, which is why it looks like it worked. If I add en_SE.UTF-8 to /etc/locale.gen and run locale-gen, this is what happens:

% sudo locale-gen
Generating locales...
  da_DK.UTF-8... done
  en_DK.UTF-8... done
  en_GB.UTF-8... done
  ru_RU.UTF-8... done
  sv_SE.UTF-8... done
  en_SE.UTF-8...[error] cannot open locale definition file `en_SE': No such file or directory
%
Sorry, yes, should be LC_TIME=sv_SE.utf8
Whiteboard: [workaround: use LC_TIME=en_SE.utf-8 instead]

Comment 94

5 months ago
(In reply to vladimir-csp from comment #81)
> To end the argument about applicability of this bug report to ISO 8601 date
> issue, I've just opened new unambiguously related one:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1509096

Thanks, that is quite similar to what I proposed when reporting bug 1502659. It is especially nice that it seems there is a chance that a proper solution to all these date/time formating issues will be implemented. Thus, I voted for your bug. Maybe other people affected by the date/time formating issues should also vote there:).



As a side note:
Listing the workarounds using root.UTF-8, en_SE.UTF-8 and other locales using whiteboard could help most people affected by the date/time formating issues who find this bug. Thus, it would be beneficial to add an updated info about workarounds back to the whiteboard. For similar reasons the bug 1509096 should be added to "See Also". Unfortunately I do not have privileges to do this.
Whiteboard: [workaround: use LC_TIME=sv_SE.UTF-8 instead]

Comment 95

5 months ago
Thanks! sv_SE workaround is working.

So, am I correct that Thunderbird and Firefox do not support custom locales in principle?

Custom locales are the proper way of customizing such things in the system. For example to customize a ru_RU locale you copy locale definition file ru_RU into ru_RU@ISO, replace content of every section with 'copy "ru_RU"' (for deduplication and smooth updates), except LC_TIME, in which a custom ISO-like string can be defined appropriate for the locale being copied. New locale will be compiled by locale-gen into ru_RU.UTF-8@ISO (@suffix is a proper way of defining variants). And this will work for everything except Thunderbird/Firefox, apparently.

Comment 96

5 months ago
(In reply to vladimir-csp from comment #95)
> Thanks! sv_SE workaround is working.
> 
> So, am I correct that Thunderbird and Firefox do not support custom locales
> in principle?
> 
> Custom locales are the proper way of customizing such things in the system.
> For example to customize a ru_RU locale you copy locale definition file
> ru_RU into ru_RU@ISO, replace content of every section with 'copy "ru_RU"'
> (for deduplication and smooth updates), except LC_TIME, in which a custom
> ISO-like string can be defined appropriate for the locale being copied. New
> locale will be compiled by locale-gen into ru_RU.UTF-8@ISO (@suffix is a
> proper way of defining variants). And this will work for everything except
> Thunderbird/Firefox, apparently.

It's not just thunderbird/firefox. Hacking the glibc locale won't work in any program that uses icu instead of glibc to do its localisation. That includes many QT programs, like qbittorrent.

I'm not happy with this state of affairs either.

Comment 97

5 months ago
You can still hack icu though. So now, you have to hack two locale definitions instead of just one lol.
> You can still hack icu though. So now, you have to hack two locale definitions instead of just one lol.

Hahha, yeah! I remember there's a push to move glibc to use CLDR. I can't wait for it to happen :)

Updated

5 months ago
Flags: needinfo?(acelists)

Comment 99

4 months ago
thunderbird v60.3.0
opensuse LEAP 15.0 / linux 4.12.14

AFAICT TB does not do well with locale times any longer. I, too, lamented the loss of choosing my preferred datetime format, basically ISO-8601. More sadly for me, a very nice addon, SuperDateFormat, was not updated for the new plugin API. So I went in search of an alternative method since the choices I had were en_US or en_US.

I quickly discovered the en_DK hack. And it works, sort of, in a vague way resembling what I wanted.

My preference for the displayed datetime is "YYYY-MM-DD HH:MM:SS" or something quite similar. Supposedly en_DK would deliver that format. Looking at the contents of en_DK, it should.

What I get is "DD/MM/YYYY, HH:MM". That pattern exists nowhere in en_DK; it appears to be bastard combination of en_US and en_DK. It exists only in TB's code logic somewhere. It is an improvement over the default; not what I expected.

And, really. WTF is so wrong about providing a custom date presentation option? Or even a limited list of pre-processed options?

Comment 100

4 months ago
(In reply to Jim Moe from comment #99)
> thunderbird v60.3.0
> opensuse LEAP 15.0 / linux 4.12.14
> 
> AFAICT TB does not do well with locale times any longer. I, too, lamented
> the loss of choosing my preferred datetime format, basically ISO-8601. More
> sadly for me, a very nice addon, SuperDateFormat, was not updated for the
> new plugin API. So I went in search of an alternative method since the
> choices I had were en_US or en_US.
> 
> I quickly discovered the en_DK hack. And it works, sort of, in a vague way
> resembling what I wanted.
> 
> My preference for the displayed datetime is "YYYY-MM-DD HH:MM:SS" or
> something quite similar. Supposedly en_DK would deliver that format. Looking
> at the contents of en_DK, it should.
> 
> What I get is "DD/MM/YYYY, HH:MM". That pattern exists nowhere in en_DK; it
> appears to be bastard combination of en_US and en_DK. It exists only in TB's
> code logic somewhere. It is an improvement over the default; not what I
> expected.
> 
> And, really. WTF is so wrong about providing a custom date presentation
> option? Or even a limited list of pre-processed options?

See my post with title "WORKAROUND" ^^^^^^^^^^

Comment 101

4 months ago
Also, I know this thread is super long, but reading it is worth it. It explains everything. Short story: firefox, thunderbird, and many other programs use icu now instead of glibc and the two disagree on a lot of formatting details.

Comment 102

4 months ago
In case it's too hard to find, the post with title "WORKAROUND" is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c88

Comment 103

3 months ago

I am a GNU C Library (glibc) developer and I support the POSIX APIs for localization and internationalization.

I am also impacted by this change.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there. Having the glibc locales and Unicode/ICU/CLDR locales in sync means fewer surprise for our users. This can include adding new "formats" like those which CLDR has for dates. For example glibc just recently added alternative date formats to support two grammatical forms for month names (%OB and %ob), particularly for Slavic/Baltic languages that use genitive/nominative cases.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

A good summary of the problem looks like this:

  • Thunderbird has stopped allowing users to customize their date formats. Modifying the CLDR data is significantly more complex (expert level). Thunderbird tries to guess your appropriate CLDR locale by reading your system locale. There is no customization allowed beyond that.

  • Previously Thunderbird using POSIX APIs and glibc's locale support. Users could build their own locales with their own formats for their system. For example I have my own 'en_US@ISO8601.UTF-8' that has ISO 8601 time for US English. It's really easy to do this.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR (which I assume is used by Mozilla because it's a multiplatofrm library).

If you have any questions about the glibc APIs or other things, I'm happy to help, and improve them.

So far none of the extensions I have found solve the problem either. I don't consider it acceptable to switch to sv_SE locales.

Hi Carlos! Thanks for bringing your perspective. I'll try to respond to it.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there.

That's wonderful! I truly hope that we'll end up all using CLDR, since I see no value in duplication of efforts.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

Comment 105

3 months ago

While I can appreciate consolidation, one thing that CLDR has done is turned what was a deliberate hack into a literal interpretation. en-DK was never meant to be a locale that had anything to do with Denmark. It was created to easily allow ISO-8601 date and time formats ofr English speakers. CLDR taking it literally as English Denmark has introduced undesirable characteristics such as number formatting and currency.

In the grand scheme of things, as an affected user I don't care as long as there is an eventual return of an easy way for me to get ISO-8601 dates and 24 hour time. I've implemented the en_SE.utf8 workaround by creaingan en_SE.utf8 system locale and modifying the envvars in the Exec lines of my user specific thunderbird.desktop file, but long term this should be offloaded to something more intuitive and accessible to less technical users.

Comment 106

3 months ago

For example I have my own 'en_US@ISO8601.UTF-8' that has ISO 8601 time for US English. It's really easy to do this.

I'm more than happy to try and create my own custom format but it seemed to be way above my technical level. That said if you can point me to a guide, documentation, video, podcast etc on how to do that I would seriously appreciate it.

Comment 107

3 months ago

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #104)

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

I don't see this as an edge case.

All users want to customize the display in their mail client.

Lack of time customization is simply a failing on behalf of the Thunderbird application.

However, I am not a Thunderbird developer, and so I don't get to make prioritization decisions, you need to do that.

This is high priority for me. The swapped en-US m/d/y will cause me to make mistakes and I can't tolerate that, so I'm going to probably switch back to Mutt, or another client after review.

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

Easy for anyone. I'll show you. It was designed this way to be easy.

dnf install glibc-locale-source
su
cp /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_US@ISO8601
vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/en_US@ISO8601.utf8

You're done. This is impossible to do with ICU/CLDR.

Even better is you can ship this in Ubuntu Universe or Fedora Copr.

Non-technical users don't even need to do anything but point at the repo, install the compiled locale, and set their LANG env var.

Again, this cannot be done with CLDR.

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

No, everything else is also easy. Just look at Windows.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

[1] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

We had exactly this kind of flexibility with system locales, but ICU/CLDR does not provide that.

Windows 10 lets you write the patterns out exactly how you want to see them, so do Linux locales.

You could implement this in Thunderbird via ICU/CLDR APIs:

UnicodeString aPattern("GyyyyMMddHHmmssSSZ", "");
new SimpleDateFormat(aPattern, new DateFormatSymbols(Locale::getUS())
...

Example taken from: http://userguide.icu-project.org/formatparse/datetime

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

I'll look at glueing this into the system locale. It looks like we already have "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);" to fetch the preference which is somewhat mislabelled as "true" for Regional settings locale (means use the system defaults). I might propose another patch to change that, but we still have to honour that setting when picking a date format.

Comment 108

3 months ago

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-) Sorry.

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

Correct, from the GUI. And that includes validation of pattern syntax etc.

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-)

No worries, it was added very recently and we all get confused :)

I'll look at glueing this into the system locale.

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

Comment 110

3 months ago

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

I agree 100%. This is what I meant by "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);", which fetches the preferences from the store. The Preferences->Advanced->Date and Time Formatting radio controls actually set this to false (use ICU/CLDR) or true (try to use the OS information).

Comment 111

3 months ago

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

We must agree it's non-zero. Quite a few users are present here alone, but a simple Google search for the old en_DK hack shows there are many users who want this and those are just the few who took the time to make such a post. While many of those posts may not be specific to TB, I think it's safe to assume that if they do use TB, they'd expect their preferred format there too.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Technical proficiencies vary wildly. But if TB no longer honors the glibc formats by any means short of personally applying a patch, I'd argue that it has been made extremely difficult for anyone. This sort of exclusion is the same reason I can't use GNOME: I don't always agree with the designers and want some choice to wiggle into a more comfortable setup that works for me. It's sad that something like this is now making me consider giving up many years of getting very comfortable with TB for an alternative MUA. First it's the loss of Google search, now this. I'm likely to start looking into kmail again -- I know it won't leave me hanging on something so basic as this. I'm losing faith in MSF. :(

Comment 112

3 months ago

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

The mechanism for locale customization exists and works. It does not matter whether a user utilizes it via terminal or via some specialized user-friendly GUI.

Comment 113

2 months ago

Here are two fixes I came up with for this bug.

The first implements a new string preference named intl.locale.date_time.lc_time_override. If this preference is found, its value is used instead of the LC_TIME environment variable value. An example value for this preference is "root.UTF-8", which yields the ISO 8601 date/time format users want. This preference eliminates the need for fiddling with the platform's locale system or changing LC_TIME when starting Thunderbird.

The second implements a new string preference named intl.locale.date_time.pattern_override. If this preference is found, its value is used as the date/time pattern instead of the one that Mozilla comes up with. An example value for this preference is "y-MM-dd hh:mm", which yields the ISO 8601 date/time format users want. This preference provides very direct, granular control over the display of date/time values.

The code modifications can be found here:

https://gitlab.com/jeff_harp/thunderbird_60.4.0_date-time_mods

See the README.md file for more details.

I've allocated as much time to this as I can. If someone else wants to pursue these modifications further, please do so. I cannot.

@Jeff Harp
Thank you for your efforts.

I really like your second fix approach. To me it is very much like how I am used to fixing the issue in other apps and platforms. It should also work cross platform.

For me, your first approach does little for me. I want this same format system wide, so I already have locale and LC_TIME customized. It seems to apply only to Linux/Unix/BSD. I can see it's validity for others though.

Comment 115

2 months ago

I've used Thunderbird since its start, I have saved many (many) clients from buying 365 with this and Libre office. This is the only problem which irks me, smacks a bit of "we know best" it just seems right to allow the user to choose the format they prefer.
Any fix would need to affect the add ons, lightning is unusable for me because of this problem.
Regards

I'm just an end-user frustrated with this situation. I have also used Thunderbird since its start, moving during the years from OS/2 to Windows to Mac and now to Linux Mint. I'm from Finland, my native language is Swedish, but my systems have always been in English (US or GB). I have lived and worked in several European countries, now in France, and I want to have date and time formats according to the country where I live. This country/language/currency cocktail is maybe complicated for the Americans, but this is the way the world works. Please fix this.
Thank you.

Comment 117

a month ago

And simply setting the advanced option "Date and Time Formatting: [x] Regional settings locale French (France)" doesn't work assuming that you have set your machine to a French locale? As I understand the bug, it's about a very specific use case which is not what you're describing.

No, it does not work. Thunderbird insists on en-US. The machine is set to regional French(France) and everything else respects that setting for date/time/hour/currency/decimal

Comment 119

a month ago

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

(In reply to Jorg K (GMT+1) from comment #119)

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

Well, I would actually be curious about how so.

This is the exact reason I'm CC'd to this bug and others: My system is using en_US as locale, but I obviously want to use 24H format, as well as changing dates for example if possible.

Current format force user to some mental exercise every time he needs to read date, time and so on.

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.
Now Thunderbird shows Regional settings locale as English (United Kingdom), dates show as DD/MM/YYYY and the time in 24-hour format. Much better like this.

Makes me think there is a problem of definitions: what is covered by the Language settings and what is covered by the Region settings. Linux, Mint, Cinnamon, or Thunderbird problem?

Comment 122

a month ago

(In reply to Clément Lefèvre from comment #120)

Well, I'm not a Linux user, but I have seen felt 100 bugs on the issue and read a million comments.

We have Thunderbird developers in Finland and Slovakia who use a local locale and get localised display. If you use the en-US locale for regional display, all apps should should 12h AM/PM. If, unlike Windows, Linux doesn't allow the locale to be configured, why don't you used a less "mental exercise"-prone locale like en-GB. Also, please read the bugs quoted in comment #2. As I understand it, this bug here is about LC_TIME=en_DK.utf8 and nothing much else.

Comment 123

a month ago

(In reply to henri.passerieux from comment #121)

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.

I still don't understand why you don't set absolutely everything to French. Then TB would do "regional settings locale French (France)". Is that not what you want?

At least on Windows, TB follows format for a selected region, I don't know that the regions per se is? You mean location?

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting. And I do want to keep the system language as English. Now it's English UK and I'm fine with that. Next time I know to select that already when installing.

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting

We do some special customizations for the old Unity from Ubuntu. I would accept a patch that extends that to cover Mint preferences if anyone is willing to write a patch :)

Comment 126

3 days ago

Any chance you consider Jeffs patch? Being able to define one's own date-format string would be very helpful.

not directly, no. The patch would need generalization (for example, it provides a single pattern, while we need patterns per style - narrow, short, medium, long, full etc.)

I'd also like to see how it fits into the conversation in https://github.com/tc39/ecma402/issues/190

To sum it up - in principle, I'm not against extending our platform capability to fine-tune intl settings, and I recognize that date/time patterns are particularly sensitive area to customization.
But I don't think we should be solving it on the Gecko level by writing shortcuts through the system to satisfy some particular UI widget.

To make it actionable: If someone is willing to write a patch which provides hardcoded custom patterns for date, time and date-time styles (narrow/short/medium/long/full).

I imagine such API to lead to us being able to add UI similar to what MacOS and Windows do [0].

It should likely be a new API in intl/locale, named maybe I18nSettings, which would manage setting and retrieving custom overrides for date/time patterns (and maybe first day of the week, like Windows does?) if they're set, or use data from locale / os preferences when not.

Happy to mentor, but I won't have time to work on that anytime soon.

[0] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

Comment 128

2 days ago

Bibi wrote in Comment #24

  • we will match the locale that your OS returns, but we'll take the values
    from CLDR since we know we have all the tables for each locale that we need.

(Just to clarify - by "locale" do you mean the locale-name ?)
But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

In particular, I would be very happy to use the CLDR "root",
see http://demo.icu-project.org/icu-bin/locexp?_=root ,
as its dates are in order Y/M/D and the times are 24-hr.
Why not simply let us specify this name, "root", directly as a preference?
If a name is specified and it is not a valid CLDR locale, then you
can use the current procedure as a fall-back.

(I am asking this because I have not been able to figure out how to
make "root" a valid POSIX locale name on my system - Linux Mint.
The recipe in Comment #59 didn't work for me.)

Comment 129

14 hours ago

(In reply to Carlos O'Donell from comment #107)

Easy for anyone. I'll show you. It was designed this way to be easy.

Carlos, I'm trying to follow your instructions but for me they are anything but "easy":

dnf install glibc-locale-source

I think this is highly distribution-dependent. I have no idea what dnf is but it doesn't exist on my system (openSUSE). I'm guessing that this command is supposed to invoke some package manager that installs the glibc locale sources. The source package turns out to be named something completely different on openSUSE -- it's called glibc-i18ndata.

vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

For me, the value of d_fmt in en_US is "<U0025><U006D><U002F><U0025><U0064><U002F><U0025><U0059>", which is completely opaque. All the other variables in this file, and every other file I've examined, are defined with similar escape sequences. Most files don't have explanatory comments (not even ones that simply unescape the values). It's not obvious whether I'm even allowed to replace an escaped value with an unescaped one, nor how to easily automatically convert an unescaped string to an escaped one. (I'm sure as hell not going to look up every single character in a Unicode table so that I can type it in manually, and writing a program to do so seems to be about as much effort.) In fact, a less technically sophisticated user wouldn't even understand the fact that the values are escaped.

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/en_US@ISO8601.utf8

You're done. This is impossible to do with ICU/CLDR.

Well, I tried copying and pasting some existing key-value pairs from other locale files to build a new one, en_CA@ISO8601. The localedef command seemed to compile my new locale, but (for reasons probably mentioned elsewhere in this conversation) Thunderbird doesn't seem to want to use it. When I run it with LC_TIME=en_CA@ISO8601.utf8 and/or LANG=en_CA@ISO8601.utf8, all the dates and times in the mail list show up in the usual en_CA format (i.e., with YYYY-MM-DD dates but hh:mm a.m./p.m. times).

You need to log in before you can comment on or make changes to this bug.