1426907 - Implement pref overrides for date and time formats: intl.date_time.pattern_override.date_* and intl.date_time.pattern_override.time_* (was: TB 60 on Linux: Setting date locale to LC_TIME=en_DK.utf8 no longer outputs yyyy-MM-dd format)

Status: RESOLVED FIXED

(Core :: Internationalization, enhancement)

Description

[Niklas Hambüchen]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20171221100108

Steps to reproduce:

In Thunderbird 52 on Ubuntu 16.04 I can change the date format to `YYYY-MM-DD` by running `env LC_TIME=en_DK.utf8 thunderbird`.

In Thunderbird 58b2 this no longer works.


Actual results:

Thunderbird shows the Date column as DD/MM/YYYY.


Expected results:

Thunderbird shows the Date column as YYYY-MM-YY, as TB 52 did.

Comment 1

[Niklas Hambüchen]

Found via https://askubuntu.com/questions/214099/how-to-change-thunderbirds-display-time-to-24h-format#comment1593436_217718

Comment 2

[Jorg K (CEST = GMT+2)]

We had a felt million bugs regarding the date/time display format changes that have occurred on Mozilla core, Firefox and Thunderbird during 2017, actually, all this started to the day one year ago on Christmas Eve 2016.

In Thunderbird you have a preference you can set under Tools > Options, Advanced, General (might we elsewhere on the Window on Linux).

If this doesn't work please, browse some of the other bugs, starting with bug 1344594 (and duplicates) then moving to bug 1389972 and then bug 1389369. You will find suggestions that will fix your issue.

Comment 3

[Niklas Hambüchen]

(In reply to Jorg K (GMT+1, mostly AFK 22-27 Dec., bustage-fix only) from comment #2)
> In Thunderbird you have a preference you can set under Tools > Options,
> Advanced, General (might we elsewhere on the Window on Linux).

I do have an option there to choose between "Application locale: English (United States)" and "Regional settings locale: English (Denmark)" on Linux, but neither has any effect on the dates being displayed.

> If this doesn't work please, browse some of the other bugs, starting with
> bug 1344594 (and duplicates) then moving to bug 1389972 and then bug
> 1389369. You will find suggestions that will fix your issue.

@jorgk: All of the issues linked are closed.

Bug 1366134 is open but there and in all other places it says that switching from _US to _DK should work as long as both locales start with en_ and Thunderbird is an en_ build.

Also, on that issue, Zibi recommended for a user who reported that this doesn't work as advertised that they should open a new bug with their specific issue.

So I'm reopening this for now, unless we can identify an appropriate open issue to set this as a duplicate for.

Zibi, could you confirm whether this is known to be currently broken in TB 58 beta and if so, which issue should this be a duplicate for?

Comment 4

[Jorg K (CEST = GMT+2)]

Regional settings locale: English (Denmark) should have worked after a restart of TB. Bug 1389369 comment #7 doesn't help?

Comment 5

[Niklas Hambüchen]

For me it doesn't work, no matter how often I restart TB.

How should https://bugzilla.mozilla.org/show_bug.cgi?id=1389369#c7 help, isn't that only about 12h vs 24h? My issue is about the entire date format, e.g. changing to YYYY-MM-DD format.

Comment 6

[Zibi Braniecki [:zbraniecki][:gandalf]]

I tested the LC_TIME on Linux/Gnome and it seems to me that if I switch the `Formats` setting in `Region & Language` in Gnome Settings, Thunderbird (Nightly) follows along nicely.

My suspicious is that multiprocess somehow makes us read the session LC_TIME rather than command line LC_TIME and that's why your command doesn't work.

Can you try to change your LC_TIME for the whole session?

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #6)
> ...
> My suspicious is that multiprocess somehow makes us read the session LC_TIME
> rather than command line LC_TIME and that's why your command doesn't work.
> 
> Can you try to change your LC_TIME for the whole session?

Niklas?

Comment 8

[Niklas Hambüchen]

I've tried again with Thunderbird Daily (thunderbird-62.0a1.en-US.linux-x86_64.tar.bz), and the Beta 60.

I still cannot get Thunderbird to show the '2018-05-23 15:57:07' format.

I believe I've tried any combination of

* Changing the system locale via `unity-control-center` in the "Regional Formats" tab of gnome's "Language Support" settings and clicking "Apply system-wide" (`locale` shows `LC_TIME=en_DK.UTF-8` right after login)
* Setting LC_TIME=en_DK.UTF-8 manually in my .xinitrc so that my Window manager (i3) reads it on startup
* Thunderbird's "Date and Time Formatting" setting

Thunderbird keeps showing the '23/05/2018, 10.32' format.

Meanwhile `date +'%x %X'` shows the format I desire as expected.

Comment 9

[Niklas Hambüchen]

Just for clarity, I can also see LC_TIME=en_DK.UTF-8 in `tr '\0' '\n' < /proc/$(pidof thunderbird)/environ | grep LC`.

So at least in theory, Thunderbird should be able to see it.

Comment 10

[Jorg K (CEST = GMT+2)]

Zibi, can you help? Even Magnus was happy in the end, see bug 1389972 comment #5. Nothing ever got implemented in bug 1337069, right?

Niklas, which locales are shown in TB's Advanced/General settings under "Date and Time" Formatting?

Comment 11

[Zibi Braniecki [:zbraniecki][:gandalf]]

Is there an equivalent of firefox about:support? I'd like Niklas to dump the output of the Intl/L10n section of it.

Also, Niklas - does it work if you change other LC_*? We may not recognize the difference between LC_ALL, LC_DATE and LC_TIME yet.

Comment 12

[Jorg K (CEST = GMT+2)]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #11)
> Is there an equivalent of firefox about:support? I'd like Niklas to dump the
> output of the Intl/L10n section of it.
Yes, "Help > Troubleshooting Information". It's 99% of what FF offers, at least from TB 60 up and trunk.

Personally I see:
Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["de"]
App Locales 	["en-US"]
Regional Preferences 	["en-GB"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-GB"]

I have a German language pack installed (but not using it), hence the "de". Otherwise I'm using "en-GB". Anyway, I don't have a problem, the reporter has.

Comment 13

[Niklas Hambüchen]

Thanks. I get this in "Help > Troubleshooting Information", when using LC_TIME=en_DK.utf-8:

 Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US"]
Regional Preferences 	["en-DK"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-GB"]
Regional Preferences 	["en-DK"]

And when also setting LC_ALL=en_DK.utf-8, I get no differences in the UI and these settings:

 Application Settings
Requested Locales   ["en-US"]
Available Locales   ["en-US"]
App Locales   ["en-US"]
Regional Preferences  ["en-DK"]
Default Locale  "en-US"
Operating System
System Locales  ["en-DK"]
Regional Preferences  ["en-DK"]

@Zibi: Is LC_DATE really a thing? In any case, if I set it, nothing changes.

> which locales are shown in TB's Advanced/General settings under "Date and Time" Formatting?

"English (United States)" and "English (Denmark)" are the two options with the above last setting.

I don't even understand what format the '23/05/2018, 10.32' is. As far as I know, neither en-US nor en-DK format times with a '.'.

Comment 14

[Zibi Braniecki [:zbraniecki][:gandalf]]

Hi Niklas,

The reason you see "23/05/2018" is because Unicode specifies that format as the default for dates in `en-DK`.

You can try it in any modern browser:

```
(new Date()).toLocaleString("en-DK");
```

It'll format your date using `en-DK` locale format specified in Unicode CLDR database.

Comment 15

[Ben]

Hello Zibi,

You're right. toLocaleString("en-DK") uses a different format than LC_TIME="en_GB.utf8" used to do for Thunderbird.

In Gnome's Region & Language settings, I have Language set to "English (United Kingdom)" and Formats to "Denmark" and Gnome gives me:

- Dates: 2018-08-14
- Times: 11:14:58
- Dates & Times: 2018-08-14T11:14:58 CEST

And I cannot get that anymore in Thunderbird. So, I don't know what Thunderbird is doing, but neither of these two options do what I want:

- Application locale: und
- Regional settings locale: English (Denmark)

You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

If you know some other way to use ISO formatting in Thunderbird, please let us know! ;-)

Comment 16

[Zibi Braniecki [:zbraniecki][:gandalf]]

I understand the confusion.

It seems to me that Gnome/POSIX is presenting different date format for en-DK than ICU/CLDR is. 
CLDR stores the default en-DK formats here: https://github.com/unicode-cldr/cldr-dates-modern/blob/master/main/en-DK/ca-generic.json#L321 - as you can see they use `dd/MM/y GGGGG` which translates to `23/05/2018` that you see.

My guess is that POSIX just doesn't know what to do with en-DK and it throws ISO format instead.

> You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

Well, it does. It just does it differently than you would expect. Unfortunately, I don't see an easy solution here, since we do not handle a local overlay on CLDR data, we do not have an easy way to add a custom overlay for en-DK.

What you could do is file an CLDR ticket - https://unicode.org/cldr/trac/ - and request en-DK locale date time format to be changed to what you believe to be the good default for en-DK.

Since no CLDR locale, as far as I know, uses ISO format, I can't even recommend setting any locale manually to Regional Preferences Locales in Gecko, because none will result in the ISO format. CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.

What I might recommend if you want your localization to be in English, you can pick one of other en-* variants that suits you better. You can see the list here: https://github.com/unicode-cldr/cldr-dates-modern/tree/master/main

Comment 17

[Ben]

Hello Zibi, Thank you for your response.

>> You may have an explanation with the toLocaleString behaviour, but Thunderbird is no longer using regional settings anymore!

> Well, it does. It just does it differently than you would expect.

What I meant is that there is no way to tell Thunderbird to use the system settings anymore; I consider that a major issue. The option "Regional settings locale" does not do on my Gnome system what it suggests it does, namely using my (system) regional setting.

> ... request en-DK locale date time format to be changed to what you believe to be the good default for en-DK.

Hmmm. You are aware that English is not spoken in Denmark (as a first language) and that en-DK is only used to get ISO format while using English?

> CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.

This is not a discussion about taste. If it were, there would be no need for localisation at all, just one format for everyone! ;-)

As far as I understand, there is a fair amount of people that were using en-DK to get the ISO format, and Thunderbird does not offer that in the latest version. I have no information about other issues that people may have found with this latest change, but if ISO format is the one that people miss, I suggest to add a third option in Thunderbird for ISO formats. Especially since there is no CLDR option to get ISO format, as you were aware of. And even if there were one, it is probably not one that works the same in POSIX.

Comment 18

[Zibi Braniecki [:zbraniecki][:gandalf]]

> What I meant is that there is no way to tell Thunderbird to use the system settings anymore; I consider that a major issue. 

That's a major change introduced post Gecko 52 - we do not use OS intl APIs anymore. There are many important reasons for a multiplatform engine to not do that.

> The option "Regional settings locale" does not do on my Gnome system what it suggests it does, namely using my (system) regional setting.

That's not true. It does use your regional settings *locale*. It just resolves different values for that regional settings locale. It uses values from Unicode CLDR which is a really good database used by Windows, MacOS, Android, Chrome and others. That means that the values you will get are the same (or very similar) to ones you will likely encounter in most other modern software.

> Hmmm. You are aware that English is not spoken in Denmark (as a first language) and that en-DK is only used to get ISO format while using English?

That's not true. `en-DK` is an established partial locale which overlays root locale `en` in Unicode with custom internationalization settings. If you believe that `en-DK`'s date format should be ISO, I recommend you report it to CLDR under the link I provided.

That decision will be made with involvement from intl experts from all organizations participating in the Unicode project and naturally is going to be opinionated in scenarios where multiple defaults are possible.

> As far as I understand, there is a fair amount of people that were using en-DK to get the ISO format, and Thunderbird does not offer that in the latest version.

Yes, that's correct. Thunderbird is using Gecko platform which moved away from using OS APIs for retrieving intl information. That has many major positive consequences, but also may in edge cases like this, lead to unwanted changes as well.

> I suggest to add a third option in Thunderbird for ISO formats. 

That's perfectly possible and I would likely accept a patch to mozIntl/OSPreferences to allow for that, but I will not be able to prioritize that anytime soon to work on it myself.

Comment 19

[Jonas Witschel]

> That's not true. It does use your regional settings *locale*. It just resolves different values for that regional settings locale. It uses values from Unicode CLDR which is a really good database used by Windows, MacOS, Android, Chrome and others. That means that the values you will get are the same (or very similar) to ones you will likely encounter in most other modern software.
I would agree if this behaviour was consistent for all platforms supported by Thunderbird. However this is not true: on Windows, you can change the date and time format (using Control Panel>Clock, Language, and Region>Change date, time, or numbers formats) e.g. to use ISO 8601, and this setting will be honoured by Thunderbird 60! I am not completely sure, but I guess the code responsible for that is https://dxr.mozilla.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#132. On Linux on the other hand, Thunderbird seems to only read the locale *name* from LC_TIME and then uses the CLDR database to construct the date format, ignoring the format specified in the system locale (https://dxr.mozilla.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#148). This seems to be an inconsistency across platforms that is likely to cause confusion.

> Since no CLDR locale, as far as I know, uses ISO format, I can't even recommend setting any locale manually to Regional Preferences Locales in Gecko, because none will result in the ISO format. CLDR just doesn't think ISO is a good localized format, and I think I might agree with them.
Well, there is en-SE (https://github.com/unicode-cldr/cldr-dates-modern/blob/master/main/en-SE/ca-generic.json#L321), which is probably the closest you can get to ISO 8601 with Unicode CLDR: in contrast to ISO 8601, the separator between date and time is a comma, but other than that it follows the specification. Unfortunately, this is not commonly included in Linux distributions as it is not part of the standard GLIBC set of locales, but you can create a symlink from /usr/share/i18n/locales/en_DK to /usr/share/i18n/locales/en_SE, run locale-gen and set LC_TIME=en_SE.UTF-8 to achieve the desired effect. (Just setting LC_TIME to use this locale without actually generating it does not seem to work as Thunderbird only issues the GTK warning "Locale not supported by C library" and ignores the missing locale.)

Comment 20

[Zibi Braniecki [:zbraniecki][:gandalf]]

> However this is not true: on Windows, you can change the date and time format (using Control Panel>Clock, Language, and Region>Change date, time, or numbers formats) e.g. to use ISO 8601, and this setting will be honoured by Thunderbird 60! I am not completely sure, but I guess the code responsible for that is https://dxr.mozilla.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#132. On Linux on the other hand, Thunderbird seems to only read the locale *name* from LC_TIME and then uses the CLDR database to construct the date format, ignoring the format specified in the system locale (https://dxr.mozilla.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#148). This seems to be an inconsistency across platforms that is likely to cause confusion.

That's a great read of the situation!

Yes, we do more on Windows, than we do on Linux. The reason being that Windows provides UI and API for us to read customized date/time patterns for styles short/medium/long.

Which means that we can fetch the *pattern* from the OS, and *localize* it using our own CLDR resources. This means, that if on Windows you'll tell us you want a date "{day} {month} {year}", and in Gecko you'll specify you want Polish language we'll present you a date "25 sierpnia 2018", which will fit properly into your UI localization.

On the other hand on Linux, we do not have a way to retrieve a pattern, that we could then localize.

This method could not be used for our public Intl API (the ECMA402 one exposed to the Web), and since our platform is converging around WebAPIs, we switched our toolkit's Intl API to use the same logic as ECMA402 API, which requires us to remain consistent across platforms.

We do extend it tho, to some degree, by, among others, allowing for the OS to provide us a pattern that the user customized to. If Linux will one day (or even Gnome itself :)), start exposing similar Regional Preferences UI to allow users to customize their patterns, we'll do the same thing as we do on Windows/MacOS.

Comment 21

[Jonas Witschel]

> Yes, we do more on Windows, than we do on Linux. The reason being that Windows provides UI and API for us to read customized date/time patterns for styles short/medium/long.
> 
> Which means that we can fetch the *pattern* from the OS, and *localize* it using our own CLDR resources. This means, that if on Windows you'll tell us you want a date "{day} {month} {year}", and in Gecko you'll specify you want Polish language we'll present you a date "25 sierpnia 2018", which will fit properly into your UI localization.
> 
> On the other hand on Linux, we do not have a way to retrieve a pattern, that we could then localize.
You can retrieve the date and time patterns for the current locale using the POSIX function nl_langinfo() (http://pubs.opengroup.org/onlinepubs/9699919799/functions/nl_langinfo.html): a call like "nl_langinfo(D_FMT)" (or "locale date_fmt" in a shell) returns a format string for strftime like "%a %b %e %H:%M:%S %Z %Y". The supported locale elements are described in http://man7.org/linux/man-pages/man3/nl_langinfo.3.html and http://man7.org/linux/man-pages/man5/locale.5.html.

Unfortunately, I can see that this might not be a nice fit for the CLDR format you require: instead of providing multiple versions of the date and time patterns (short/medium/long/full), there is only one version for date (D_FMT), time (T_FMT) and date and time combined (D_T_FMT). I think the only (ugly) solution here would be to map the single pattern provided by the operating system to all four CLDR versions: there simply seems to be no standard way to specify locale information on Linux other than what is defined by the POSIX specification, which offers much less flexibility than the more recent CLDR specifications.

Comment 22

[Zibi Braniecki [:zbraniecki][:gandalf]]

> You can retrieve the date and time patterns for the current locale using the POSIX function nl_langinfo() 

This returns the default value, not user-customized. The only reason we allow fetching into the OS is to respect user manual customizations in OSes that allow users to customize.

Retrieving a default pattern for a given locale that is different from a default pattern for the same locale that we have in CLDR would introduce internal inconsistency for much larger population (currently, there still is an inconsistency risk between UI that uses Intl API and mozIntl API which extends to fetch into the OS, but it affects only users who manually customized their format).

The further limitations of the POSIX model you listed is just another vector for which I'd prefer not to do that. It's much easier to take the pattern from Windows/MacOS which at worst use different CLDR version, than from POSIX which uses completely different database.

Comment 23

[Jonas Witschel]

> This returns the default value, not user-customized. The only reason we allow fetching into the OS is to respect user manual customizations in OSes that allow users to customize.
For Linux, customisation of the locale means either modifying the file of the current locale or creating a new one (e.g. en_XX for Arch Linux: https://xyne.archlinux.ca/projects/locale-en_xx/). I completely understand the point of not supporting the first of these approaches to avoid inconsistencies. Not supporting the second approach (i.e. using the OS date format if the locale name cannot be matched to a valid CLDR locale) is a bit inconvenient with regard to customisations.

BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting? I would have thought "true" means "use the date format of the default Thunderbird locale", while "false" means "use the date format of the system locale [as specified by CLDR]". However Thunderbird seems to always use the date format from the locale in LC_TIME, regardless of that setting. Am I misunderstanding its function, or does it not apply to dates?

> The further limitations of the POSIX model you listed is just another vector for which I'd prefer not to do that. It's much easier to take the pattern from Windows/MacOS which at worst use different CLDR version, than from POSIX which uses completely different database.
FWIW, the people from GLIBC, current maintainers of the POSIX locales, are working on consolidating the POSIX and CLDR locale information: https://sourceware.org/ml/libc-locales/2016-q2/msg00246.html. There might be still some way to go though ;)

Comment 24

[Zibi Braniecki [:zbraniecki][:gandalf]]

> Not supporting the second approach (i.e. using the OS date format if the locale name cannot be matched to a valid CLDR locale) is a bit inconvenient with regard to customisations.

That would be quite hard to fit. If you create your own locale `en-XY`, and we're trying to negotiate the best locale for the app that has plural rule categories, localization resources in this locale, date/time and relative time format patterns and so on, then your custom locale `en-XY` would only work for one aspect of the whole process.

It also likely doesn't help that POSIX locales and BCP47 locales which we use extend differently :/

> BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting?

https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences

tl;dr - we will match the *locale* that your OS returns, but we'll take the values from CLDR since we know we have all the tables for each locale that we need.

> FWIW, the people from GLIBC, current maintainers of the POSIX locales, are working on consolidating the POSIX and CLDR locale information: https://sourceware.org/ml/libc-locales/2016-q2/msg00246.html.

That's an awesome news!

I expect *a lot* of problems glibc is facing to go away once they converge around CLDR. Mainly because CLDR is well maintained by localization and linguistic experts from all major IT companies in the World. It is a Wikipedia of intl data. glibc, well, glibc is a good effort from the 80ties.
With the recent move of Microsoft to CLDR, I hope that the move of glibc will complete the unification.

Comment 25

[Jonas Witschel]

> That would be quite hard to fit. If you create your own locale `en-XY`, and we're trying to negotiate the best locale for the app that has plural rule categories, localization resources in this locale, date/time and relative  time format patterns and so on, then your custom locale `en-XY` would only work for one aspect of the whole process.
I understand. Well, luckily in the case of ISO dates, using the en-SE locale for LC_TIME works great, even though it requires some manual work to create the POSIX locale first.

> > BTW, what is the purpose of the "intl.regional_prefs.use_os_locales" setting?
> 
> https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences
> 
> tl;dr - we will match the *locale* that your OS returns, but we'll take the
> values from CLDR since we know we have all the tables for each locale that
> we need.
Ah, so the setting only applies if the language of Thunderbird and the system locale (from LC_TIME) differ. That explains why I did not see any difference, as both were the same in my case (en-US for Thunderbird and en-DK in LC_TIME). Thank you for the link, that was very helpful!

Comment 26

[Marko Korhonen]

Thanks for the en_SE workaround! However, I still have a problem: after creating the en_SE locale and setting LC_TIME, I get the ISO date but I still have my time in 12h, f.ex 2018-08-23, 8:19 PM. Is there any way to change this?

Comment 27

[Zibi Braniecki [:zbraniecki][:gandalf]]

You're in a particular luck, I think! One thing we do handle, because Gnome handles it is hour-cycle!

If you have Gnome Preferences, just go to Date&Time and set your hour cycle to the preferred one. If not, you can use any dconf-editor to get to org.gnome.desktop.interface.clock-format and set it to 12|24.

Comment 28

[Marko Korhonen]

Thanks a lot! That worked a treat

Comment 29

[Niklas Hambüchen]

I tried the symlink workaround but couldn't get it working.

I ran (on Ubuntu 16.04):

    sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/en_SE
    sudo locale-gen

but it didn't print out a `en_SE.UTF-8...` and correspondingly I got `warning: setlocale: LC_TIME: cannot change locale (en_SE.utf-8): No such file or directory` at startup.

Did I miss a step?

Comment 30

[Jonas Witschel]

You also have to enable the new locale in /etc/locale.gen by adding "en_SE.UTF-8 UTF-8" as a new line (which I missed in my original description), so the complete command sequence would be something like:

sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/en_SE
echo 'en_SE.UTF-8 UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen

Comment 31

[Niklas Hambüchen]

Thank you very much, at last some relief :)

> I suggest to add a third option in Thunderbird for ISO formats. 

That sounds like a really good idea.

Comment 32

[Corey Hickey]

I ran into this bug with Thunderbird 60.

Using en_DK is a very long-standing recommendation for getting ISO 8601 dates:
http://kb.mozillazine.org/Date_display_format

It is disappointing to find this broken now. Using en_SE as described above is at least an improvement, but I would definitely like a definitive option in the preferences to use ISO date format.

Thanks,
Corey

Comment 33

[Lorenzo]

Any update on this?

With the TB 60 update and 50% of my add-ons not working I also lost the Super Date Format one which I was using to work around this [https://addons.thunderbird.net/en-US/thunderbird/addon/super-date-format/]

My information is as follows (Thunderbird 60.0 packaged from Manjaro):

Internationalization & Localization
Application Settings
Requested Locales 	["en-GB","en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["en-US","und"]
Default Locale 	"und"
Operating System
System Locales 	["en-GB"]
Regional Preferences 	["it-IT"]

Yet my system locale environment variables are:


LC_TIME=it_IT.UTF-8
LANG=en_GB.utf8

Thanks

Comment 34

[Eligijus]

There is no way to change date/time into ISO format after update to Thunderbird 60.2.1 on Linux Mint OS.

Advanced > General > Date and Time Formating don't affect anything.

Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["en-US"]
Default Locale 	"und"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-US"]


Also add-on "Super Date Formata" is now incompatible, and add-on "ConfigDate" settings don't affect anything.

Comment 35

[Magnus Melin [:mkmelin]]

Please see whiteboard and previous comments in this bug.

Comment 36

[Bart]

LC_TIME is set to 'nl_NL.UTF-8' for me.

Calendar: works after setting 'Date and Time formatting' in Advanced Preferences.
E-mail: no luck. LC_TIME is not respected.

org.gnome.desktop.interface.clock-format is set to '24h'


Internationalization & Localization

Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["nl-NL"]
Default Locale 	"und"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["nl-NL"]

Comment 37

[Daniel AJ Sokolov]

I can confirm this bug on Thunderbird 60.2.1 (64-bit) on a Linux Mint 18.3 machine.

My system clock is 24 hours mode, but Thunderbird now has a confusing date that starts with the year and ends in am or pm - whether I prefer to use the application locale or the OS locale. And I can't find how to change the application locale in Thunderbird.

Comment 38

[john]

for those experiencing issues on gnu/linux

test the mozilla release (instead of the one from your distro):

https://ftp.mozilla.org/pub/thunderbird/releases/60.0/linux-x86_64/

(or wget https://ftp.mozilla.org/pub/thunderbird/releases/latest/README.txt)

tar xvf thunderbird-60.0.tar.bz2
cd thunderbird/
./thunderbird-bin

to instead start it from your applications' list:

## to run do   bash this-script.sh
## assuming thunderbird/ is extracted in ~/Downloads/
  for i in 16x16 22x22 24x24 32x32 48x48 64x64 128x128 256x256; do
    mkdir -pv ~/.icons/hicolor/$i/apps/
    cp -v ~/Downloads/thunderbird/chrome/icons/default/default${i/x*}.png \
              ~/.icons/hicolor/$i/apps/thunderbird.png
  done

Comment 39

[Andreas]

I am using Thunderbird 60 with Debian Stretch & KDE5, and for me any locale settings for emails does not work anymore - I tried several workarounds without success.

My whole system is set to de_CH, the the Calendar works fine with date formats, but not emails, which is a pain in the neck.

This should be a MAJOR issue as it reduces functionality for anybody non-US immensely. At least there should be some way to "overwrite" the date format in the settings if Thunderbird is not able to correctly ascertain the correct system locale and format. Because the current option (using "Regional settings locale") does absolutely nothing.

Comment 40

[Zibi Braniecki [:zbraniecki][:gandalf]]

Can you paste the `Internationalization & Localization` results from about:support ?

The date/time formatting should follow OS date/time if the language between OS and Thunderbird matches.

> This should be a MAJOR issue as it reduces functionality for anybody non-US immensely.

I'm not sure if you're correct. Majority of the users do use date/time format that matches their app's locale. I'm not trying to shy away from identifying the cause and fixing it, just putting things in perspective.

Comment 41

[Andreas]

These are the results:

Application Settings
Requested Locales 	["de-CH","en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US","und"]
Regional Preferences 	["de-CH"]
Default Locale 	"und"
Operating System
System Locales 	["de-CH"]
Regional Preferences 	["de-CH"]

Comment 42

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thank you!

The entry "Application Settings > Regional Preferences 	["de-CH"]" indicates to me that Gecko uses `de-CH` to format date and time in your app.

If that's not the case, we have a bug. Can you provide a screenshot of a place in the UI where the date/time is formatted differently than when you do `new Date(2018, 10, 22, 15, 23).toLocaleString("de-CH")` (which, in my case shows "22.11.2018, 15:23:00")? 

What OS are you on?

Comment 43

[Andreas]

I am on Debian Stretch with KDE 5. Didn't find a way to include a screenshot here, but you can look at it here:

https://dl84.panaxis.ch/screenshot_thunderbird.png

Basically, it is everywhere, except for the Calendar - there the date is formatted correctly.

Comment 44

[Ken Fallon]

> > This should be a MAJOR issue as it reduces functionality for anybody non-US immensely.
> 
> I'm not sure if you're correct. Majority of the users do use date/time
> format that matches their app's locale. 

They do so because users have no option - hence this bug report :)

Unfortunately Linux chooses to use rfc3066[1] for Locales. This represents two completely different things as one value. IE: What LANGUAGE you speak, combined with a field separator of underscore, and then what REGION you are in. 

The first part has 487 languages, is defined by ISO 639[2], and is maintained by the International Information Centre for Terminology (Infoterm). 

The second part has 193 countries, is defined by ISO_3166-1[3], and is maintained by either United Nations Terminology Bulletin Country Names, or Country and Region Codes for Statistical Use of the UN Statistics Division.

Treating them as the same value would result in needing 93,991 entries to be complete, and the addition of one new region brings in an additional 487 entries. 

There are lots of reasons why people would want to use values not provided by the default Locales. As an example, the country of Ireland only gives the option of English and Irish languages, while the 2011 national census shows a total of 182 different languages in use. It continues to say "514,068 residents spoke a language other than Irish or English at home in 2011." Given ROI has a population of 4,792,500 this represents 10% of the install base that may wish to select a language other than Irish or English.[4]

The same extends to the Netherlands where we are restricted to Dutch, Limburgish, Low Saxon, and West Frisian. Despite 22.61% of the residents been described as non Dutch. Even then there is a sizable population of Dutch speakers that prefer to install English operating systems.

When you jump a flight to somewhere you are by default keeping the first part and changing the second. Something as simple as been able to switch country to get the correct paper format in the printer would be great.

So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.

Please point me to anywhere else I can open a bug on this.

Comment 45

[Zibi Braniecki [:zbraniecki][:gandalf]]

> They do so because users have no option - hence this bug report

Umm, I don't think I can agree that all, or even majority, of users want a different date/time locale format than their selected product localization.

> So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.

We do. In particular, we separate our the language portion of the locale when we compare. And we normalize rfc3066 into bcp46. If you're interested in reading more, please take a look at https://firefox-source-docs.mozilla.org/intl/locale.html and the following chapter.

> https://dl84.panaxis.ch/screenshot_thunderbird.png

Jorg - can you take a look at what API call is used for the localization of the column in this screenshot? Based on the users support information, we should use `mozIntl.DateTimeFormat` with default locale which should be `de-CH` which should result in the date being formatted to `22.11.2018` while the screenshot shows `10/24/2018`. I suspect that we either use the generic `Intl.DateTimeFormat` or somehow mess up locale selection (by passing it to the constructor rather than leaving as undefined maybe?)

Comment 46

[Ken Fallon]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #45)
> > They do so because users have no option - hence this bug report
> 
> Umm, I don't think I can agree that all, or even majority, of users want a
> different date/time locale format than their selected product localization.
> 
I also can't agree but is it possible to get a breakdown of how may Windows/MAC users have non default Locales ? I would love to see some actual data for regions outside of the US.

> > So to all developers out there YES use Locales, but split them into language and region, and only ever use them as the sane defaults.
> 
> We do. In particular, we separate our the language portion of the locale
> when we compare.

Unfortunately even if you do, I can't give them to you as I am limited by the installed Locales.

Comment 47

[Jorg K (CEST = GMT+2)]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #45)
> Jorg - can you take a look at what API call is used for the localization of
> the column in this screenshot?
Sure, the C++ API, code here:
https://searchfox.org/comm-central/search?q=FormatPRTime&case=false&regexp=false&path=mailnews%2F
First hit, mailnews/base/src/nsMsgDBView.cpp
672 rv = mozilla::DateTimeFormat::FormatPRTime(dateFormat,

Comment 48

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thanks!

So that would indicate that for some reason this code https://searchfox.org/comm-central/source/mozilla/intl/locale/DateTimeFormat.cpp#94-97 does not do its job.

What it should do is hit: https://searchfox.org/comm-central/source/mozilla/intl/locale/gtk/OSPreferences_gtk.cpp#148 which in turn should hit https://searchfox.org/comm-central/source/mozilla/intl/locale/OSPreferences.cpp#171 and then modify the hourCycle based on data from GTK (talking about linux path).

Jorg - can you reproduce it and sprinkle printfs to see where it fails?

I'm wondering if the `mLocale` somehow doesn't end up being the `LocaleService::GetRegionalPreferencesLocale` here.

Comment 49

[Jorg K (CEST = GMT+2)]

I'm afraid not since I'm on Bill's own OS Windows ;-) - Maybe you can get Aceman to help.

Comment 50

[Andreas]

The newest 60 version (60.2.1) which just became available to me on Debian resolved the issue, I guess there was a generic issue already which also fixed my problem - thanks for your help!

Comment 51

[Zibi Braniecki [:zbraniecki][:gandalf]]

Interesting. Anyone can give it a try and confirm if it's fixed for them? Maybe we're chasing ghosts.

Comment 52

[Lorenzo]

Hi,

I can confirm that in 60.2.1 x64 Linux (through Manjaro), the setting in 'Preferences > Advanced > Date and Time Formatting" is now honoured for emails (requires restart to take effect). The possible options are either English US or (I assume) the LC_TIME. In fact I have tested both with my system LC_TIME which is it_IT.UTF-8 as well as running from a terminal after changing LC_TIME to en_GB.UTF-8. Indeed this is also reported in the 'Help > Troubleshooting information' as the "Regional Preferences".

Hope this helps.
PS: I still believe that more customizability (as offered by the add-on cited above) would be nice, e.g. something similar to the date +FORMAT command in linux ;)

Comment 53

[sbastig]

This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.

Comment 54

[Zibi Braniecki [:zbraniecki][:gandalf]]

> This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.

Interesting, because we read LC_TIME - https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#44

Comment 55

[Jonas Witschel]

(In reply to sbastig from comment #53)
> This bug still persists on 60.2.1, thunderbird ignores LC_TIME if LC_ALL is set,however unsetting LC_ALL and LANG solves the problem.
That is not a bug, LC_ALL is supposed to work like this (http://man7.org/linux/man-pages/man3/setlocale.3.html):
> [...] first (regardless of category), the environment variable LC_ALL is inspected, next the environment variable with the same name as the category [...], and finally the environment variable LANG.  The first existing environment variable is used.

Generally, LC_ALL should not be used at all, it is more of a "last resort" solution.

Comment 56

[sbastig]

I did not actively set LC_ALL, it seems that some other program has set it. Possibly it was set by `dpkg-reconfigure locales`, but I am not certain. Anyway, I have removed it now and then LC_TIME started to take effect.

Comment 57

[Link @ QPG]

I've been using the aforementioned en_SE.utf8 workaround via a modified desktop file. After creating the en_SE locale, I essentially copied /usr/share/applications/thunderbird.desktop to ~/.local/share/applications/thunderbird.desktop and then made the following replacement:

s/Exec=thunderbird/Exec=env LC_TIME=en_SE.utf8 thunderbird/

This has lead to a somewhat usable interim fix, but the root of the problem appears to be caused by Unicode CLDR treating en_DK as if it were intended literally, rather acknowledging its origin as a workaround to bring ISO-8601 time formatting to English locales.

At the end of the day, I don't care how it happens, but I need ISO-8601 time formatting (YYYY-MM-DD dates and 24 hour time) in conjunction with the English language for the United States.

I also find it strange that my Thunderbird Preferences/Advanced/Date and Time Formatting show the Application locale as English (United Kingdom) rather than English (United States), but that is not really relevant to the issue of the en_DK ISO-8601 workaround breaking after the upgrade to Thunderbird 60.

Comment 58

[Einhard Leichtfuß]

Just a small addition to the workaround using en_SE:

There is also the `root' locale in the CLDR, which does not insert a comma between date and time.  Like `en-SE', it is not part of the glibc locale set, hence requires the same fiddling.

Comment 59

[Jethro Beekman]

Thanks Einhard, that works perfectly. To clarify, the full workaround is:

sudo ln -s /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/root
echo 'root.UTF-8 UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen

And then set LC_TIME=root.UTF-8

Comment 60

[jflorian]

Any help here for a Fedora user?  I can't use the custom locale as described -- I don't have a locale.gen file, at minimum.  I'm using KDE/Plasma and have most all locale settings there set to United States - American English (en_US), except for Time which I set to Sweden - English (en_SE) (where they do time right apparently!).  The samples that Plasma's System Settings shows for the various formats look perfect.  In bash, I see:

$ locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_SE.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

That looks good.  In TB, however, date/times are still shown like 10/31/18, 9:38 PM.  I've switched Edit > Preferences > Advanced > General > Date & Time Formatting to both "Application locale: English (UK)" and "Regional Settings: English (US)" and in both cases I see the exact same results (like 10/31/18, 9:38 PM).

Comment 61

[Marcin Zajaczkowski]

@jflorian, I had a problem with en_SE in Fedora as "not supported by C library" and I ended up with "lt_LT" which has a proper short date format. As days and others are not English I set it only for Thunderbird in the launcher file (as mentioned above in the comment #57):
> Exec=env LC_TIME=lt_LT.UTF-8 thunderbird %u

The short date is printed as with en_DE before: 2018-11-04 01:57. The day names in Lightning don't seem to be affected and I haven't encountered any visible side effects until now.

Comment 62

[localebug]

I can confirm the bug in TB 60.2.1 on Kubuntu 18.04.1 with KDE 5.12.6.

$ locale                                                                                                                                                                                                                       
locale: Cannot set LC_CTYPE to default locale: No such file or directory                                                                                                                                                                     
locale: Cannot set LC_MESSAGES to default locale: No such file or directory                                                                                                                                                                  
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_DE.UTF-8
LANGUAGE=en_DE.UTF-8
LC_CTYPE="en_DE.UTF-8"
LC_NUMERIC="en_DE.UTF-8"
LC_TIME="en_DE.UTF-8"
LC_COLLATE="en_DE.UTF-8"
LC_MONETARY="en_DE.UTF-8"
LC_MESSAGES="en_DE.UTF-8"
LC_PAPER="en_DE.UTF-8"
LC_NAME="en_DE.UTF-8"
LC_ADDRESS="en_DE.UTF-8"
LC_TELEPHONE="en_DE.UTF-8"
LC_MEASUREMENT="en_DE.UTF-8"
LC_IDENTIFICATION="en_DE.UTF-8"
LC_ALL=en_DE.UTF-8

Edit > Preferences > Advanced > General shows both the "Application locale" and the "Regional settings locale" as "English (United States)".


I consider this a high-priority issue; TB and the Lightning calendar are the cornerstone of my job and the date and time formats are very confusing to me. I would also suggest that, as a fallback, at least rely on ISO-8601 time formatting. (I'm assuming en_US is the standard fallback as nothing on my system should indicate en_US to TB.)

Comment 63

[Magnus Melin [:mkmelin]]

Looks to me you haven't set up the locale. Probably need something like

  locale-gen en_DE.UTF-8
  sudo pkg-reconfigure locales

Comment 64

[jflorian]

(In reply to Marcin Zajaczkowski from comment #61)

> The short date is printed as with en_DE before: 2018-11-04 01:57.

Confirmed.

> The day names in Lightning don't seem to be affected and I haven't encountered any
> visible side effects until now.

My calendar definitely has the day names affected, not at the top of the calendar grid, but in the "Events in the Next 14 days" area shown at the top of my calendar tab along with the Events sidebar on my mail tab.  There I see things like "2018 m. lapkricio 8d., ketvirtadienis 13:00" (more or less; I couldn't copy/paste and there are lots of diacritics I'm not attempting).  This solution doesn't look like it'll work for me.  Thanks for the response though.

Comment 65

[Marcin Zajaczkowski]

(In reply to jflorian from comment #64)
> My calendar definitely has the day names affected, not at the top of the
> calendar grid, but in the "Events in the Next 14 days" area shown at the top
> of my calendar tab along with the Events sidebar on my mail tab.  There I
> see things like "2018 m. lapkricio 8d., ketvirtadienis 13:00" (more or less;

I'm not affected probably thanks to:
> Thunderbird preferences -> General -> Date Text Format -> Short: 2018-11-05
(which I was set by default in my 10 years+ profile or I changed it long time ago and I forgot about that)

It's just an another workaround. I would be happy having the old en_DK behavior back in Thunder 60+.

Comment 66

[Link @ QPG]

(In reply to Marcin Zajaczkowski from comment #65)
> I'm not affected probably thanks to:
> > Thunderbird preferences -> General -> Date Text Format -> Short: 2018-11-05
> (which I was set by default in my 10 years+ profile or I changed it long
> time ago and I forgot about that)
> 
> It's just an another workaround. I would be happy having the old en_DK
> behavior back in Thunder 60+.

My Thunderbird (thunderbird:amd64/xenial-security 1:60.2.1+build1-0ubuntu0.16.04.4 uptodate) has no Date Text Format settings under Preferences/General.

Comment 67

[Magnus Melin [:mkmelin]]

I think he means the (lightning) preference on Preferences | Calendar | General

Comment 68

[Marcin Zajaczkowski]

Yes, my bad.

> Thunderbird preferences -> Calendar -> General -> Date Text Format -> Short: 2018-11-05

Comment 69

[vladimir-csp]

I've just upgraded TB 52.9.1 -> 60.3.0 and was hit with this issue. I would like to put some weight on providing ISO date option, for the sake of sanity, I guess. 
I can not find anything related to CLDR/en_DK/ISO date situation, other than one alternative opinion (https://bugreports.qt.io/browse/QTBUG-59382?focusedCommentId=350361&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-350361). Does anyone know is CLDR was ever bugreported about this? searching '"en_dk" site:unicode.org/mail-arch' yields no results.

Comment 70

[localebug]

(In reply to Magnus Melin [:mkmelin] from comment #63)
> Looks to me you haven't set up the locale.

Thank you, you were right about this. Fixing that made another potential issue visible: Dates are now shown as DD.MM.YY although explicitly defined as DD.MM.YYYY in the new glibc locale (with: d_fmt   "%d.%m.%Y" ). But that's the same across KDE (date in taskbar, Dolphin etc.). Yet it seems inconsistent that TB would look for the glibc locale but then rely on another setting buried elsewhere in the system (and I don't know where KDE enforces DD.MM.YY).

Comment 71

[:aceman]

For me setting LC_TIME by itself also does nothing (not shown in TB prefs).
Only when I set LC_ALL then the en_DK is picked up and used (e.g. in message date display) and shown as English (Denmark) in TB preferences.

Unsetting LANG and LC_ALL also makes it work in TB, however then the 'locale' command shows strange output:
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME=en_DK.utf8
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

So that does not seem as safe, as one would want to have the others set to some locale. Maybe you need to set all of them explicitly?

Comment 72

[Jonathan Kamens]

Can someone help me understand if bug 1505435 is a duplicate of this one or a different issue?

In a nutshell, what I reported there is that on Ubuntu, `Intl.DateTimeFormat().resolvedOptions().locale` is insisting on using en-GB date formatting no matter what I do. I've set locale environment variables, about:config preferences, the "Date and Time Formatting" preference in the UI (which is set to "Regional settings locale: English (United States)"), none of it seems to help.

I have both en-GB and en-US locales installed because Ubuntu requires the en-GB package as a dependency of the en-US package, so you can't install en-US without also installing en-GB.

I have tried reading the code in DateTimeFormat.cpp to understand what is supposed to be happening, and frankly it is so convoluted that I found it impenetrable.

If someone here could perhaps explain to me what _should_ be happening, I'd be happy to run Thunderbird under a debugger and try to understand what _is_ happening and how it's broken. Am I gathering correctly that no one has yet been able to do that?

Comment 73

[Zibi Braniecki [:zbraniecki][:gandalf]]

> Can someone help me understand if bug 1505435 is a duplicate of this one or a different issue?

Different.

We have two "scenarios" in which we format Intl data (in this case, date/time) - we either do this as part of "privileged" code in the UI of the Gecko app, or as part of the "unprivileged" website.

Bug 1505435 is about what we do for websites. That's what the `Intl.DateTimeFormat().resolvedOptions().locale` do.

This bug is about what we do in the privileged code.

The difference between those two paths is not huge, but can be meaningful in edge cases. We use the same API, but we feed it with different data. For "chrome privileged" UI we use `mozIntl.DateTimeFormat()` which looks into several operating system environment variables and in result *should* format your date/time closer to your OS preferences.

For the web-exposed API we *do not* provide that information because it would mean leaking it and increasing fingerprinting.

I still don't understand why this bug exists as indicated in comment 48 and comment 54. As for bug 1505435 - for UI we should always use mozIntl, not Intl, if we can.

Comment 74

[gshollingsworth+mozilla]

I can confirm this behavior in TB 60.2.1 on Linux Mint 18 Cinnamon 3.0.7 64-bit Linux Kernel 4.4.0-134-generic.

user agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1
buildid: 20181010142946

In the prior version I was using, I had implemented the en_DK.UTF-8 workaround from the above mentioned Mozillazine article to have the date/time formated like the ISO date/time format.

I have congigured 24hr time in the desktop environment.

I have LC_ALL defined but set to null value.

I have LC_TIME=en_DK.UTF-8

All other LC_ variables are en_US.UTF-8.

Date format was ISO like in the prior version, it changed to the format as reported by the OP.

After reading this thread, I have tested both the en_SE.UTF-8 and root.UTF-8 locales for LC_TIME and can confirm both produce an ISO like date/time format en_SE.UTF-8 'YYYY-MM-DD, HH:mm' and root.UTF-8 'YYYY-MM-DD HH:mm'

I have chosen to use the LC_TIME=root.UTF-8 workaround since 1) it does not include the comma and 2) seems to be more generic/universal of a locale definition less tied to language or region. That better fits the intent of the ISO date/time format.

My view of the locale treatment in Linux vs. Windows & Mac; Linux does have different support for customization than either Windows or Mac, but it does support customization. I can chose a different locale for the LC_TIME than all the others, my expectation is that the format specified in the generated locale file is what I should see. I do not know about Macs, but I know Windows implemented this through the registry and presented a UI and API for the custom format. I could edit the registry to change it. In Linux I can find and load a locale definition file in /usr/share/i18n/locales, create my own, or modify an existing one. I could modify the en_US file to achieve the same goal. The format expected for date/time is the format specified in the file specified by LC_TIME. If I so chose I think I could define a custom date/time format not used anywhere else in the world and set LC_TIME to it, I would expect that format to be used. I have experimented in Windows before to do just that. I do not expect the format to come from some other reference that the system itself does not use. my 2 cents.

Comment 75

[James Ralston]

I am also seeing this problem, on Red Hat Enterprise Linux 7.6, which updated to thunderbird-60.3.0-1.el7_5.x86_64. (RHEL 7.5 had thunderbird-52.9.1-1.el7_5.x86_64.)

For the longest time, I have used the trick of setting LC_TIME to en_DK.UTF-8 in order to get ISO-8601 -style date and time specifications:

$ cat ~/.i18n
LC_TIME="en_DK.UTF-8"

$ env -u LC_TIME date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: Mon 12 Nov 2018 08:14:15 PM EST
locale date: 11/12/2018
locale time: 08:14:15 PM

$ env LC_TIME=en_DK.UTF-8 date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: 2018-11-12T20:14:39 EST
locale date: 2018-11-12
locale time: 20:14:39

If I instead set LC_TIME to "lt_LT.UTF-8" (per Martin's comment 61), in Thunderbird, the date/time is displayed in ISO-8601 format, even though lt_LT uses YYYY.MM.DD instead of YYYY-MM-DD:

$ env LC_TIME=lt_LT.UTF-8 date +"locale date+time: %c%nlocale date: %x%nlocale time: %X"
locale date+time: 2018 m. lapkričio 12 d. 20:15:19
locale date: 2018.11.12
locale time: 20:15:19

Other than ensuring that Preferences→Advanced→General→Date and Time Formatting is set to "Regional settings locale", the only thing I have to do is launch Thunderbird with the lt_LT.UTF-8 LC_TIME setting. E.g., I copied the Thunderbird launcher out of the GNOME menu and modified it to be:

env LC_TIME=lt_LT.UTF-8 thunderbird %u

Otherwise, /etc/locale.conf (which is sourced by /etc/profile.d/lang.sh) sets LANG to en_US.UTF-8.

I wish this weren't so complicated (and/or buggy), but hopefully the lt_LT.UTF-8 trick works for others.

Comment 76

[jmkrnet]

(In reply to gshollingsworth+mozilla from comment #74)
> ... My view of the locale treatment in Linux vs. Windows & Mac; Linux does have
> different support for customization than either Windows or Mac, but it does
> support customization. I can chose a different locale for the LC_TIME than
> all the others, my expectation is that the format specified in the generated
> locale file is what I should see. ... In Linux I can find and load a locale
> definition file in /usr/share/i18n/locales, create my own, or modify an existing
> one. ... If I so chose I think I could define a custom date/time format not used
> anywhere else in the world and set LC_TIME to it, I would expect that format to
> be used. ... I do not expect the format to come from some other reference that
> the system itself does not use. my 2 cents.

Completely agree that is basically why I reported the bug 1502659 (the usable workarounds are listed there maybe more clearly as there is a lot less comments:).

Comment 77

[[e-mail]]

ISO-8601 format dates are the preference of quite a few people across languages and cultures for many reasons including those that lead to the standard being created in the first place.  As it is an international standard, globally accepted, for formatting date and time and embraced by most technical and many non-technical users alike, I believe it would be justifiable to make a special case in the date/time formatting dialog to add the option, right in the advanced/general UI preferences, to select this international, global standard for date/time formatting regardless of other I10N choices.

The extremely user-unfriendly hoops one has to jump through to present the date in a format that is an international standard are utterly absurd.  That there's a massive comment flood of highly technical people struggling to figure out how to do something so simple and basic is a clear proof that the UI model needs a correction.

ISO-8601 is /the/ global standard for date-time.  There is not another one.  There's no competition.  Adding an ISO-8601 option will not open the door to requests to add radio buttons for other standards because they're aren't any other global standards for the unambiguous representation of date and time.  Enabling easy selection of ISO-8601 simply makes a standards compliant experience something normal users can achieve, rather than an absurdly complicated, fragile exercise in root-level config file hacking.

Comment 78

[jflorian]

(In reply to [e-mail] from comment #77)

> ISO-8601 is /the/ global standard for date-time.  There is not another one. 
> There's no competition.  Adding an ISO-8601 option will not open the door to
> requests to add radio buttons for other standards because they're aren't any
> other global standards for the unambiguous representation of date and time. 
> Enabling easy selection of ISO-8601 simply makes a standards compliant
> experience something normal users can achieve, rather than an absurdly
> complicated, fragile exercise in root-level config file hacking.

Well said.  I just want to add the MSF should take the lead here and not just with TB.  Please make this simple, if not the startup default.  It's time for the world to end the absurdities with the ludicrous number of time formats and the ridiculous complexities they impose on software and its users.  ISO-8601 is the global standard.  Let's embrace it and let the other formats fail for what they really are, relics of an un-unified world.  Those who wish to cling to the past should be the ones making herculean efforts, not those who want to improve the situation.

Comment 79

[Jonathan Kamens]

This issue is not about ISO 8601 date formatting. It's about the fact that the behavior of Thunderbird changed to no longer allow the user to set the time/date format as per the locale. Please file a separate bug (enhancement request, really) if you want Thunderbird to natively support the ISO 8601 date format, and stop commenting about it here.

Comment 80

[Link @ QPG]

An explicit option to enable ISO-8601 time formatting in the Thunderbird UI is definitely something that belongs in it's own enhancement request.

The assertion that this bug has nothing to do with ISO-8601 couldn't be further off the mark, though. While the string "ISO-8601" isn't in the bug title, the parenthetical description "(LC_TIME=en_DK.utf8 behaves differently than it used to)" makes it very much about ISO-8601 support. Setting LC_TIME to en_DK.utf8 has been a long standing documented solution to the lack of sensible date formatting options for English speaking users.

It's the entire reason that the en_DK locale exists.

Comment 81

[vladimir-csp]

To end the argument about applicability of this bug report to ISO 8601 date issue, I've just opened new unambiguously related one:
https://bugzilla.mozilla.org/show_bug.cgi?id=1509096

Comment 82

[Jorg A. Willekens]

As Jonathan Kamens said, this is not about ISO 8601.  It is about the fact that you can create new locales: And people do!  Proof above is "localebug" who uses en_DE.

I use en_LU, and it has worked perfectly over the years.  Now?  I get the C locale, which is basically en_US. The log entry I see is:

(thunderbird:5925): Gtk-WARNING **: 09:23:34.665: Locale not supported by C library.
	Using the fallback 'C' locale.


It is not even hard to make a new locale: you copy the one you want and make a few changes, then you add it to the locale-gen configuration file (location depends on distro) and then you run locale-gen, and modify your /etc/defaults/locale.  You do that once, and you're fine.

That's why this system exists.

On Windows I do the same: I take en_GB and modify the settings to use our local date/time and currency settings.

On Linux, I usually take /usr/share/i18n/locales/fr_LU, copy it to /usr/share/i18n/locales/en_LU, and modify the field "language" under LC_IDENTIFICATION.  Why do this?  Because if I use fr_LU, I get my dates formatted like this: "23/11/2018, à 9h34" and the interface will use French weekdays.  I don't want that.

This post is that I fully disagree with the choice to just support only "well known locales".  Yes, you can do this with using LC_ environment variables and mix-and-matching, but the flexibility is so much worse than making your own.

Remember, Linux is supposed to give you Freedom.

Comment 83

[Jorg A. Willekens]

> 
> (thunderbird:5925): Gtk-WARNING **: 09:23:34.665: Locale not supported by C
> library.
> 	Using the fallback 'C' locale.
> 

Sorry, this is when you don't have the locale installed. (Quick test on a machine that wasn't configured like this, my mistake)  However, if you do have a custom locale, the warning disappears but, the dates remain in C locale.

Comment 84

[Eric Toombs]

This is infuriating. I observe that thunderbird is in fact responding to changes in the LC_TIME environment variable, including LC_TIME=en_DK.UTF-8, but not the way it's supposed to. Specifically. I have tried C, ru_RU.UTF-8, en_GB.UTF-8, en_DK.UTF-8, and da_DK.UTF-8. These are the results:

en_GB is supposed to be 26/11/18, but thunderbird does 26/11/2018. It gets the time right, though.

en_DK is supposed to be 2018-11-26 and 17:07, but thunderbird does 26/11/2018 and 17.07. Note the . instead of a :. Really weird.

da_DK is supposed to be 26-11-2018 and 17:07, but thunderbird does 26.11.2018 and 17.07.

C is supposed to be 17:07, but thunderbird does 5:07 PM. It gets the date right, though.

ru_RU actually works like it's supposed to, but given the above results, I think this might be a coincidence.

There is no way this is intended behaviour. To me, this looks like some kind of memory corruption, or a broken linked list, or something. If over the coming weeks it **** me off enough, I'll step through the date rendering code with a debugger and maybe figure out what in the hell's going on.

Under Preferences>Advanced>General, Date and Time Formatting was set to Regional settings locale the whole time.

When I switch this preference to Application locale, everything basically still remains broken, in the same way. I don't feel like testing this in great detail. Oddly enough, though, even here, ru_RU is the only one that works like it's supposed to, causing English (US) dates to be displayed.

Comment 85

[Zibi Braniecki [:zbraniecki][:gandalf]]

>  but not the way it's supposed to.

It would be useful to understand what do you reference to as what's "supposed to be".

We follow Unicode CLDR which provides date/time patterns for locales. You can review them for example here:

http://www.unicode.org/cldr/charts/34/summary/ru.html#2060
http://www.unicode.org/cldr/charts/34/summary/fr.html#2106

and so on. If you see an example where Gecko does not follow CLDR locale, please provide the output of the Intl section of about:support (bottom of the page) and STR. Thank you!

Comment 86

[Eric Toombs]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #85)
> >  but not the way it's supposed to.
> 
> It would be useful to understand what do you reference to as what's
> "supposed to be".
> 
> We follow Unicode CLDR which provides date/time patterns for locales. You
> can review them for example here:
> 
> http://www.unicode.org/cldr/charts/34/summary/ru.html#2060
> http://www.unicode.org/cldr/charts/34/summary/fr.html#2106
> 
> and so on. If you see an example where Gecko does not follow CLDR locale,
> please provide the output of the Intl section of about:support (bottom of
> the page) and STR. Thank you!

Thank you for replying so quickly, Zibi. This explains everything. I am comparing to the output of glibc's strftime(). The CLDR is obviously at odds with glibc as to the correct way of displaying the dates in all of these locales. Except for Russia. Apparently, Russia is much more assertive and decisive than the other countries I tested when it comes to communicating with these two organisations ;)

So, not-a-bug, sort of?

Is there a value of LC_TIME that the CLDR will interpret as ISO? I can set LC_TIME to that just for thunderbird. That would be a pretty good workaround. I'll start rummaging around the CLDR documentation and see what I find.

Comment 87

[Magnus Melin [:mkmelin]]

See whiteboard. At least pretty close, LC_TIME=en_SE.utf-8

Comment 88

[Eric Toombs]

I just read the previous posts. I really should have actually read this thread more thoroughly before. Looks like everything was already addressed. And I could have saved an enormous amount of time debugging if I had known to break on mozilla::DateTimeFormat::FormatPRTime. (whoops...)

I think it does make sense to have a value of LC_TIME that causes ISO dates to be rendered, but the en_DK hack frankly never made sense to me. What we need is a locale like LC_TIME=iso8601. There is a precedent for a non-localised locale: LC_TIME=C. Unfortunately, C is a complete locale usable for anything and corresponding roughly to en_US. There is no precedent for a partial locale AFAIK. I think it will be really really hard to sell the CLDR maintainers on this one.

It might still be possible to sell the CLDR maintainers on the en_DK hack, though. Failing that, I see two alternatives:

0. a new preference in thunderbird for ISO dates, as was suggested earlier in the thread.

1. a new preference for the local C standard library's strftime(). It is a part of the ISO C standard, therefore it is virtually guaranteed to be present on all platforms in some form or other.

Incidentally, it is icu that thunderbird uses to do the date formatting. Specifically, udat_open() is used to translate the locale, e.g. en-DK, into a format string.


A GOOD WORKAROUND
=================
I observe that sv_SE displays all dates and times exactly to ISO 8601 everywhere I've seen them so far. These are the steps to switch when using archlinux. I'm assuming it's basically the same in other distros.

0. Generate sv_SE. (Do this only once.)
$ sudo vim /etc/locale.gen
***uncomment sv_SE.UTF-8***
$ sudo locale-gen

1. Tell thunderbird to use sv_SE.UTF-8 from now on.
$ LC_TIME=sv_SE.UTF-8 thunderbird

Since en_DK is probably not working as expected in every application that uses icu to do its date formatting, this should fix them too. I already observe this workaround fixing the dates in qbittorrent! Yay! :D Today has been a good day.

You might have to learn the days of the week in Swedish. Whatever. It's basically just English with a funny accent.

Comment 89

[Eric Toombs]

(In reply to Magnus Melin [:mkmelin] from comment #87)
> See whiteboard. At least pretty close, LC_TIME=en_SE.utf-8

I observe there is no such locale as en_SE in glibc. I looked in /etc/locale.gen. I am using glibc version 2.28.

Comment 90

[Magnus Melin [:mkmelin]]

You just have to generate it

  locale-gen en_SV.UTF-8
  sudo pkg-reconfigure locales

FWIW, this is my global setup (my mother tongue happens to be Swedish, but I only want the time/date formatted locally - application texts and such in English).

Comment 91

[Marvin]

Is it possible for a Thunderbird extension to override the used date format string?

Comment 92

[Eric Toombs]

(In reply to Magnus Melin [:mkmelin] from comment #90)
> You just have to generate it
> 
>   locale-gen en_SV.UTF-8
I'm assuming you meant _SE.
>   sudo pkg-reconfigure locales
> 
> FWIW, this is my global setup (my mother tongue happens to be Swedish, but I
> only want the time/date formatted locally - application texts and such in
> English).

locale-gen ignores its argument. Adding en_SE.UTF-8 to locale-gen won't do anything. Your thunderbird is probably just failing over to sv_SE.UTF-8, which is why it looks like it worked. If I add en_SE.UTF-8 to /etc/locale.gen and run locale-gen, this is what happens:

% sudo locale-gen
Generating locales...
  da_DK.UTF-8... done
  en_DK.UTF-8... done
  en_GB.UTF-8... done
  ru_RU.UTF-8... done
  sv_SE.UTF-8... done
  en_SE.UTF-8...[error] cannot open locale definition file `en_SE': No such file or directory
%

Comment 93

[Magnus Melin [:mkmelin]]

Sorry, yes, should be LC_TIME=sv_SE.utf8

Comment 94

[jmkrnet]

(In reply to vladimir-csp from comment #81)
> To end the argument about applicability of this bug report to ISO 8601 date
> issue, I've just opened new unambiguously related one:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1509096

Thanks, that is quite similar to what I proposed when reporting bug 1502659. It is especially nice that it seems there is a chance that a proper solution to all these date/time formating issues will be implemented. Thus, I voted for your bug. Maybe other people affected by the date/time formating issues should also vote there:).



As a side note:
Listing the workarounds using root.UTF-8, en_SE.UTF-8 and other locales using whiteboard could help most people affected by the date/time formating issues who find this bug. Thus, it would be beneficial to add an updated info about workarounds back to the whiteboard. For similar reasons the bug 1509096 should be added to "See Also". Unfortunately I do not have privileges to do this.

Comment 95

[vladimir-csp]

Thanks! sv_SE workaround is working.

So, am I correct that Thunderbird and Firefox do not support custom locales in principle?

Custom locales are the proper way of customizing such things in the system. For example to customize a ru_RU locale you copy locale definition file ru_RU into ru_RU@ISO, replace content of every section with 'copy "ru_RU"' (for deduplication and smooth updates), except LC_TIME, in which a custom ISO-like string can be defined appropriate for the locale being copied. New locale will be compiled by locale-gen into ru_RU.UTF-8@ISO (@suffix is a proper way of defining variants). And this will work for everything except Thunderbird/Firefox, apparently.

Comment 96

[Eric Toombs]

(In reply to vladimir-csp from comment #95)
> Thanks! sv_SE workaround is working.
> 
> So, am I correct that Thunderbird and Firefox do not support custom locales
> in principle?
> 
> Custom locales are the proper way of customizing such things in the system.
> For example to customize a ru_RU locale you copy locale definition file
> ru_RU into ru_RU@ISO, replace content of every section with 'copy "ru_RU"'
> (for deduplication and smooth updates), except LC_TIME, in which a custom
> ISO-like string can be defined appropriate for the locale being copied. New
> locale will be compiled by locale-gen into ru_RU.UTF-8@ISO (@suffix is a
> proper way of defining variants). And this will work for everything except
> Thunderbird/Firefox, apparently.

It's not just thunderbird/firefox. Hacking the glibc locale won't work in any program that uses icu instead of glibc to do its localisation. That includes many QT programs, like qbittorrent.

I'm not happy with this state of affairs either.

Comment 97

[Eric Toombs]

You can still hack icu though. So now, you have to hack two locale definitions instead of just one lol.

Comment 98

[Zibi Braniecki [:zbraniecki][:gandalf]]

> You can still hack icu though. So now, you have to hack two locale definitions instead of just one lol.

Hahha, yeah! I remember there's a push to move glibc to use CLDR. I can't wait for it to happen :)

Comment 99

[Jim Moe]

thunderbird v60.3.0
opensuse LEAP 15.0 / linux 4.12.14

AFAICT TB does not do well with locale times any longer. I, too, lamented the loss of choosing my preferred datetime format, basically ISO-8601. More sadly for me, a very nice addon, SuperDateFormat, was not updated for the new plugin API. So I went in search of an alternative method since the choices I had were en_US or en_US.

I quickly discovered the en_DK hack. And it works, sort of, in a vague way resembling what I wanted.

My preference for the displayed datetime is "YYYY-MM-DD HH:MM:SS" or something quite similar. Supposedly en_DK would deliver that format. Looking at the contents of en_DK, it should.

What I get is "DD/MM/YYYY, HH:MM". That pattern exists nowhere in en_DK; it appears to be bastard combination of en_US and en_DK. It exists only in TB's code logic somewhere. It is an improvement over the default; not what I expected.

And, really. WTF is so wrong about providing a custom date presentation option? Or even a limited list of pre-processed options?

Comment 100

[Eric Toombs]

(In reply to Jim Moe from comment #99)
> thunderbird v60.3.0
> opensuse LEAP 15.0 / linux 4.12.14
> 
> AFAICT TB does not do well with locale times any longer. I, too, lamented
> the loss of choosing my preferred datetime format, basically ISO-8601. More
> sadly for me, a very nice addon, SuperDateFormat, was not updated for the
> new plugin API. So I went in search of an alternative method since the
> choices I had were en_US or en_US.
> 
> I quickly discovered the en_DK hack. And it works, sort of, in a vague way
> resembling what I wanted.
> 
> My preference for the displayed datetime is "YYYY-MM-DD HH:MM:SS" or
> something quite similar. Supposedly en_DK would deliver that format. Looking
> at the contents of en_DK, it should.
> 
> What I get is "DD/MM/YYYY, HH:MM". That pattern exists nowhere in en_DK; it
> appears to be bastard combination of en_US and en_DK. It exists only in TB's
> code logic somewhere. It is an improvement over the default; not what I
> expected.
> 
> And, really. WTF is so wrong about providing a custom date presentation
> option? Or even a limited list of pre-processed options?

See my post with title "WORKAROUND" ^^^^^^^^^^

Comment 101

[Eric Toombs]

Also, I know this thread is super long, but reading it is worth it. It explains everything. Short story: firefox, thunderbird, and many other programs use icu now instead of glibc and the two disagree on a lot of formatting details.

Comment 102

[Eric Toombs]

In case it's too hard to find, the post with title "WORKAROUND" is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c88

Comment 103

[Carlos O'Donell]

I am a GNU C Library (glibc) developer and I support the POSIX APIs for localization and internationalization.

I am also impacted by this change.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there. Having the glibc locales and Unicode/ICU/CLDR locales in sync means fewer surprise for our users. This can include adding new "formats" like those which CLDR has for dates. For example glibc just recently added alternative date formats to support two grammatical forms for month names (%OB and %ob), particularly for Slavic/Baltic languages that use genitive/nominative cases.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

A good summary of the problem looks like this:

• Thunderbird has stopped allowing users to customize their date formats. Modifying the CLDR data is significantly more complex (expert level). Thunderbird tries to guess your appropriate CLDR locale by reading your system locale. There is no customization allowed beyond that.

• Previously Thunderbird using POSIX APIs and glibc's locale support. Users could build their own locales with their own formats for their system. For example I have my own '[e-mail]-8' that has ISO 8601 time for US English. It's really easy to do this.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR (which I assume is used by Mozilla because it's a multiplatofrm library).

If you have any questions about the glibc APIs or other things, I'm happy to help, and improve them.

So far none of the extensions I have found solve the problem either. I don't consider it acceptable to switch to sv_SE locales.

Comment 104

[Zibi Braniecki [:zbraniecki][:gandalf]]

Hi Carlos! Thanks for bringing your perspective. I'll try to respond to it.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there.

That's wonderful! I truly hope that we'll end up all using CLDR, since I see no value in duplication of efforts.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

Comment 105

[Link @ QPG]

While I can appreciate consolidation, one thing that CLDR has done is turned what was a deliberate hack into a literal interpretation. en-DK was never meant to be a locale that had anything to do with Denmark. It was created to easily allow ISO-8601 date and time formats ofr English speakers. CLDR taking it literally as English Denmark has introduced undesirable characteristics such as number formatting and currency.

In the grand scheme of things, as an affected user I don't care as long as there is an eventual return of an easy way for me to get ISO-8601 dates and 24 hour time. I've implemented the en_SE.utf8 workaround by creaingan en_SE.utf8 system locale and modifying the envvars in the Exec lines of my user specific thunderbird.desktop file, but long term this should be offloaded to something more intuitive and accessible to less technical users.

Comment 106

[Ken Fallon]

For example I have my own '[e-mail]-8' that has ISO 8601 time for US English. It's really easy to do this.

I'm more than happy to try and create my own custom format but it seemed to be way above my technical level. That said if you can point me to a guide, documentation, video, podcast etc on how to do that I would seriously appreciate it.

Comment 107

[Carlos O'Donell]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #104)

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

I don't see this as an edge case.

All users want to customize the display in their mail client.

Lack of time customization is simply a failing on behalf of the Thunderbird application.

However, I am not a Thunderbird developer, and so I don't get to make prioritization decisions, you need to do that.

This is high priority for me. The swapped en-US m/d/y will cause me to make mistakes and I can't tolerate that, so I'm going to probably switch back to Mutt, or another client after review.

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

Easy for anyone. I'll show you. It was designed this way to be easy.

dnf install glibc-locale-source
su
cp /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_US@ISO8601
vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/[e-mail]8

You're done. This is impossible to do with ICU/CLDR.

Even better is you can ship this in Ubuntu Universe or Fedora Copr.

Non-technical users don't even need to do anything but point at the repo, install the compiled locale, and set their LANG env var.

Again, this cannot be done with CLDR.

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

No, everything else is also easy. Just look at Windows.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

[1] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

We had exactly this kind of flexibility with system locales, but ICU/CLDR does not provide that.

Windows 10 lets you write the patterns out exactly how you want to see them, so do Linux locales.

You could implement this in Thunderbird via ICU/CLDR APIs:

UnicodeString aPattern("GyyyyMMddHHmmssSSZ", "");
new SimpleDateFormat(aPattern, new DateFormatSymbols(Locale::getUS())
...

Example taken from: http://userguide.icu-project.org/formatparse/datetime

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

I'll look at glueing this into the system locale. It looks like we already have "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);" to fetch the preference which is somewhat mislabelled as "true" for Regional settings locale (means use the system defaults). I might propose another patch to change that, but we still have to honour that setting when picking a date format.

Comment 108

[Carlos O'Donell]

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-) Sorry.

Comment 109

[Zibi Braniecki [:zbraniecki][:gandalf]]

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

Correct, from the GUI. And that includes validation of pattern syntax etc.

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-)

No worries, it was added very recently and we all get confused :)

I'll look at glueing this into the system locale.

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

Comment 110

[Carlos O'Donell]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

I agree 100%. This is what I meant by "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);", which fetches the preferences from the store. The Preferences->Advanced->Date and Time Formatting radio controls actually set this to false (use ICU/CLDR) or true (try to use the OS information).

Comment 111

[jflorian]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

We must agree it's non-zero. Quite a few users are present here alone, but a simple Google search for the old en_DK hack shows there are many users who want this and those are just the few who took the time to make such a post. While many of those posts may not be specific to TB, I think it's safe to assume that if they do use TB, they'd expect their preferred format there too.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Technical proficiencies vary wildly. But if TB no longer honors the glibc formats by any means short of personally applying a patch, I'd argue that it has been made extremely difficult for anyone. This sort of exclusion is the same reason I can't use GNOME: I don't always agree with the designers and want some choice to wiggle into a more comfortable setup that works for me. It's sad that something like this is now making me consider giving up many years of getting very comfortable with TB for an alternative MUA. First it's the loss of Google search, now this. I'm likely to start looking into kmail again -- I know it won't leave me hanging on something so basic as this. I'm losing faith in MSF. :(

Comment 112

[vladimir-csp]

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

The mechanism for locale customization exists and works. It does not matter whether a user utilizes it via terminal or via some specialized user-friendly GUI.

Comment 113

[Jeff Harp]

Here are two fixes I came up with for this bug.

The first implements a new string preference named intl.locale.date_time.lc_time_override. If this preference is found, its value is used instead of the LC_TIME environment variable value. An example value for this preference is "root.UTF-8", which yields the ISO 8601 date/time format users want. This preference eliminates the need for fiddling with the platform's locale system or changing LC_TIME when starting Thunderbird.

The second implements a new string preference named intl.locale.date_time.pattern_override. If this preference is found, its value is used as the date/time pattern instead of the one that Mozilla comes up with. An example value for this preference is "y-MM-dd hh:mm", which yields the ISO 8601 date/time format users want. This preference provides very direct, granular control over the display of date/time values.

The code modifications can be found here:

https://gitlab.com/jeff_harp/thunderbird_60.4.0_date-time_mods

See the README.md file for more details.

I've allocated as much time to this as I can. If someone else wants to pursue these modifications further, please do so. I cannot.

Comment 114

[gshollingsworth+mozilla]

@Jeff Harp
Thank you for your efforts.

I really like your second fix approach. To me it is very much like how I am used to fixing the issue in other apps and platforms. It should also work cross platform.

For me, your first approach does little for me. I want this same format system wide, so I already have locale and LC_TIME customized. It seems to apply only to Linux/Unix/BSD. I can see it's validity for others though.

Comment 115

[u634361]

I've used Thunderbird since its start, I have saved many (many) clients from buying 365 with this and Libre office. This is the only problem which irks me, smacks a bit of "we know best" it just seems right to allow the user to choose the format they prefer.
Any fix would need to affect the add ons, lightning is unusable for me because of this problem.
Regards

Comment 116

[henri.passerieux]

I'm just an end-user frustrated with this situation. I have also used Thunderbird since its start, moving during the years from OS/2 to Windows to Mac and now to Linux Mint. I'm from Finland, my native language is Swedish, but my systems have always been in English (US or GB). I have lived and worked in several European countries, now in France, and I want to have date and time formats according to the country where I live. This country/language/currency cocktail is maybe complicated for the Americans, but this is the way the world works. Please fix this.
Thank you.

Comment 117

[Jorg K (CEST = GMT+2)]

And simply setting the advanced option "Date and Time Formatting: [x] Regional settings locale French (France)" doesn't work assuming that you have set your machine to a French locale? As I understand the bug, it's about a very specific use case which is not what you're describing.

Comment 118

[henri.passerieux]

No, it does not work. Thunderbird insists on en-US. The machine is set to regional French(France) and everything else respects that setting for date/time/hour/currency/decimal

Comment 119

[Jorg K (CEST = GMT+2)]

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

Comment 120

[Clément Lefèvre]

(In reply to Jorg K (GMT+1) from comment #119)

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

Well, I would actually be curious about how so.

This is the exact reason I'm CC'd to this bug and others: My system is using en_US as locale, but I obviously want to use 24H format, as well as changing dates for example if possible.

Current format force user to some mental exercise every time he needs to read date, time and so on.

Comment 121

[henri.passerieux]

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.
Now Thunderbird shows Regional settings locale as English (United Kingdom), dates show as DD/MM/YYYY and the time in 24-hour format. Much better like this.

Makes me think there is a problem of definitions: what is covered by the Language settings and what is covered by the Region settings. Linux, Mint, Cinnamon, or Thunderbird problem?

Comment 122

[Jorg K (CEST = GMT+2)]

(In reply to Clément Lefèvre from comment #120)

Well, I'm not a Linux user, but I have seen felt 100 bugs on the issue and read a million comments.

We have Thunderbird developers in Finland and Slovakia who use a local locale and get localised display. If you use the en-US locale for regional display, all apps should should 12h AM/PM. If, unlike Windows, Linux doesn't allow the locale to be configured, why don't you used a less "mental exercise"-prone locale like en-GB. Also, please read the bugs quoted in comment #2. As I understand it, this bug here is about LC_TIME=en_DK.utf8 and nothing much else.

Comment 123

[Jorg K (CEST = GMT+2)]

(In reply to henri.passerieux from comment #121)

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.

I still don't understand why you don't set absolutely everything to French. Then TB would do "regional settings locale French (France)". Is that not what you want?

At least on Windows, TB follows format for a selected region, I don't know that the regions per se is? You mean location?

Comment 124

[henri.passerieux]

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting. And I do want to keep the system language as English. Now it's English UK and I'm fine with that. Next time I know to select that already when installing.

Comment 125

[Zibi Braniecki [:zbraniecki][:gandalf]]

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting

We do some special customizations for the old Unity from Ubuntu. I would accept a patch that extends that to cover Mint preferences if anyone is willing to write a patch :)

Comment 126

[Frank Steiner]

Any chance you consider Jeffs patch? Being able to define one's own date-format string would be very helpful.

Comment 127

[Zibi Braniecki [:zbraniecki][:gandalf]]

not directly, no. The patch would need generalization (for example, it provides a single pattern, while we need patterns per style - narrow, short, medium, long, full etc.)

I'd also like to see how it fits into the conversation in https://github.com/tc39/ecma402/issues/190

To sum it up - in principle, I'm not against extending our platform capability to fine-tune intl settings, and I recognize that date/time patterns are particularly sensitive area to customization.
But I don't think we should be solving it on the Gecko level by writing shortcuts through the system to satisfy some particular UI widget.

To make it actionable: If someone is willing to write a patch which provides hardcoded custom patterns for date, time and date-time styles (narrow/short/medium/long/full).

I imagine such API to lead to us being able to add UI similar to what MacOS and Windows do [0].

It should likely be a new API in intl/locale, named maybe I18nSettings, which would manage setting and retrieving custom overrides for date/time patterns (and maybe first day of the week, like Windows does?) if they're set, or use data from locale / os preferences when not.

Happy to mentor, but I won't have time to work on that anytime soon.

[0] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

Comment 128

[Len Weisberg]

Bibi wrote in Comment #24

• we will match the locale that your OS returns, but we'll take the values
  from CLDR since we know we have all the tables for each locale that we need.

(Just to clarify - by "locale" do you mean the locale-name ?)
But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

In particular, I would be very happy to use the CLDR "root",
see http://demo.icu-project.org/icu-bin/locexp?_=root ,
as its dates are in order Y/M/D and the times are 24-hr.
Why not simply let us specify this name, "root", directly as a preference?
If a name is specified and it is not a valid CLDR locale, then you
can use the current procedure as a fall-back.

(I am asking this because I have not been able to figure out how to
make "root" a valid POSIX locale name on my system - Linux Mint.
The recipe in Comment #59 didn't work for me.)

Comment 129

[Tristan Miller]

(In reply to Carlos O'Donell from comment #107)

Easy for anyone. I'll show you. It was designed this way to be easy.

Carlos, I'm trying to follow your instructions but for me they are anything but "easy":

dnf install glibc-locale-source

I think this is highly distribution-dependent. I have no idea what dnf is but it doesn't exist on my system (openSUSE). I'm guessing that this command is supposed to invoke some package manager that installs the glibc locale sources. The source package turns out to be named something completely different on openSUSE -- it's called glibc-i18ndata.

vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

For me, the value of d_fmt in en_US is "<U0025><U006D><U002F><U0025><U0064><U002F><U0025><U0059>", which is completely opaque. All the other variables in this file, and every other file I've examined, are defined with similar escape sequences. Most files don't have explanatory comments (not even ones that simply unescape the values). It's not obvious whether I'm even allowed to replace an escaped value with an unescaped one, nor how to easily automatically convert an unescaped string to an escaped one. (I'm sure as hell not going to look up every single character in a Unicode table so that I can type it in manually, and writing a program to do so seems to be about as much effort.) In fact, a less technically sophisticated user wouldn't even understand the fact that the values are escaped.

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/[e-mail]8

You're done. This is impossible to do with ICU/CLDR.

Well, I tried copying and pasting some existing key-value pairs from other locale files to build a new one, en_CA@ISO8601. The localedef command seemed to compile my new locale, but (for reasons probably mentioned elsewhere in this conversation) Thunderbird doesn't seem to want to use it. When I run it with LC_TIME=[e-mail]8 and/or LANG=[e-mail]8, all the dates and times in the mail list show up in the usual en_CA format (i.e., with YYYY-MM-DD dates but hh:mm a.m./p.m. times).

Comment 130

[Simon Marquardt (smrqdt)]

(In reply to Len Weisberg from comment #128)

But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

At least for me this is not required. If you start Thunderbird with "LC_TIME=root thunderbird" it uses CLDR’s “root“ locale time and date format, even if it does not exist as a installed system locale. There’s a Gtk-Warning about fallback to C locale, but this does not seem to have impact, as Thunderbird ignores it anyway.

As a hotfix, I overrode the system desktop file for Thunderbird by copying it to ~/.local/share/applications/ and put "env LC_TIME=root " before the binaries in all Exec lines. No tinkering with the POSIX locales.

(tested on Arch Linux and Thunderbird 60.6.1)

Comment 131

[gshollingsworth+mozilla]

(In reply to simon.marquardt from comment #130)

(In reply to Len Weisberg from comment #128)

But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

At least for me this is not required. If you start Thunderbird with "LC_TIME=root thunderbird" it uses CLDR’s “root“ locale time and date format, even if it does not exist as a installed system locale. There’s a Gtk-Warning about fallback to C locale, but this does not seem to have impact, as Thunderbird ignores it anyway.

As a hotfix, I overrode the system desktop file for Thunderbird by copying it to ~/.local/share/applications/ and put "env LC_TIME=root " before the binaries in all Exec lines. No tinkering with the POSIX locales.

(tested on Arch Linux and Thunderbird 60.6.1)

If all you want is YYYY-mm-DD HH:MM date format in Thunderbird, then yes you do not need to tinker with the POSIX locales. If you want the rest of the system to have the same date time format then you do have to tinker with the POSIX locales because that is how the system date time format is defined.

The change in Thunderbird caused us to have to change our workaround to ensure the LC_TIME variable value matched the new approach using a CLDR locale name.

We have to keep in mind that "root" is not really a locale per se in CLDR but a fallback, base, or reference. If it is decided that "root" is unnecessary in CLDR, then this workaround will likely stop working in Thunderbird. But the system date time format will continue to be as we desire no matter what we choose as a name.

Comment 132

[Len Weisberg]

I am asking the CLDR folks for help with all of this. Please see my new ticket:
https://unicode.org/cldr/trac/ticket/12027
"CLDR should support (some variants of) ISO-8601 date formats."

Comment 133

[Greg]

Mozilla developers, Thunderbird is the only app on my system that screws up the date. This isn't a case of there not being an API or something. The fault lies with Mozilla developers. Thunderbird should at least display date & time format similar to how other apps are handling this. Every other app on my system keeps 24-hour time and displays the date according to GNOME's Region & Language setting.

Can you please mark this for priority bug fixing?

In the meantime, is there any about:config setting or extension that can remedy this situation? Date and time is important when dealing with emails.

Screenshots of the problem:
https://imgur.com/a/XlqdXxV

Comment 134

[Zibi Braniecki [:zbraniecki][:gandalf]]

Every other app on my system keeps 24-hour time and displays the date according to GNOME's Region & Language setting.

Can you provide Steps to Reproduce for when Thunderbird doesn't follow your selection of 12/24h clock in Gnome? Because it should.

Please, also attach the Internationalization section of your about:support.

Thank you!

source of where we look up Gnome 12/24 clock settings - https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#144

I'm wondering if that's one of the places in UI where Thunderbird doesn't use Gecko APIs that respect Gnome's 12/24h.

Comment 135

[Victor Engmark]

Can you provide Steps to Reproduce for when Thunderbird doesn't follow your selection of 12/24h clock in Gnome? Because it should.

1. Run thunderbird -profilemanager
2. Set up a new profile
3. Set up an email account
4. Look at the date column values

Actual behaviour: The values are shown in a weird format: "day of month without leading zero/month with leading zero/two-digit year, hour without leading zero:minute with leading zero [am|pm]", for example "1/05/19 6:09 am".

Expected behaviour: Dates should show up either as printed by the relevant utility:

$ date
Wed May 29 18:54:02 NZST 2019

or as the by now universal coarse-grained ISO format of "YYYY-MM-DD hh:mm Z".

Internationalisation & Localisation settings:

Application Settings
Requested Locales 	["en-NZ","en-US"]
Available Locales 	["en-GB"]
App Locales 	["en-GB","und"]
Regional Preferences 	["en-NZ"]
Default Locale 	"und"
Operating System
System Locales 	["en-NZ"]
Regional Preferences 	["en-NZ"]

Comment 136

[Zibi Braniecki [:zbraniecki][:gandalf]]

The values are shown in a weird format:

I'm afraid there are multiple separate issues conflated in this bug into one.

In your case, you want to use en-NZ, and we format it according to Unicode/CLDR convention for en-NZ locale:

(new Date()).toLocaleString("en-NZ");
"29/05/2019, 12:37:11 am"

So, in your case, assuming he locale selection looks reasonable (we only have en-GB language pack, so the strings will be in en-GB, but your intl formatters use en-NZ), I can only recommend you reporting to CLDR that en-NZ date time format is broken.

Greg had, I believe, a different problem. He wanted his 12/24h setting to be respected, and I pointed out that our code does, in fact, respect 12/24h manual override from Gnome.

I'm afraid that everyone who doesn't like how Tb displays their datetime format come to this bug stating "just make it work", but each one may be describing a different problem solvable in a different way.

Comment 137

[Clément Lefèvre]

Same as comment #133, My clock is on 24H in Gnome while locales are en_US, and it shows time as 12H (and date as US format, which I would like to change too).

Here's i18n part of about:support:

Internationalization & Localization

  Application Settings
  Requested Locales: ["en-US"]
  Available Locales: ["en-US"]
  App Locales: ["en-US","und"]
  Regional Preferences: ["en-US"]
  Default Locale: "und"

  Operating System
  System Locales: ["en-US"]
  Regional Preferences: ["en-US"]

Comment 138

[mtfurlan]

So according to this thread, thunderbird should ignore whatever the system locales, read the name of a locale out of LC_TIME or whatever, and then look up stuff in the CLDR.

So why must the contents of LC_TIME be a real locale on the system?
I would think I should be able to run LC_ALL=root thunderbird and get usable dates as it just looks up what the CLDR claims root is.
But that doesn't work, root needs to be a real locale to get usable dates.

Any idea why it needs to be a real locale? Can that be fixed so thunderbird is completely separate from system locales, and maybe it's less confusing as to why thunderbird only kinda respects what locale is set?

Looking back closer, this was noted by Jonas Witschel months ago: https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c19

So steps(for debian at least) all in one place for the next person unfortunate enough to encounter this thread:

sudo cp /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/root
echo 'root UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen
sed 's/Exec=\/usr\/bin\/thunderbird/Exec=env LC_TIME=root \/usr\/bin\/thunderbird/' /usr/share/applications/thunderbird.desktop > ~/.local/share/applications/thunderbird.desktop 

Comment 139

[Mikhail Ryazanov]

Thunderbird 60.8.0 (from Debian).

I think, the new Thunderbird has a more serious problem with locales. I use my own custom locale, named "en_EU", which defines the date formats in the "European style" (German and Russian at least) but with English months and days names (like "Sun, 28 Jul 2019" and "28.07.2019"), and the 24-hour time format with colons (like "13:00" and "13:00:00").

So I have LC_TIME=en_EU.utf8 set, and all other programs that I use can understand this and show dates/times in my custom format. Thunderbird 45 also worked perfectly with it. But the current Thunderbird version apparently cannot use it at all, even though LC_TIME=en-DK does work (but I do not need that, I want my own format).

Can it be made to honor the actual system locale, whatever it is? Or at least provide customizable date/time formats, if the proper system-locale usage, which was working fine in the previous versions, magically became impossible in the past couple years?..

Comment 140

[Zibi Braniecki [:zbraniecki][:gandalf]]

Can it be made to honor the actual system locale, whatever it is?

As long as it matches one of the CLDR ones.

Or at least provide customizable date/time formats,

I'm open to this, but it requires quite some work and noone has committed to put that work in.

Comment 141

[u634361]

Honour the system locale, what does that mean any more? Yes there are some local anomalies as to the format of the date, and yes these can be offered as the default setting, BUT, and this is the whole point of free software, the end user should be able to override the suggestion and use whatever they want. Of course ISO dates will eventually become the norm (if only because it is the current Chinese format:) and as a programmer this is my preferred format, so ISO dates must be available - not a great problem really as most (if not all) programming languages use this format anyway.

As I am approaching retirement, I may dig into the code and see what can be done, although there is plenty of good code oth there already as most of the large open source projects do this anyway.

Comment 142

[u634361]

(In reply to Marc from comment #141)

Honour the actual locale, what does that mean any more? Yes there are some local anomalies as to the format of the date, and yes these can be offered as the default setting, BUT, and this is the whole point of free software, the end user should be able to override the suggestion and use whatever they want. Of course ISO dates will eventually become the norm (if only because it is the current Chinese format:) and as a programmer this is my preferred format, so ISO dates must be available - not a great problem really as most (if not all) programming languages use this format anyway.

As I am approaching retirement, I may dig into the code and see what can be done, although there is plenty of good code out there already as most of the large open source projects do this anyway.

Comment 143

[Len Weisberg]

This ticket was opened 2017-12-22 18:26 PST (yes, ironically, that is
how bugzilla formats the date in the tooltip under "2 years ago",
actually more like 1.7 years ).

I appreciate that this bug has wandered into many issues and also that
there may be no one available to work on it. But I would just like some
help with working around what I believe is the main problem:
I would like help in cajoling TBird to use CLDR's "root" locale time and
date format. I feel certain that I am by no means alone in wanting this.

Some workarounds have been presented here, (eg in Comment #59), but they
don't seem to work for me. I am guessing that the key thing missing is a
file to be installed as a "root" locale. (I also have a hunch that
this file could be very simple if it is intended only to act as the
value of LC_TIME for running TBird.)
Could one of the experts here please supply this "root" locale file
and fill in and/or correct the following outline:

As root user:

1. copy the simplified file (found at ...) to /usr/share/i18n/locales/root
2. run: echo 'root.UTF-8 UTF-8' >> /etc/locale.gen
3. run: locale-gen
   As ordinary user:
4. set TBird Prefs -> Advanced -> "Date and Time Formatting"
   to ...
5. quit TBird and restart with LC_TIME=root.UTF-8

Does this seem feasible? Can we get some help with this?
(BTW, I am using Linux Mint 19.1 - based on Ubuntu 18.04)

Thanks much,
-Len

Comment 144

[jflorian]

@len what you detail looks about right. The workaround shown in comment #138 was the one that got me to happiness. Adapted to Fedora, it went like this:

sudo dnf install -y glibc-locale-source
sudo cp /usr/share/i18n/locales/{en_DK,root}
sudo localedef -v -i root root
sed 's/Exec=thunderbird/Exec=env LC_TIME=root thunderbird/' \
       /usr/share/applications/mozilla-thunderbird.desktop \
       > ~/.local/share/applications/mozilla-thunderbird.desktop

As for TBird Prefs -> Advanced -> "Date and Time Formatting", I see "Regional settings locale: root" selected. I don't recall what I saw there or had set before doing the above. Best of luck!

Comment 145

[Len Weisberg]

Well, I see now that this workaround really is as simple as described
in Comment #138 (and others). And no need for a special locale file.

The only weak link now is the stability of the CLDR's "root" locale.
There has actually been some activity on the ticket I filed with
the CLDR folks (see Comment #132), but I have no idea what it means
in terms of anything concrete.

Comment 146

[Mikhail Ryazanov]

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #140)

Can it be made to honor the actual system locale, whatever it is?

As long as it matches one of the CLDR ones.

Every OS provides standard APIs for locales and allows end-users to customize these locales to their personal needs. What was the purpose of breaking this approach, which has been working well for many years, in favor of some third-party libraries with incomplete hard-coded stuff? Did it solve any problems? So far I can only see that is has created problems for many users. And worst of all, these problems apparently cannot be circumvented by any reasonable means (I do not consider custom-patching the source code and recompiling everything as a viable solution).

Or at least tell us how to define a custom CLDR locale, if Thunderbird developers completely refuse to use the standard OS locale APIs, as they did in previous versions.

Or at least provide customizable date/time formats,

I'm open to this, but it requires quite some work and noone has committed to put that work in.

There was some patch in comment #113.

Comment 147

[Zibi Braniecki [:zbraniecki][:gandalf]]

What was the purpose of breaking this approach

Gecko is not an OS. And unfortunately we do not provide a full UI/UX for customizing the locales to personal needs.

in favor of some third-party libraries with incomplete hard-coded stuff?

Please, educate yourself on the topic you're writing an opinion on.

Did it solve any problems?

Yes. It's been a very important and large scale refactor of the ecosystem which allowed us to remove large amount of unmaintained code and simplify the maintenance of the intl module. If you are curious about more, please, read the changelog, or my blog posts from last years about it.
If you did not have time or interest, assume your ignorance in the topic and avoid claiming knowledge.

So far I can only see that is has created problems for many users.

Your visibility of the impact is limited. How often do you triage the Intl module?

And worst of all, these problems apparently cannot be circumvented by any reasonable means

They absolutely can. It just takes time to write customization overlays, or platform bindings. If you have interest in that, I'm happy to mentor.

Or at least tell us how to define a custom CLDR locale

I doubt there's a concept of "custom CLDR locale", but feel free to read the docs and find out yourself - http://cldr.unicode.org/

There was some patch in comment #113.

That patch would be a good start, but requires a bit of work to get it into making it applicable for upstream (see Comment 127 ).

I'm tempted to restrict comments in this bug. The signal to noise ratio is declining and developer conversation about how to fix this bug vs. "I want X" as well.
Since it's in TB I'll leave it open to TB triage owners to decide, but I'm likely going to stop responding since I am repeating myself to people who seem to have enough interest to voice their opinion, but not enough to dive into the problem domain.

Comment 148

[Erik Johnson]

Comment #144 This is fantastic, thank you @jflorian

Comment 150

[Greg]

My gnome LC_TIME locale is set to "en_DK.UTF8", which should produce time format YYYY-MM-DD HH:MM

Thunderbird is the only application that doesn't honor this. Is there a way to get it honored until this bug is fixed proper? Can someone post/repost a way to fix this now (if a way exists)? Thanks!

Comment 151

[Bernhard Groll]

(In reply to Greg from comment #150)

My gnome LC_TIME locale is set to "en_DK.UTF8", which should produce time format YYYY-MM-DD HH:MM

Thunderbird is the only application that doesn't honor this. Is there a way to get it honored until this bug is fixed proper? Can someone post/repost a way to fix this now (if a way exists)? Thanks!

You can find this information by reading the comments above, but since I have just spent a considerable time reading them all myself and more people might start looking at the latest comments first, here is the gist:

Thunderbird still honors your LC_TIME but it has changed how its value is interpreted; in the past Thunderbird used the locale definition found under that name on your system, now it looks up data like the date format in the Common Locale Data Repository (CLDR), which also has a locale named "en_DK" but with a different date and time format. This format is what you see now in Thunderbird.

It sounds like the general change in locale resolution has made things easier in terms of code maintenance and cross-platform consistency; and since CLDR is considered a reputable source for the locale definitions, the issue of this ticket is not actually considered a bug by the developers, or at least not one in Thunderbird. It would have to be fixed in the CLDR.

Now, for a solution that can be applied by the users right now, the best way seems to be setting LC_TIME to a pseudo locale called "root", which is provided by the CLDR and actually uses ISO 8601 date and time format. "root" does not exist as a locale on your system but you can just create it as a symlink to en_DK. See comment 59 for detailed steps.

Comment 152

[Tristan Miller]

(In reply to Eric Toombs from comment #102)

In case it's too hard to find, the post with title "WORKAROUND" is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c88

Eric, you are referring someone who says he is using openSUSE to a post containing an Arch-specific workaround. That workaround doesn't work on openSUSE.

(In reply to Bernhard Groll from comment #151)

You can find this information by reading the comments above, but since I have just spent a considerable time reading them all myself and more people might start looking at the latest comments first, here is the gist:

And Bernhard, the workaround from Comment 59 that you recommend is also distribution-specific. As others have already reported here, it doesn't work on Fedora and it doesn't work on openSUSE. The commenter you are responding to doesn't mention what system he is using, so it's impossible to say whether your recommendation will help him.

Using this Bugzilla issue to document workarounds for all the different systems supported by Thunderbird is not particularly efficient. It's very hard for users to find these workarounds in this gargantuan thread, and it is very hard for them to determine whether or not any given workaround is applicable to their system. Besides this, devising, critiquing, and improving these workarounds is probably not an intended use of the issue tracker. It would be better if we could set up an editable list of workarounds, and an associated discussion on same, on a wiki somewhere, and then direct users to it when they ask for a workaround. But I'm not sure if there's any existing wiki that would be appropriate for this; the most appropriate ones I know of, like the official Arch and openSUSE wikis, are all distribution-specific.

Comment 153

[Tristan Miller]

I should clarify that when I say "that workaround doesn't work", I'm not referring to the general idea of setting up a new locale, but rather to the detailed series of steps for creating and registering such a locale. Different *nix distributions put the locales and associated configuration files in different places, and use different commands for compiling/registering/enabling these locales. Very experienced users may be able to read, say, a set of Debian-specific instructions and, by consulting the relevant documentation for Debian and for their own distribution, and with a little (or a lot of) trial and error, successfully adapt those instructions to their own system. But most people coming to this thread probably lack the knowledge, time, or patience to do this. And among the rest, not all of them are going to come back and post their adapted workaround as a clear and detailed series of instructions.

Comment 154

[haarp]

Also affected.

On older Thunderbirds, LC_TIME=en_DK.UTF-8 would display a weird time format using . as hour/minute separators, similar to what Eric Toombs experienced in comment 84.

On Thunderbird 68.2.2, I am getting the hybrid date format experienced by everyone in this bug report. sv_SE.UTF-8 only partially solves the problem, as long date formats with the weekday would use ths Swedish name for the day.
However using LC_TIME=en_DK (without .UTF-8), which is NOT installed on my system, makes Thunderbird complain:

Gtk-WARNING **: 16:14:12.189: Locale not supported by C library.
Using the fallback 'C' locale.

but use the correct ISO-8601 date format regardless! Very weird. Indeed, Thunderbird must have some terrible broken internal mangling of locales to come to that conclusion.

Anyway, LC_TIME=en_DK works.

Comment 155

[baptx]

@haarp thanks for the hint. On Linux, adding LC_TIME=en_DK in /etc/default/locale did work for me also but only if en_DK was not installed. It also worked with LC_TIME=C.UTF-8 which I think is a better solution since it removes the warning "Using the fallback 'C' locale." when starting Thunderbird and it will not produce a warning when you run the command "update-locale". It will also work even if you have en_DK locale installed or not. It looks like en_DK support was removed in Thunderbird?
In Firefox, I noticed an interesting behavior that may be related to this Thundebird bug. Usually, if we use an English version of Firefox, and want to use an HTML input type=time with a 24-hour clock instead of AM/PM (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/time), we need to set intl.regional_prefs.use_os_locales to true in about:config. It will follow the operating system locale and also use a date like DD/MM/YYYY instead of MM/DD/YYYY (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/date).
However if we use LC_TIME=C.UTF-8 (or LC_TIME=en_DK without installing en_DK locale), we don't need to set intl.regional_prefs.use_os_locales to true, the operating system locale is selected.
I noticed a weird behavior also with the preference intl.locale.requested in Firefox. If we set it to "en" (without quotes), while using LC_TIME with en_DK.UTF-8 or en_IE.UTF-8 for example, we don't need to set intl.regional_prefs.use_os_locales to true, in order to select the operating system locale.
When using LC_TIME=C.UTF-8 (or LC_TIME=en_DK without installing en_DK locale), we can set intl.locale.requested to "en" in order to use the web browser locale instead of the operating system locale.

Comment 156

[Jeff Harp]

Attachment: Date/time formatting modifications

Here's a patch that implements a more robust, two-tiered solution to this problem. It significantly improves upon what I proposed in my earlier comment (113), and takes into account the concerns Zibi raised in comment 127.

This approach provides simplicity for users who just want the ISO formatting (as associated with the root locale), but also provides a way for advanced users to customize every aspect of date/time formatting to address their particular needs.

All the preferences have a common prefix of "intl.locale.date_time.pattern_override".

The first tier is a simple boolean preference named "root". When true, date/time values are formatted based on the CIDR root locale. This provides the ISO formatting others have requested.

The second tier is a boolean preference named "custom", along with 11 string preferences that hold patterns (7 for date, 3 for time, and 1 for the connector) that are used to implement fully custom date/time formatting. The default values for the 11 string preferences are based on the CIDR root locale and hard coded values found in DateTimeFormat.cpp.

When both of the new boolean preferences are false, date/time formatting takes place normally, based on the user's platform locale.

This patch modifies 3 files:

intl/locale/DateTimeFormat.cpp
intl/locale/DateTimeFormat.h
modules/libpref/init/all.js

These changes are outside the "comm" folder, so they impact Firefox and other projects as well. I don't know what that means in terms of how one would go about submitting a patch like this for formal consideration, as I have no prior experience dealing with Mozilla as a developer.

For what it's worth, the patch works nicely with Thunderbird 68.4.1 and Firefox 74.0.1, the versions available from the Ubuntu Linux 18.04 LTS repositories, and which I use daily.

Comment 157

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thank you for the patch Jeff! I'll review it shortly!

Comment 158

[Zibi Braniecki [:zbraniecki][:gandalf]]

This patch looks really good!

== Big picture ==

Unfortunately, the code you're reworking is really old, and out of date with how Gecko intl works today.
If it was only about altering code, I'd be ok with updating the code as is, and keeping the refactors for later but because your patch introduces preferences, and preferences are tricky to maintain/deprecate, I'd like to iron out the code before we introduce prefs.

To put things in context - I'm working on a project that is meant to introduce Rust backed DateTime formatting, which will hopefully replace ICU uses in long term and give us much better performance.
The API is going to be fairly well aligned with ICU and ECMA402 tho, so all alignments we can do today to get this class more in line with those modern APIs will serve us well when we switch to Rust.

== Patch comments ==

Comments:

1. I think it would be better if you pulled root patterns from ICU rather than hardcoding them in CPP code.
2. I think the particular custom prefs should not be set by default at all. If the user wants to set one, they would add it (or the UI in Preferences would add it).
3. I'm not sure if there's a value in the boolean flags. A set preference is a signal in itself. Therefore, I think I'd prefer to limit that to one pref per option. I understand that the ergonomics won't be perfect, but we are addressing an edge case, so I think it's okay to make such tradeoff.
4. Unless there's a strong use case, I'd really love to avoid having to also support overriding the connector.
5. I think we should revisit what aDateFormatSelector and aTimeFormatSelector we support.

The fifth point warrants a separate bug I believe, but I'm going to provide the rationale here.
We currently support 13 different selectors, and many of them are obsolete or misguided. I'd like us to coalesce around the dateStyle/timeStyle proposal - https://github.com/tc39/proposal-intl-datetime-style - or a subset of it (long/short for example).

This would give us a number of benefits:

• matching consistent styles between JS and C
• matching Ecma402 standards
• good foundation for in-OS hooks so that we can take those styles from OS later

I can see us starting with none/short/long for both date and time for now, and that would narrow it down to 6.

Here's an example of how we do it for localization needs. Those two pieces could be merged then - https://searchfox.org/mozilla-central/source/intl/l10n/FluentBundle.cpp#267-312

As for the others:

• kDateFormatMonthLong and kDateFormatWeekday are not patterns. They're symbols. We should have DateTimeFormat::GetSymbol that uses https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1DateFormatSymbols.html to return weekday/month in styles long/short/etc.
• kTimeFormatSeconds and kTimeFormatNoSeconds should be converted to kTimeFormatLong and kTimeFormatShort
• kDateFormatYearMonth is not used in Firefox and I hope neither is in Thunderbird so we should be able to just remove it.
• kDateFormatYearMonthLong is used in one place in Firefox. I'd like to suggest that we either introduce DateTimeOptions bag like ECMA402 does, or, for simplicity, start with a variant of the method that take a skeleton string instead of styles. A skeleton is a semantically concise version of an options bag, so the caller in Firefox's NavHistory would do DateTimeFormat::FormatPRExplodedTime("yyyyMMMM", &aTime, monthYear); since they want this level of precision.

One we have this, we would introduce the following four prefs:

intl.locale.date_time.pattern_override.date_long
intl.locale.date_time.pattern_override.date_short
intl.locale.date_time.pattern_override.time_long
intl.locale.date_time.pattern_override.time_short

If those values are not set, use Gecko logic. If they're set to root, use ICU's root patterns, if they're set to a non-root string, use it as a pattern.

== Proposal ==

What does it mean for this patch?

If this big-picture proposal sounds good to you, I suggest that we:

1. File a bug to extend the API with GetSymbol and skeleton
2. File a bug to clean up aDateFormatSelector and aTimeFormatSelector to use the new methods
3. Update the patch in this bug to use the new conventions from (1) and (2)

How does the plan sound to you Jeff?

Comment 159

[Len Weisberg]

It's exciting to see what looks like major progress toward solving the
problem addressed by this bug. Just a reminder of a CLDR ticket I filed
that could perhaps someday mean a replacement for the special name "root".
But do note that the Unicode website has been moved and reorganized
and my ticket is now at:
https://unicode-org.atlassian.net/browse/CLDR-12027
"CLDR should support (some variants of) ISO-8601 date formats."

Comment 160

[Jeff Harp]

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #158)

How does the plan sound to you Jeff?

Thanks Zibi. I appreciate your quick and thorough response.

I honestly was viewing the submission of my patch as an endpoint, rather than a starting point. I modified the current code to address what I (and others) consider to be a serious bug, whether or not Mozilla considers it a bug. I was doing it as a user who happens to be a developer, not because I was seeking to become more involved in contributing to Mozilla code development. And I was hoping the patch would be accepted with minimal changes so that users could benefit from it sooner, rather than later.

I stopped actively developing software about two years ago to become primary caregiver for my elderly parents. That's a 24/7 "job" allowing little time for anything else. And right now my father is critically ill in hospital with a systemic bacterial infection and fighting for his life.

So I'm unable to contribute further at this time. I hope someone else will jump in and pick up where I left off. If I'm able, I may work on this again in the future, but I currently have no idea when that might be.

Comment 161

[Zibi Braniecki [:zbraniecki][:gandalf]]

So I'm unable to contribute further at this time. I hope someone else will jump in and pick up where I left off. If I'm able, I may work on this again in the future, but I currently have no idea when that might be.

Thank you for your response! I completely understand!

I wasn't asking for you to take on fixing this, rather for your opinion on my proposal on how to fix it. :)
If we agree on the approach and an end point, I can file bugs and hopefully either find time to work on it myself, or flag the bugs as good first bugs for other contributors to take!

Comment 162

[thatdoesntworkhere]

Zibi -> Please-Please tell me this is still proceeding !
We must stop this madness of setting time/date format based on locale/language preferences or hooks to a quagmire of internal OS settings.
In the age of ISO the idea that my mailbox time/date formatting must conform to UK/English, French/Canada, etc. is beyond frustrating.
And sadly with the multitude of Unix derivatives adopting all forms of non-standard, personalized desktops you can't depend on the time displayed on the desktop or many of the former default OS/Desktop system-time hooks being correct or even existing. SystemD, Cinnamon, etc. I have wandered the internet and found a myriad of complaints and a lot of unsuccessful answers.
If there is any way to create an interface where clients can select their own preferred time/date format options, check the result and then possibly adjust for time zone offset that would be great. I would then use that interface for other custom applications as well.

• sign me up for testing !!
  Another annoyance I did find a "fix" for but should be an interface selectable option; the "Date" field for mail received today only shows the time. Why ?? Why ??

Comment 163

[Zibi Braniecki [:zbraniecki][:gandalf]]

Zibi -> Please-Please tell me this is still proceeding !

As you can see, there's no assignee for this bug. That means that it is not proceeding, but it is available for grabs.

There's some conversation on privacy of Intl APIs in https://github.com/tc39/ecma402/issues/443 which may result in us closing the gap between date/time settings we use for internal APIs and expose to the Web. If that will be the case, we'll be able to expose your Windows/MacOS/Linux/Android internationalization preferences to the Web APIs, which would help a lot.
But that's a slow process unfortunately, and we currently do not have available engineers to work on that :(

Comment 164

[Ken Fallon]

Does anyone know how to change this page ? "http://kb.mozillazine.org/Date_display_format"

It contains information about the no longer working workaround, and it would be more honest if there was a redirect to this bug.

This page was last modified 13:44, 13 February 2017. This page has been accessed 476,809 times.

Comment 165

[latgarf]

As a user, because of this bug alone, I'm quitting T-Bird after many years of use. Sylpheed is my new client. Inability to have the YYYY-MM-DD date format is ridiculous.

Comment 166

[u634361]

Thanks Latgarf for the pointer, Sylpheed is great and fast!! and it allows you to configure things the way you want, it took me 20 seconds to display ISO dates:)
Don't be too hard on Thunderbird, this is how almost all projects end up, stale and boring. Once the money men get involved the software ends up telling you how to work, instead of enabling you to work the way you want. There are rumours about a fork, removing the bloat (who uses calendar?) and the privacy / security issues, but there always are, I love open source :)

Comment 167

[Ken Fallon]

You might also consider Kmail, and Claws mail both of which support setting custom dates.

Comment 168

[Zibi Braniecki [:zbraniecki][:gandalf]]

As a user,

Bugzilla is not intended for "as a user" type of communication. Please, read the Bugzilla Etiquette - https://bugzilla.mozilla.org/page.cgi?id=etiquette.html

If you're interested in continuing work on the patch and proposal from comment 158, feel free to take this bug, it's currently unowned.

Comment 169

[Erik Johnson]

I've just updated to 78.3.1 (64-bit) and now my LC_TIME locale settings are no longer recognised, again. Comment #144 did work again for the last year up until now

Comment 170

[Dan Minor [:dminor]]

Jeff, thank you for the patch! I'll continue the work and hopefully get this landed.

Comment 171

[Dan Minor [:dminor]]

Try job here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=30560324f1e554b8d94059a5ea07ad457f37b9c7

Comment 172

[Dan Minor [:dminor]]

Attachment: Bug 1426907 - Call OSPreferences::GetDateTimePattern once in FormatUDateTime; r=zbraniecki!

Now that we've removed a lot of the special cases we were supporting in the date
and time format selectors, we can call GetDateTimePattern once with the two
selectors instead of calling it twice and combining the results. This will
simplifies the code and will make it easier to handle overriding patterns using
prefs.

A side effect of this change is that on OS X, you get a slightly different result
if you ask for the long format date and time all at once than if you ask for the
two separately and then combine them. The test expectations have been updated
accordingly.

Comment 173

[Dan Minor [:dminor]]

Attachment: Bug 1426907 - Make OSPreferences::GetPatternForSkeleton public and use in DateTimeFormat; r=zbraniecki!

This allows for the removal of duplicated code between DateTimeFormat and
OSPreferenes.

Depends on D94431

Comment 174

[Dan Minor [:dminor]]

Attachment: Bug 1426907 - Add OSPreferences::OverrideDateTimePattern method; r=zbraniecki!

This new method allows for the user to override the long and short date and
time patterns by prefs. If no overrides are set or it fails while processing
the prefs, it will fallback to the existing methods for determining the patterns.
Since the user may have only overriden one of date or time, it is necessary to
be able to look up the other separately and combine the results.

This adds a prefix callback that watches the new prefs and flushes the cache if
they change. Otherwise, the user would have to restart the browser to see the
results of changing a pref, and would make testing more difficult. Unregistering
this callback required changes to the destructor, which was previously defined
separately on each operating system. A new RemoveObservers method has been added
to handle OS specific cleanup.

Depends on D94432

Comment 175

[Dan Minor [:dminor]]

Updated try run here: https://treeherder.mozilla.org/#/jobs?repo=try&selectedTaskRun=LGHwQwTZQi-_J0szU9zyFw.0&revision=5e9993c0c0f664fec150a1f3cd4f3f5dd7f26cd4 and windows fixed here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad9c3b00064b764271ec7897cc9f862d3be9136f.

Comment 176

[Dan Minor [:dminor]]

Attachment: Bug 1426907 - Rework DateTimeFormat gtests to not rely on hard-coded connector patterns; r=zbraniecki!

The connector patterns can change over time, we should instead check for the date time components
that we expect to be present.

Comment 177

[Dan Minor [:dminor]]

Attachment: Bug 1426907 - Update OSPreferences API to use UTF-8 rather than UTF-16; r=zbraniecki!

We'll want to use UTF-8 when we switch to using ICU4x because Rust is all UTF-8. We
can switch the external facing APIs now, and update the internal implementations
later.

Depends on D94432

Comment 178

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7774282861fe
Rework DateTimeFormat gtests to not rely on hard-coded connector patterns; r=zbraniecki
https://hg.mozilla.org/integration/autoland/rev/4602056737ce
Call OSPreferences::GetDateTimePattern once in FormatUDateTime; r=zbraniecki
https://hg.mozilla.org/integration/autoland/rev/e1137eef3ae3
Make OSPreferences::GetPatternForSkeleton public and use in DateTimeFormat; r=zbraniecki
https://hg.mozilla.org/integration/autoland/rev/1a5d1964ff10
Update OSPreferences API to use UTF-8 rather than UTF-16; r=zbraniecki
https://hg.mozilla.org/integration/autoland/rev/5a4ccc7684fb
Add OSPreferences::OverrideDateTimePattern method; r=zbraniecki

Comment 179

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/7774282861fe
https://hg.mozilla.org/mozilla-central/rev/4602056737ce
https://hg.mozilla.org/mozilla-central/rev/e1137eef3ae3
https://hg.mozilla.org/mozilla-central/rev/1a5d1964ff10
https://hg.mozilla.org/mozilla-central/rev/5a4ccc7684fb

Comment 180

[Niklas Hambüchen]

Fantastic!

Can you advise if / when a nightly is out that includes the fixes so I can try it?

Thanks!

Comment 181

[Dan Minor [:dminor]]

It looks like it is in today's nightly, from [1] it looks like it was built from https://hg.mozilla.org/mozilla-central/rev/83bf4fd3b1fbca9dcbe23de9d1a1503eed62569a, which according to [2] would include these changes.

Please test and let us know if you encounter any issues. Thanks!

[1] http://ftp.mozilla.org/pub/thunderbird/nightly/2020/10/2020-10-28-10-21-40-comm-central/thunderbird-84.0a1.en-US.linux-i686.txt
[2] https://hg.mozilla.org/mozilla-central/pushloghtml

Comment 182

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thank you :dminor for taking ownership of this and fixing this long standing issue!

@all, This is just the platform scaffolding for the custom pattern overrides. It should allow Thunderbird and/or Firefox to write front end UI in Preferences to override patterns. It may also be possible to extend WebExtensions to allow extensions to override those.

For now, if you want to test it, you need to understand patterns: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

And then add some or all of the following settings with your patterns of choice:

• intl.date_time.pattern_override.time_(short | medium | long | full)
• intl.date_time.pattern_override.date_(short | medium | long | full)

Comment 183

[Shane Kerr]

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #182)

Thank you :dminor for taking ownership of this and fixing this long standing issue!

@all, This is just the platform scaffolding for the custom pattern overrides. It should allow Thunderbird and/or Firefox to write front end UI in Preferences to override patterns. It may also be possible to extend WebExtensions to allow extensions to override those.

For now, if you want to test it, you need to understand patterns: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

And then add some or all of the following settings with your patterns of choice:

• intl.date_time.pattern_override.time_(short | medium | long | full)
• intl.date_time.pattern_override.date_(short | medium | long | full)

So good to see progress on this long-standing annoyance! Yay!!!

I downloaded the nightly Thunderbird and updated the prefs.js file for it to include the following:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm:ss Z (EEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm:ss Z");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm:ss");
user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm:ss Z (EEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm:ss Z");
user_pref("intl.date_time.pattern_override.time_medium", "HH:mm:ss");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

I wasn't sure where each of the various pattern overrides were applied, so this is just a guess (a big unknown for me is when the date override is used versus the time override).

This gets very close to an ISO 8601 format, so is a nice improvement. Unfortunately the date display in both the list of mails and in the mail display itself is still not completely correct, as for some reason it places a comma between the date and time, like this:

2014-12-10, 19:04

I guess that it's using something like "short date", "short time" which is possibly reasonable but what I really want is them concatenated without the comma in between. A slightly adjusted version of the preferences is even closer:

user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd HH:mm:ss");
user_pref("intl.date_time.pattern_override.time_short", " ");

(Note that there has to be a space in the time_short override, otherwise it is ignored and the default time formatting is used.)
This results in a time/date format with a comma at the end, like this:

2014-12-11 18:31:26,

This is probably good enough, but being able to get rid of that comma would complete the ISO 8601 ascension. :-D

Comment 184

[Dan Minor [:dminor]]

Thank you for testing this :)

The problem is that we apply the overrides for date and time separate and then combine them by looking up the locale specific "connector" which in your case appears to be ,. I think the solution would be to add another pref which allows the user to override the connector as well.

I had deliberately ignored empty values for the overrides, so that the prefs could exist but not have an effect. I hadn't thought about the use case of putting the entire pattern in a single override, but if I implement the connector override, this shouldn't be necessary.

I'll check with Zibi, but I don't see any problems with adding an override for connectors.

Comment 185

[Dan Minor [:dminor]]

I filed a follow up in Bug 1674212 to add an override pref for the connector pattern.

Comment 186

[Len Weisberg]

First, many many thanks for doing this! One question:

The date in the message-list pane for messages between 1-7 days old has
(for me) the format: EEE HH:mm eg: Thu 14:03
Will I be able to customize this format? I don't see it in any of
the example preferences.

Comment 187

[Len Weisberg]

To continue my previous comment, #186:
BTW, what I would like for myself is EEE-d HH:mm . I admit this is
just a personal preference, but this brings up another point:

I would like to say how much I love the approach of just letting the user
design the format for themself, avoiding all the endless decision-making
about the exact ISO-like variations to offer.
This solution strikes me as 100% kludge-free! THANKS!

Comment 188

[Shane Kerr]

(In reply to Dan Minor [:dminor] from comment #185)

I filed a follow up in Bug 1674212 to add an override pref for the connector pattern.

Great Dan, thank you very much. When a build arrives with that additional setting I'll give that a test.

I'm looking forward to having these date formatting options available! :-D

Comment 189

[jmkrnet]

Good to see, this is hopefully solved by giving users a possibility to configure custom date/time patterns (instead of an endless debate how to name the date/time format everybody wants:).

Unfortunately for us poor Debian users we now are stuck on TB 78 ESR:

• This means I cannot benefit form the custom date/time patterns solution.

• It also means the "LC_ALL=root.UTF-8 thunderbird" workaround I use no longer appears to function. (Details of this workaround are described in various comments to this bug (comment 58+59, 74, 94, 113, 130, 138, 144, 151) and I also have my exact details described in bug 1502659 comment 2.)

Is there any solution/workaround suitable for TB 78 ESR?

Comment 190

[Jonathan Kamens]

(In reply to Bakhelit from comment #189)

Unfortunately for us poor Debian users we now are stuck on TB 78 ESR:

There's no reason Debian users can't download the newest TB release from thunderbird.net, unpack it locally, put it at the front of their search path, and use it instead of the Thunderbird that ships with Debian.

Comment 191

[Tristan Miller]

(In reply to Jonathan Kamens from comment #190)

There's no reason Debian users can't download the newest TB release from thunderbird.net, unpack it locally, put it at the front of their search path, and use it instead of the Thunderbird that ships with Debian.

Well, except that distributions tend to patch packages to fix various compatibility issues and to streamline integration with the rest of the OS. (openSUSE, which I use, applies 26 such patches to the current stable release of Thunderbird. I have no idea what Debian does.) But using the official release is worth a try, provided you first back up your profile. Any change/loss in functionality you may encounter may well be easier to live with than the date formatting issue.

Comment 192

[Jonathan Kamens]

I'm sorry, I could have been clearer. While you're correct that distributions do tend to include minor patches in their builds of applications like Thunderbird, I have personally used the version of Thunderbird distributed by the Thunderbird team on Debian-based systems on many occasions and I have not encountered any significant compatibility issues.

Comment 193

[jmkrnet]

(In reply to Jonathan Kamens from comment #192)

I'm sorry, I could have been clearer. While you're correct that distributions do tend to include minor patches in their builds of applications like Thunderbird, I have personally used the version of Thunderbird distributed by the Thunderbird team on Debian-based systems on many occasions and I have not encountered any significant compatibility issues.

I prefer to have all software on my systems installed from Debian repositories, because it makes it possible to handle all software installations, removals and especially updates using one management system which greatly simplifies its configuration and scheduling of software updates.

My current systems for example handle all updates automatically without any user intervention and usually without any need to change configs of installed applications during the whole Debian stable release lifetime. In fact with the exceptions of Firefox and Thunderbird which in recent years often totally disrupt my configs with each ESR update, I usually get away with updating my applications configs only when switching to new Debian stable release. So, thanks for doing ESR, without it I would probably need to switch to some other browser and email client, because dealing with disrupted configs each time Mozilla's rapid release machine forces new non-ESR version on us would be unbearable for me.

Thus, if there is no chance of fixing TB 78 ESR I will have to accept it and wait for next TB ESR in Debian repositories. Still, I would like to know why the "root" locale workaround does not work any more? Did Unicode CLDR database removed the "root" fallback? Or did Mozilla change the locale handling again to for example force the default "en-US" instead of "root" locale?

Comment 194

[Jonathan Kamens]

(In reply to Bakhelit from comment #193)

I prefer to have all software on my systems installed from Debian repositories, because it makes it possible to handle all software installations, removals and especially updates using one management system which greatly simplifies its configuration and scheduling of software updates.

That is certainly a reasonable preference, and I strive to operate the same way myself.

However, that's not the same as "Unfortunately for us poor Debian users we now are stuck on TB 78 ESR." You're not. There is a workaround for that, which you can avail yourself of should you choose to do so. It's your choice not to, a perfectly reasonable one, but I don't think it's reasonable to portray that here as a trap with no way out.

It's just a fact of life that it's simply impossible for Thunderbird's release and bug-fix schedules to always align in the most optimal way with Debian's (or any other OS's).

The maintainers of Thunderbird put a lot of effort into putting out a lot of binary releases for a lot of platforms. I don't think it's particularly polite or generous to imply that those don't exist simply because you personally prefer not to stray from the packages distributed by your OS maintainer.

Comment 195

[jmkrnet]

Sorry, I could wrote something less dramatic instead of "Unfortunately for us poor Debian users we now are stuck on TB 78 ESR." - at that time I could not see that it could be seen as ungrateful, impolite or whatever.

I was interested in finding out if there is a chance to consider using the fix from this bug also for TB 78 ESR. If it is not technically possible or is too much work, I was hoping to at least learn how to change/update my workaround for TB 78 ESR or if it is even possible at all.

I certainly do not want to imply no alternative options exist or that devs and maintainers of TB or FF do their job badly - quite the opposite I am sure they try do their best. I am also aware that even if they did ther job badly, users have no "moral" right to complain since many devs and maintainers do their work for free. So, I am of course very greatful for the work all TB and FF devs and maintainers do - that is why I at least try to help by reporting bugs/problems I find when configuring my TB and FF. I hope this explains my views better:).

Comment 196

[[e-mail]]

For people wanting to test this awesome new feature, it wasn't immediately clear how to do so on Ubuntu given that thunderbird-trunk is no longer in ppa:ubuntu-mozilla-daily/ppa. Downloading the latest binary from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ (I downloaded 89.0a1) and adding the lines per https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c183 to the active profiles prefs.js file does the trick. Date formats are lovely:

2021-04-05, 20:02
00:17

I am not following how to set the connector override to eliminate the comma. That would be a readability improvement. This is an awesome usability enhancement.

The message pane seems to only support date_short, time_short and time_short, which is fine by me, this is my preference.

Calendar events seem to take the Date Text Format assigned, but despite adding the 8 user_pref definitions for time/date short/medium/long/full (and seeing them in config editor), only short and long are available (but "long" is actually the format specified as "full", including the day of week). This is also fine given the formatting overrides.

I've configured mine like:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm");
user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.time_medium", "HH:mm");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

The patterns are parsed as expected - YAY!!

One quirk is that upcoming events in the today pane duplicate the time - as if the display is date_full time_short rather than just the date format ("long/full") specified in the preferences.

-David

Comment 197

[baptx]

Why is the LC_TIME=C.UTF-8 workaround not working anymore to display the international date format YYYY-MM-DD when replying to emails? I am using the deb package Thunderbird 78.7.1 (64-bit) from Lubuntu 20.04.
I saw other workarounds but they are only compatible with the beta / nightly version and according to https://www.thunderbird.net/en-US/organizations/, the stable version of Thunderbird is the ESR (currently version 78) unlike Firefox. I would not risk using the beta or nightly version for writing professional emails, there was already a terrible privacy issue with the stable version of Thunderbird in the past (https://bugzilla.mozilla.org/show_bug.cgi?id=1201782#c29), so I would recommend only using the beta or nightly version for testing without private content.

Comment 198

[José M. Muñoz]

Please note that the connector override introduced in bug 1674212 in pref intl.date_time.pattern_override.date_time_short was renamed to intl.date_time.pattern_override.connector_short in bug 1706318 for consistency. There are also medium, long and full connectors, but they are currently not used.

Comment 199

[Thomas D. (:thomas8)]

Let's expose the awesome work which has happened here for the annals of history and get the record right!

Starting out from a rather peripheral change of behaviour of the output format when using LC_TIME=en_DK.utf8 (was that even a bug?), this bug has matured into a full-fledged RFE which has implemented much-requested pref overrides for date and time formats:

• intl.date_time.pattern_override.date_*
• intl.date_time.pattern_override.time_*
  Substitute * with any of short/medium/long/full. The short connector is used for all formats:
• intl.date_time.pattern_override.connector_short (implemented in bug 1674212, renamed in bug 1706318).

Patches landed on mozilla-central so this really belongs to Core: Internationalization, of which Firefox and Thunderbird are consumers.

Comment 200

[Thomas D. (:thomas8)]

I would like to thank everyone involved on this bug for implementing these awesome date and time formatting prefs, especially Dan M., Zibi, Jörg K., Jeff H., and Niklas H.! Great job! 👍🏻👍🏻👍🏻

Comment 201

[José M. Muñoz]

If comment #199 was meant to be a summary, it's missing the information from comment #198:

• intl.date_time.pattern_override.date_*
• intl.date_time.pattern_override.time_*
  substitute * with any of short/medium/long/full. The short connector is used for all formats:
• intl.date_time.pattern_override.connector_short

Comment 202

[Thomas D. (:thomas8)]

(In reply to José M. Muñoz from comment #201)

If comment #199 was meant to be a summary, it's missing the information from comment #198:

Thank you! I've updated my comment accordingly. intl.date_time.pattern_override.connector_short was not implemented here in bug 1426907, but it's good to have the full summary here.

Comment 203

[Dan Minor [:dminor]]

(In reply to Thomas D. (:thomas8) from comment #200)

I would like to thank everyone involved on this bug for implementing these awesome date and time formatting prefs, especially Dan M., Zibi, Jörg K., Jeff H., and Niklas H.! Great job! 👍🏻👍🏻👍🏻

Thank you! 😀

Comment 204

[Corey Hickey]

I'm somewhat mystified by much of the discourse within this bug, but the end result looks pretty understandable. I am looking forward to when this ends up in my distribution's Thunderbird package.

Thank you, very much, as well.

-Corey

Comment 205

[Thomas D. (:thomas8)]

(In reply to [e-mail] from comment #196)

This is an awesome usability enhancement.

I've configured mine like:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm");

I think that's not how you're supposed to use it. All the pattern_override.date_* prefs are only supposed to have date placeholders, not also time placeholders. Adding both date and time into the date pattern override looks like a recipe for UI failures imo. Someone correct me if I'm wrong.

user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm v");

Same here - pattern_override.time_* should have times only, not dates also.

user_pref("intl.date_time.pattern_override.time_medium", "HH:mm");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

The patterns are parsed as expected - YAY!!

Given your setup, that's quite surprising... probably just using the your short formats, which happen to be correct.

One quirk is that upcoming events in the today pane duplicate the time - as if the display is date_full time_short rather than just the date format ("long/full") specified in the preferences.

So then the duplicate time should be the result of your wrong setup, as explained above.
HTH.

Comment 206

[Thomas D. (:thomas8)]

Attachment: Screenshot: How to create date/time format override prefs including connector in Thunderbird's own Config Editor

(In reply to [e-mail] from comment #196)

For people wanting to test this awesome new feature, it wasn't immediately clear how to do so on Ubuntu given that thunderbird-trunk is no longer in ppa:ubuntu-mozilla-daily/ppa. Downloading the latest binary from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ (I downloaded 89.0a1) and adding the lines per https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c183 to the active profiles prefs.js file does the trick.

Thanks David for providing this helpful first explanation.

The feature is currently available in Thunderbird Beta 90.0b2 and Daily 91.0a1, both available for download from Thunderbird homepage.

How to create date and time format override prefs using Thunderbird's Config Editor

It is possible, but not required to edit prefs.js directly - you can just use Thunderbird's inbuilt config editor and create these prefs.
≡ > Preferences > General > [Config Editor...] button at the very bottom.

Type the full name of the pref which you want to create into the search box:
intl.date_time.pattern_override.date_short
As the pref does not exist, it will show up in the results list as a new pref which you can create by first selecting (o) String and then using the the [+] button (marked purple in the screenshot). It will then ask you for a value, and you can enter yyyy-MM-dd (you can always edit the value again later). Then do the same by analogy for intl.date_time.pattern_override.time_short.

Date formats are lovely:

2021-04-05, 20:02
00:17

I am not following how to set the connector override to eliminate the comma. That would be a readability improvement.

How to change the date/time connector (e.g. from comma to space)

As seen in the screenshot: The connector must have date and time placeholders in curly brackets.

• Date: {1}
• Time: {0}
• Strings must be single-quoted to avoid parsing, but some simple characters like space () or comma (,) work without quoting.
• Apparently the format follows https://unicode.org/reports/tr35/tr35-dates.html#Date_Time_Combination_Examples

intl.date_time.pattern_override.connector_short = {1} {0} (single space between placeholders)
Result for short date and short time combo (see screenshot):
2021-06-24 21:00

Regular ASCII characters which you want to display must be single-quoted (otherwise the date-time display may get truncated):

intl.date_time.pattern_override.connector_short = {1}'T'{0}
Result for short date and short time combo:
2021-06-24T21:00

Hope that helps!

Thomas

Comment 207

[Mikko Rantalainen]

Apparently the pattern format is documented here: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

We still need documentation for the overrides. Guessing from the names, I'd assume that intl.date_time.pattern_override.time_* is time only (as in clock, e.g. 23:59) and intl.date_time.pattern_override.date_ is supposed to be date (as in "2021-06-28") without a time and intl.date_time.pattern_override.connector_short defines how those should be combined. Default seems to be {1}, {0} meaning that you get date followed by comma, space and time. The actual implementation seems to require both {0} and {1} so you cannot customize that too much: https://hg.mozilla.org/mozilla-central/file/tip/intl/locale/OSPreferences.cpp

The expected mapping for e.g. E vs EE Vs EEE vs EEEE vs EEEEE vs EEEEEE should be defined somewhere for Thunderbird.

Also, how to combine short date with long time for the message reply header?

Comment 208

[[e-mail]]

This link seems to be the maintained ICU documentation page for CLDR date symbols:
https://unicode-org.github.io/icu/userguide/format_parse/datetime/#date-field-symbol-table

H(In reply to Thomas D. (:thomas8) from comment #206)
Fixing my preferences so I think they comply with comment 206, (thanks Thomas, it helped a lot!)

intl.date_time.pattern_override.connector_short	{1} {0}	
intl.date_time.pattern_override.date_full	yyyy-MM-dd (EEEEEE)
intl.date_time.pattern_override.date_long	yyyy-MM-dd	
intl.date_time.pattern_override.date_medium	yy-MM-dd	
intl.date_time.pattern_override.date_short	MM-dd	
intl.date_time.pattern_override.time_full	yyyy-MM-dd HH:mm Z z
intl.date_time.pattern_override.time_long	yyyy-MM-dd HH:mm z
intl.date_time.pattern_override.time_medium	HH:mm Z
intl.date_time.pattern_override.time_short	HH:mm

I notice that only the short form of time seems to be used. As in comment 207 above, there are some limits to using these new options in the calendar. The only mailbox pane guidance I can find is this http://kb.mozillazine.org/Date_display_format - which also doesn't reference long/short time. With the above time.pattern_override settings and experimenting with the preference mail.ui.display.dateformat.today values I get the following results:

Value	Example
0	16:44
1	2021-06-28 16:44
2	21-06-28 16:44
3	16:44
4	16:44

It'd be awesome to add the following to make full use of this lovely feature:

Preference	Example Value	Example result
mail.ui.display.dateformat.today	{}{short}	16:44
mail.ui.display.dateformat.thisweek	{short}' '{medium}	06-18 16:44 -0700
mail.ui.display.dateformat.default	{long}' '{medium}	2021-06-18 16:44 -0700
calendar.date.format	{full}' '{long}	21-06-18 (Tu) 16:44 (PDT)
mailnews.reply.dateformat	{long}' '{full}	2021-06-18 16:44 -0700 (PDT)

Comment 209

[José M. Muñoz]

Sadly you are correct, mail.ui.display.dateformat.today values 3 and 4 don't work any more :-(

They had fallen under the knife for a short while in TB 53, but got reimplemented in bug 1329841. Internally they related to kDateFormatYearMonth and kDateFormatWeekday. Now they've been removed again in bug 1669573.

Comment 210

[José M. Muñoz]

... removed in bug 1669573 and bug 1671532.

Comment 211

[José M. Muñoz]

Testing the remaining values of mail.ui.display.dateformat.today
0 gives S13:04 with intl.date_time.pattern_override.time_short set to 'S'HH:mm
1 gives L2021-06028, S13:04 with intl.date_time.pattern_override.date_long set to 'L'yyyy-MM-dd
2 gives S21-06028, S13:04 with intl.date_time.pattern_override.date_short set to 'S'yy-MM-dd
That confirms your observations. BTW, why do you have a date pattern yyyy-MM-dd in two of your time overrides?

And finally there is bug 1672548 which is about restoring at least the weekday display for mail.ui.display.dateformat.* value 4.

Comment 212

[[e-mail]]

DOH!
Jose, the answer is just incomplete clean up. I was putting random parameters in to see where they showed up and never finished cleaning up the mistake @thomas8 pointed out.

Thanks for the link to bug 1672548. I'd argue that with the new override pattern features which really cover pretty much any concievable use case, the way to leverage that configuration power in display formats is to enable calling them. As indicated in my proposed preferences modification, calling time and date formats individually provides a nice combinatoric feature effect from the explicitly individual preference configuration for {date} and {time}, but my needs would be more than sufficiently suited if mail.ui.display.dateformat.X took the form:

Value	meaning	Default example (no override specified, derived from locale)
0	Time (short)	10:23 AM
1	Date/time short	12/31/1999 10:23 AM
2	Date/time medium	Fri, Dec 31 2003 10:23 AM
3	Date/time long	Friday, December 31 2003 10:23 AM
4	Date/time full	Friday, December 31 2003 10:23 AM PDT

Likewise, modifying the preferences->calendar->Date Text Format: drop down selector to likewise enable selection of short/medium/long/full would seem consistent with the purpose of the new preferences.

one question: intl.date_time.pattern_override.connector_short implies intl.date_time.pattern_override.connector_(short|medium|long|full) - is this true?

Comment 213

[José M. Muñoz]

You are missing the original option 4 that bug 1672548 tries to restore: "Fri, 10:55". Yes, the short connector pattern is always used, see 1706318 comment #3.

Comment 214

[[e-mail]]

"Let's hope that no one asks for the long version." :-)

I admit I'm coming to this topic from the long thread about requesting treating ISO 8061 as a special case outside of ICU localization and through that lens I can't help but see creating exception formats for mail.ui.display.dateformat.X in a similar light. Is there a reason why my proposal to map the mail.ui.display.dateformat.X interpretations to the intl.date_time.pattern_override.X_X pattern definitions wouldn't achieve your goal? If you wanted value "4" to correspond to "Fri, 10:55" it could be achieved with:

intl.date_time.pattern_override.connector_short     {1},{0}
intl.date_time.pattern_override.time_full           HH:mm
intl.date_time.pattern_override.date_full           ccc

I apologize if I'm letting my excitement for arbitrary ICU date field symbol construction get in the way.

Comment 215

[José M. Muñoz]

Oops, yes, if want to misuse the "full" to be a weekday only and then drive the mail.ui.display.dateformat.thisweek to use this 'fake' "full", then it can be done. BTW, should it be ccc or E?

Comment 216

[Thomas D. (:thomas8)]

I have documented this feature for Thunderbird:

• https://support.mozilla.org/kb/customize-date-time-formats-thunderbird
• https://enterprise.thunderbird.net/manage-updates-policies-and-customization/thunderbird-preferences-enterprise/customize-thunderbirds-date-and-time-formats

Also encountered SUMO Bug 1723190 on the way - cunning!

Comment 217

[Jethro Beekman]

I just updated to Ubuntu 20.04/Thunderbird 78 and the LC_TIME=root.UTF-8 workaround doesn't work anymore. Can anyone point me to the correct setup for these versions?

Comment 218

[Magnus Melin [:mkmelin]]

Jethro: please scroll, two comment up. That has all the info you could need.

Comment 219

[Jethro Beekman]

Sorry if I'm being obtuse, but I don't see it. Are you referring to comment 216? The linked content appears to discuss options for TB 91+.

Comment 220

[Magnus Melin [:mkmelin]]

Ah, I missed that. For 78, perhaps comment 154 or so. But 91 is available for download.

Comment 221

[Jethro Beekman]

Yes I was using that workaround (or from comment 144 really) but as I mentioned in comment 217 that stopped working.

Comment 222

[Jethro Beekman]

Actually the workaround from the whiteboard (sv_SE.UTF-8) does work