1186262 - Add a Toolkit module to localize time intervals in the user interface

Status: RESOLVED DUPLICATE of bug 1407240

(Toolkit :: General, defect)

Description

[Bernardo Rittmeyer [:rittme]]

Some bugs like Bug 433238 and Bug 1184265 need to display a relative time, like: 
3 second ago or 5 days old.
We should create a general helper module to help with that.

Comment 1

[Bernardo Rittmeyer [:rittme]]

Attachment: MozReview Request: Bug 1186262 - Adds a relative time helper module to toolkit. r=MattN

Bug 1186262 - Adds a relative time helper module to toolkit. r=MattN

Comment 2

[:Paolo Amadini]

https://reviewboard.mozilla.org/r/13799/#review12381

Just saw this and wanted to leave a drive-by review. It's nice if we can have shared module for this functionality and reuse strings across the interface! It probably requires some thought as generalized localization is defintely not easy.

In fact we have an existing use case for relative times, the Downloads code. It would be good if we can ensure we create a module we can use there, and either use it from the start or file a bug to use it. That's a good validation point because if we create something that's not easy to reuse, there's no point in having a shared module in the first place.

I've not looked up if we have other similar code and maybe strings around, in mozilla-central or B2G.

::: toolkit/modules/RelativeTimeHelper.jsm:18
(Diff revision 1)
> +   * @param {string} substituteString
> +   *        The string we should substitute the date into.
> +   *        Should be a string we can substitute the date into. i.e.: "%S old".

A double substitution can work in English but not in other languages - you probably need combined strings for the various cases that have to be handled, and just pass in an enum for which case you want (or call a different function).

::: toolkit/modules/RelativeTimeHelper.jsm:7
(Diff revision 1)
> +this.EXPORTED_SYMBOLS = ["RelativeTimeHelper"];

nit: just "RelativeTime.jsm", exporting the "RelativeTime" symbol.

::: toolkit/modules/RelativeTimeHelper.jsm:29
(Diff revision 1)
> +  getRelativeTime(substituteString, startDate, endDate = Date.now()) {

Hm, maybe "formatTimeSpan" or "toLocaleTimeSpan" or something that equally conveys the purpose?

::: toolkit/modules/tests/xpcshell/test_relativeTime.js:33
(Diff revision 1)
> +      expectedString: "a second",

If I remember correctly tests with strings should be fine (though they'll fail in other locales), not sure if we have to mark them in a special way.

You should probably have a review from someone in the localization team, like Francesco, earlier rather than later.

Comment 3

[Francesco Lodolo [:flod]]

(In reply to :Paolo Amadini from comment #2)
> If I remember correctly tests with strings should be fine (though they'll
> fail in other locales), not sure if we have to mark them in a special way.

> relativeTimeDays = a day;#1 days

Basically it's fine as long as you can use the variable in every plural form, i.e. 

relativeTimeDays = #1 translation-of-day;#1 translation-of-days

That's because in some locales you *need* the number to make sense.

As far as I can tell we .replace() the #1, so we're fine.

Comment 4

[Francesco Lodolo [:flod]]

But looking at the code I don't understand something.

> a second old; 3 minutes old; 4 days ago.

How do you plan to use this module? Because if you plan to concatenate it with other strings, that's really really wrong and broken and will work only in English.

Comment 5

[Francesco Lodolo [:flod]]

Different product and plural form (in Gaia we use CLDR plural forms), but this gives you an idea of the complexity
https://github.com/mozilla-b2g/gaia/blob/master/shared/locales/date/date.en-US.properties

Comment 6

[Bernardo Rittmeyer [:rittme]]

(In reply to Francesco Lodolo [:flod] from comment #4)
> But looking at the code I don't understand something.
> 
> > a second old; 3 minutes old; 4 days ago.
> 
> How do you plan to use this module? Because if you plan to concatenate it
> with other strings, that's really really wrong and broken and will work only
> in English.

I was trying to leave to the user the option to add a different "qualifier" to the date, like "old" or "ago". I'm not sure it would work this way in every language.

I would not concatenate this with other strings. What I'm doing in the code that uses this module would be something like this:
> %1$S (%2$S old)
Where the 1st string is a username and the 2nd is the username age.

The code in Bug 1184265 would use something like:
> Last sync: %S ago
Where %S would be the time to the last sync.

Do you think this could work correctly this way?

Comment 7

[Francesco Lodolo [:flod]]

(In reply to Bernardo Rittmeyer [:rittme] from comment #6)
> The code in Bug 1184265 would use something like:
> > Last sync: %S ago
> Where %S would be the time to the last sync.
> 
> Do you think this could work correctly this way?

Unfortunately I don't think it will work for some complex languages. 

Take for example downloads.properties for the download manager
http://hg.mozilla.org/releases/mozilla-aurora/file/default/toolkit/locales/en-US/chrome/mozapps/downloads/downloads.properties#l88

># LOCALIZATION NOTE (timeLeftDouble2): %1$S time left; %2$S time left sub units
># example: 11 hours, 2 minutes remaining; 1 day, 22 hours remaining
>timeLeftDouble2=%1$S, %2$S remaining

This seems to be a perfect case for this library, but it's broken: "remaining" is an adjective, it needs to be declined according to the noun's genre and number, and for example it might be different for hours, minutes, and seconds. 
Some locales solve this by doing "Remaining time: %1$S, %2$S", but that's working around the original localizability issue.

While I consider this approach interesting, I think it will work only if we consider explicit cases, e.g.
relativeTimeDays = a day;#1 days
relativeTimeDaysAgo = a day ago;#1 days ago
relativeTimeDaysRemaining = a day remaining;#1 days remaining

And that would increase exponentially the number of strings needed, and probably how useful this can be.

Comment 8

[Mark Hammond [:markh] [:mhammond]]

(In reply to Francesco Lodolo [:flod] from comment #7)
> While I consider this approach interesting, I think it will work only if we
> consider explicit cases, e.g.
> relativeTimeDays = a day;#1 days
> relativeTimeDaysAgo = a day ago;#1 days ago
> relativeTimeDaysRemaining = a day remaining;#1 days remaining
> 
> And that would increase exponentially the number of strings needed, and
> probably how useful this can be.

That looks alot like the link you shared before (https://github.com/mozilla-b2g/gaia/blob/master/shared/locales/date/date.en-US.properties) - is there any scope for ripping what we need from gaia into something that can be shared?

Comment 9

[Francesco Lodolo [:flod]]

(In reply to Mark Hammond [:markh] from comment #8)
> That looks alot like the link you shared before
> (https://github.com/mozilla-b2g/gaia/blob/master/shared/locales/date/date.en-
> US.properties) - is there any scope for ripping what we need from gaia into
> something that can be shared?

If you mean "taking from Gaia into Gecko", plural forms are completely different and incompatible. We'd need to recreate something on Gecko's side.

If you mean "create something that can be used by both Gaia and Gecko", sadly it's not an option. We don't ship Gecko localizations in Gaia, and we have locales that exist only in Gaia's ecosystem.

What we have in Gecko is custom and pretty hacky
https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_and_Plurals#Developing_with_PluralForm

Gaia, on the other hand, uses CLDR plural rules
Reference: http://cldr.unicode.org/index/cldr-spec/plural-rules
Code: https://github.com/mozilla-b2g/gaia/blob/master/shared/js/l10n.js#L119

In Gaia's world the need for relative dates/times is fundamental (calendar, SMS, email, downloads, time to charge, etc.), I don't think it is as needed in Gecko.

Comment 10

[Axel Hecht [:Pike]]

The naughty bit is that plural forms don't get relative dates/times right.

See http://www.unicode.org/cldr/charts/27/by_type/date_&_time.fields.html#Relative_Day_Short for the full glory, cldr has data on these things.

I think the right way to fix this is to add this functionality to the JavaScript Intl API, https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Intl, and to hopefully use ICU in the back-end to implement it (which is mostly an api exposing cldr, thus putting that out there).

Surplus, we can use it on both gaia and firefox.

Gandalf, have you heard of current movement to add APIs like this? Could you ping our contacts if anybody does something like this already?

Comment 11

[Zibi Braniecki [:zbraniecki][:gandalf]]

(In reply to Axel Hecht [:Pike] from comment #10)
> I think the right way to fix this is to add this functionality to the
> JavaScript Intl API,
> https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/
> Global_Objects/Intl, and to hopefully use ICU in the back-end to implement
> it (which is mostly an api exposing cldr, thus putting that out there).

Agree.

> Gandalf, have you heard of current movement to add APIs like this? Could you
> ping our contacts if anybody does something like this already?

The current state is that if I'm interested in extending Intl API (we are), I should drive it.

I may actually consider doing that next quarter, but having no experience as a spec driver I'm just not sure how much work it will be.

For now, the best thing you can do to express the intention to add a new feature and make stakeholders take this into account when thinking about next revision of Intl API, add it to http://wiki.ecmascript.org/doku.php?id=globalization:strawman

Comment 12

[Zibi Braniecki [:zbraniecki][:gandalf]]

Actually, I'll be very soon reworking this as part of bug 1172621 (prettyDate part). I'll look into this then.

Comment 13

[Zibi Braniecki [:zbraniecki][:gandalf]]

Sorry, meant bug 1170963.

Comment 14

[Matthew N. [:MattN]]

Here is my thinking on this bug:
* This wasn't available from the Intl API which would be the ideal solution. It doesn't seem like something we need to block on for a few consumers which can easily be updated to Intl later.
* We have l10n_date from Gaia so we should consider using it.
** Comment 5 explains that plural forms work differently in Gaia so we can't use it as-is.
*** Any reason we can't port the plural form logic from https://github.com/mozilla-b2g/gaia/blob/master/shared/js/l10n.js for this new module.

For the 2 bugs in comment 0 where we only care about the past, can we implement the proper plural forms according to http://www.unicode.org/cldr/charts/27/by_type/date_&_time.fields.html#Relative_Day_Short and de-scope this to only handle days for now (like bug 1184265 comment 1 is fine with)? We can make the function generic so it can support more precision later as required but leave that to follow-ups as needed and based on how this trial with days works.

We could use separate strings for the various cases e.g.
-2=%S days ago
-1=yesterday
0=today
past-zero=%S days ago
past-one=%S day ago
past-two=%S days ago
past-few=%S days ago
past-many=%S days ago
past-other=%S days ago

Any objections to this for 42 or 43?

Comment 15

[Zibi Braniecki [:zbraniecki][:gandalf]]

(In reply to Matthew N. [:MattN] from comment #14)
> * We have l10n_date from Gaia so we should consider using it.

We are phasing away/refactoring it in Gaia. I'd recommend you to wait for us to come up with the next iteration (which may be a shim for the Intl API extension), before you take it for toolkit.

Comment 16

[Francesco Lodolo [:flod]]

(In reply to Matthew N. [:MattN] from comment #14)
> We could use separate strings for the various cases e.g.
> -2=%S days ago
> -1=yesterday
> 0=today
> past-zero=%S days ago
> past-one=%S day ago
> past-two=%S days ago
> past-few=%S days ago
> past-many=%S days ago
> past-other=%S days ago
> 
> Any objections to this for 42 or 43?

I'm totally against it, it would confuse the heck out of localizers, not counting tools. Tools are designed to display plural forms together, and having a mix of "standards" in Gecko is bad.

Even Loop, which is using a patched variant of l10n.js, uses the Gecko format for plurals
http://hg.mozilla.org/mozilla-central/file/cb8bdb8ffaef/browser/locales/en-US/chrome/browser/loop/loop.properties#l173

If we decide to restrict this module to days, we don't need to introduce new formats.

today=today
yesterday=yesterday
daysAgo=#1 day ago;#1 days ago

Comment 17

[Axel Hecht [:Pike]]

Let's take the two bugs we have into account:

past-many=%S days ago

is the string we want to insert, and the call sites are

Last Sync: %S

and

%1$S (%2$S old)

AKA, the strings proposed here only work in one particular UX context, but not the other.

I think that the API will lead to usage where the string works in English with some level of brute force, but we should expect that the usage is going to be awkward or more awkward in other languages.

Sadly this is independent of Intl.Foo vs RelativeDate.jsm vs date.js, the generic terms we're getting out of those are only usable in a very constrained context, and the further you walk away from that, the more awkward the UX is going to be, in English already.

Getting more natural language into the UI is a good goal, but it's also means that each use case is going to be subtly different.

I've talked to stas about using l20n for this, but we don't know the buglist for that yet. Sadly, the buglist is long, and has some beefy bugs in it.

Comment 18

[Zibi Braniecki [:zbraniecki][:gandalf]]

I plan to work on the relative date API proposal next week. If this bug can wait a bit I should have something that you can shim for now (and we'll be shimming in Gaia as well).

Comment 19

[Mark Hammond [:markh] [:mhammond]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #18)
> I plan to work on the relative date API proposal next week. If this bug can
> wait a bit I should have something that you can shim for now (and we'll be
> shimming in Gaia as well).

Is this something that's still alive?

Comment 20

[Zibi Braniecki [:zbraniecki][:gandalf]]

Yeah!

Actually, we now how a full mozIntl library - https://github.com/l20n/mozintl that has RelativeTimeFormat API and that same API is now being standardized https://github.com/tc39/ecma402/#current-proposals :)

Feel free to use it! (although if you don't use l20n.js you may have to modify a bit how you get the values from l10n data)

Comment 21

[Mark Hammond [:markh] [:mhammond]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #20)
> Actually, we now how a full mozIntl library -
> https://github.com/l20n/mozintl that has RelativeTimeFormat API and that
> same API is now being standardized
> https://github.com/tc39/ecma402/#current-proposals :)

That looks awesome \o/

> Feel free to use it! (although if you don't use l20n.js you may have to
> modify a bit how you get the values from l10n data)

mozilla-central doesn't use l20n and you can probably answer this question must faster than I can - can you give me some idea of what you mean here? There's already a well established l10n process for mozilla-central, so can you suggest how we might integrate l20n with that? How would our existing localizers add support for this?

Comment 22

[Zibi Braniecki [:zbraniecki][:gandalf]]

> can you give me some idea of what you mean here? There's already a well established l10n process for mozilla-central, so can you suggest how we might integrate l20n with that? How would our existing localizers add support for this?

Axel, can you suggest how to tackle that? I'm not familiar with the current state of Toolkit l10n but if we use .properties then we can probably create a version of mozIntl that uses the Toolkit L10n API to pull .properties strings for those formatters.

Does it sound like a solution or would you prefer some other route?

Comment 23

[Edouard Oger [:eoger]]

Any update on this?

Comment 24

[Zibi Braniecki [:zbraniecki][:gandalf]]

Not much on my part. This week we started working on getting L20n and mozIntl into Gecko, but it'll take some time. In result of my work I expect to have gre/modules/IntlRelativeTime which will provide you that feature, but I don't have an ETA yet. I even didn't start filing bugs for this yet so can't link anything.

If you need it soon, the best way forward might be to land your own IntlRelativeTime that matches the API we plan for JS and uses either .properties or .JSON from CLDR to store data.

If you're willing to wait, I'll link the bugs once we start filling them and can aim to land IntlRelativeTime as the first formatter.

Comment 25

[Zibi Braniecki [:zbraniecki][:gandalf]]

Ok, I opened a new bug (bug 1270140) to tackle the RelativeTimeFormat API. If Waldo will agree, I'll write it as chrome-only (for now) API on Intl object and we'll be able to use it.

We need this for L20n in Toolkit as well, so it makes sense to merge the effort.

If Waldo will object, I'll probably try to land some variant of RelativeTimeFormat jsm module and may want to reuse CLDR data that we have (although I'm not sure how can I access ICU from JSM module to parse the CLDR data).

Comment 26

[Axel Hecht [:Pike]]

Clearing my needinfo, generally speaking, I think we should not reinvent the wheel and use Intl.

Comment 27

[Edouard Oger [:eoger]]

Since bug 1407240 has landed, should we close this?

Comment 28

[Zibi Braniecki [:zbraniecki][:gandalf]]

Sure! I'd be happy to see anyone who was looking forward to this API try the mozIntl.RelativeTimeFormat for their use case and verify that it serves it well!

Comment 29

[Zibi Braniecki [:zbraniecki][:gandalf]]

Also, notice that the original title referred to intervals. `RelativeTimeFormat` formats time delta. The difference is between "5 minutes" and "in 5 minutes"/"5 minutes ago"

The former is an interval, the other is a vector :)

We will cover the former as part of the UnitFormat or a revision of NumberFormat, but that'll take time.