1543641 - Consider applying Reiwa patches for ICU63 (Beta) and ICU60 (ESR)

Status: RESOLVED FIXED

(Core :: JavaScript: Internationalization API, task)

Description

[André Bargull [:anba]]

We should consider applying the following two patches to Beta and ESR60 to support the new Reiwa era.

ICU 60: https://github.com/unicode-org/icu/pull/599
ICU 63: https://github.com/unicode-org/icu/pull/608

Comment 1

[André Bargull [:anba]]

Zibi, do you know if this topic has already been discussed somewhere?

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

I don't think so.

Comment 3

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 1: Update in-tree ICU to release 63.2. r=jwalden!

Comment 4

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 2: Update tzdata in ICU data files to 2019a. r=jwalden!

Depends on D27342

Comment 5

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 3: Remove special-casing for Etc/UCT. r=jwalden!

Etc/UCT resp. UCT is a link to Etc/UTC starting with tzdata 2019a.

Depends on D27343

Comment 6

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 1: Update ICU update script to use new GitHub location. r=jwalden!

Comment 7

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 2: Update in-tree ICU to release 60.3. r=jwalden!

Depends on D27345

Comment 8

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 3: Update tzdata in ICU data files to 2019a. r=jwalden!

Depends on D27347

Comment 9

[André Bargull [:anba]]

Attachment: Bug 1543641 - Part 4: Remove special-casing for Etc/UCT. r=jwalden!

Etc/UCT resp. UCT is a link to Etc/UTC starting with tzdata 2019a.

Depends on D27348

Comment 10

[André Bargull [:anba]]

Updating to ICU 63.2 for Beta resp. to ICU 60.3 for ESR will also include the tzdata update to 2019a (bug 1539197).

Comment 11

[André Bargull [:anba]]

The new era is correctly displayed with these updates:

js> print(Intl.DateTimeFormat("en-u-ca-japanese",{era:"long"}).format(new Date(2019,4,1))) 
5 1, 1 Reiwa
js> print(Intl.DateTimeFormat("ja-u-ca-japanese",{era:"long"}).format(new Date(2019,4,1)))  
令和1年5月1日

Comment 12

[André Bargull [:anba]]

Try builds:

Beta: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f7047bd6b60e037f7c6cebe07b90af2083b8a0bd
ESR: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f9f7d6e0dbb1aa31ae1c7e250c5c02e1a171b6f0

Comment 13

[André Bargull [:anba]]

[Tracking Requested - why for this release]:
It'd be nice to get these patches into Beta and ESR to ensure we display the correct era (Reiwa instead of Heisei) for dates starting with May 1st.

Chromium also seems to consider back-porting the relevant changes for M74 (https://bugs.chromium.org/p/chromium/issues/detail?id=952305).

Comment 14

[André Bargull [:anba]]

Comment on attachment 9057956 [details]
Bug 1543641 - Part 1: Update in-tree ICU to release 63.2. r=jwalden!

Beta/Release Uplift Approval Request

• User impact if declined: This is an ICU maintenance update to add support for the new Reiwa era to the Japanese imperial calendar. Without this change formatting dates will still use the previous Heisei era. Release notes: https://github.com/unicode-org/icu/releases/tag/release-63-2.

Additionally the ICU update also includes tzdata 2019a, which is only correcting some previous daylight saving time dates. Release notes: https://mm.icann.org/pipermail/tz-announce/2019-March/000055.html.

• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Part 1:
  This change only updates data, but performs no code changes. The data changes are further more limited to replacing previous placeholder strings with the correct name for the new era and enabling the new era (it was previously set in an "ignored" state within ICU).

Part 2:
Updates our local copy of tzdata and test files, the actual change to use the new tzdata is already included in Part 1.

Part 3:
The tzdata update changes the canonicalisation for the time zones "UCT" and "Etc/UCT", which are now both canonicalised to "UTC".

Supporting the new era in Nightly depends on two different bugs (bug 1543642 and bug 1543644).

• String changes made/needed:

Comment 15

[André Bargull [:anba]]

Comment on attachment 9057959 [details]
Bug 1543641 - Part 1: Update ICU update script to use new GitHub location. r=jwalden!

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: See the beta uplift request.
• User impact if declined: See the beta uplift request.
• Fix Landed on Version:
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Part 1:
  Update the source location of ICU, so the following changes can be more easily performed resp. reproduced. (The update script is run by hand and not part of any automatic process.)

Part 2:
This change match Part 1 of the Beta patches. Additionally it includes a bug fix to select the correct time zone on Windows systems. Previously this issue was considered to apply only to ICU63 or later (see bug 1513934), but it seems to apply to older versions as well.

Part 3:
Matches Part 2 of the Beta patches.

Part 4:
Matches Part 3 of the Beta patches.

• String or UUID changes made by this patch:

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 9057959 [details]
Bug 1543641 - Part 1: Update ICU update script to use new GitHub location. r=jwalden!

Update for era change in Japan (which happens May 1).
OK for ESR 60.7 uplift.

Comment 17

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Did these land in nightly?

I still see:

new Intl.DateTimeFormat('en-JP-u-ca-japanese', {era:'long'}).format(new Date(2019,4,1))
/*
5 1, 31 Heisei
*/

and

new Intl.DateTimeFormat('ja-JP-u-ca-japanese', {era:'long'}).format(new Date(2019,4,1))
/*
平成31年5月1日
*/

on 68.0a1 (2019-04-30) (64-bit)

Comment 18

[André Bargull [:anba]]

(In reply to Emma Humphries, Bugmaster ☕️🎸🧞‍♀️✨ (she/her) [:emceeaich] (UTC-8) needinfo? me from comment #17)

Did these land in nightly?

It should get into the next Nightly (bug 1543642).

---

There are different ICU versions used across the branches: Nightly uses ICU 64, Beta uses ICU 63, and ESR uses ICU 60. So we also need to use different patches for each branch and can't simply backport a patch, let's say the patch for Nightly and apply it to Beta. It's all a bit messy. :-)

Comment 19

[Liz Henry (:lizzard) (relman/hg->git project)]

Thanks. I'm ok with uplifting now and if there is any cleanup necessary we still have a chance to fix & land.

Comment 20

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-esr60/rev/595ba23233cd
https://hg.mozilla.org/releases/mozilla-esr60/rev/19feda08113d
https://hg.mozilla.org/releases/mozilla-esr60/rev/300c484335a9
https://hg.mozilla.org/releases/mozilla-esr60/rev/2f42c2c54638

Comment 21

[Pascal Chevrel:pascalc]

Comment on attachment 9057956 [details]
Bug 1543641 - Part 1: Update in-tree ICU to release 63.2. r=jwalden!

Low risk, uplift accepted for 67 beta 16, thanks.

Comment 22

[Andreea Pavel [:apavel]]

When trying to import the second patch to uplift to beta i get the following:

mozilla@ubuntu ~/mozilla-unified beta(+9) $ arc patch D27343 --nobranch
Updating to the revision's base commit
109 files updated, 0 files merged, 7 files removed, 0 files unresolved
Created and checked out bookmark arcpatch-D27342.
Downloading binary data for 'icudt63l.dat'...
Downloading binary data for 'icudt63l.dat'...
OKAY Successfully committed patch.

This diff is against commit e51de31c7f97d217a94f4d4eb90a92e1164f105e, but
the commit is nowhere in the working copy. Try to apply it against the
current working copy state? (.) [Y/n]

If I choose Y, then I get the following issue for the 3rd patch:

mozilla@ubuntu ~/mozilla-unified beta(+10) $ arc patch D27344 --nobranch
Updating to the revision's base commit
124 files updated, 0 files merged, 7 files removed, 0 files unresolved
Created and checked out bookmark arcpatch-D27342.
Downloading binary data for 'icudt63l.dat'...
Downloading binary data for 'icudt63l.dat'...
OKAY Successfully committed patch.
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'metaZones.res'...
Downloading binary data for 'metaZones.res'...
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'metaZones.res'...
Downloading binary data for 'metaZones.res'...
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'zoneinfo64.res'...
Downloading binary data for 'metaZones.res'...
Downloading binary data for 'metaZones.res'...

Patch Failed!
Exception
Command failed with error #255!
COMMAND
HGPLAIN=1 hg import --no-commit -

STDOUT
applying patch from stdin

STDERR
patching file js/src/tests/non262/Intl/DateTimeFormat/timeZone_notbackward_links.js
Hunk #1 FAILED at 0
1 out of 1 hunks FAILED -- saving rejects to file js/src/tests/non262/Intl/DateTimeFormat/timeZone_notbackward_links.js.rej
patching file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backzone_links.js
Hunk #1 FAILED at 0
1 out of 1 hunks FAILED -- saving rejects to file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backzone_links.js.rej
patching file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backzone.js
Hunk #1 FAILED at 0
1 out of 1 hunks FAILED -- saving rejects to file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backzone.js.rej
patching file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backward_links.js
Hunk #1 FAILED at 0
Hunk #2 FAILED at 66
Hunk #3 FAILED at 97
3 out of 3 hunks FAILED -- saving rejects to file js/src/tests/non262/Intl/DateTimeFormat/timeZone_backward_links.js.rej
patching file js/src/builtin/intl/TimeZoneDataGenerated.h
Hunk #1 FAILED at ... (879 more bytes) ...
(Run with --trace for a full exception trace.)

Comment 23

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-beta/rev/c62cd84953ed
https://hg.mozilla.org/releases/mozilla-beta/rev/1b1ad8962c6f
https://hg.mozilla.org/releases/mozilla-beta/rev/e3264123c6a2

Comment 24

[Liz Henry (:lizzard) (relman/hg->git project)]

Noted for ESR 60.7.0 and 67 as "Font and date adjustments to accommodate the new Reiwa era in Japan"