1405993 - Update our in-tree ICU to 60

Status: RESOLVED FIXED

(Core :: JavaScript: Internationalization API, enhancement, P2)

Description

[André Bargull [:anba]]

ICU 60 is scheduled for release this month.

http://site.icu-project.org/download/60

Comment 1

[André Bargull [:anba]]

:jfkthame, do we still need to regenerate any Unicode tables (bug 1376834, comment #1), or is this no longer an issue since bug 1402271 landed?

Thanks,
André

Comment 2

[Jonathan Kew [:jfkthame]]

We still have a small amount of Unicode data that we manage ourselves (because ICU doesn't provide it, or at least doesn't expose it in a form that's usable for us), but it's a vastly cut-down version of what we had previously.

The scripts used to maintain this remnant are in intl/unicharutil/tools; at the time of updating ICU, we should re-run genUnicodePropertyData.pl and genSpecialCasingData.pl there to refresh our data tables. (This is definitely needed if the ICU update bumps the Unicode version that's being supported, so that additional characters or scripts are being added, etc, but to be on the safe side we should probably do it even if the Unicode version is unchanged.)

Comment 3

[André Bargull [:anba]]

Try with ICU60-RC:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f54890bdbfc55c03880c44d8485ce248105fd42f

- Static analysis busted (bug 1407298)
- Doesn't compile on Windows 2012
- Errors in layout/reftests/text-transform/{all-title.html, all-title-ref.html}, I guess because the update script removed "font-size: 12px;" from the CSS style sheets: https://hg.mozilla.org/try/rev/e921d316359036ae8d3e8e6b397756bb7b02da81

Comment 4

[André Bargull [:anba]]

(In reply to André Bargull [:anba] from comment #3)
> - Doesn't compile on Windows 2012

Filed http://bugs.icu-project.org/trac/ticket/13411

Comment 5

[André Bargull [:anba]]

New Try with the issues mentioned in comment #3 fixed looks good:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9975fdb22ba0814156aa77e8fc1e72234d7ff8c5

Comment 6

[André Bargull [:anba]]

ICU60 release is scheduled for Nov 1st (https://sourceforge.net/p/icu/mailman/message/36082968/).

Comment 7

[André Bargull [:anba]]

Attachment: bug1405993-part1-sprintf-icu.patch

Allow to call snprintf in ICU sources.

Comment 8

[André Bargull [:anba]]

Attachment: bug1405993-part2-remove-old-patches.patch

These patches are no longer necessary for ICU60.

u_setMemoryFunctions-callconvention-anachronism-msvc.diff
- https://ssl.icu-project.org/trac/ticket/13030

bug-1387476-timezone-detection.diff
- https://ssl.icu-project.org/trac/ticket/12770

bug-1380083
- http://bugs.icu-project.org/trac/ticket/13178

bug-1373763-convertToPosix-stack-value-out-of-scope.diff
- https://ssl.icu-project.org/trac/ticket/13263

bug-1387937.diff
- http://bugs.icu-project.org/trac/changeset/40324

Comment 9

[André Bargull [:anba]]

Attachment: bug1405993-part3-update-icu.zip

Update to ICU60.

Comment 10

[André Bargull [:anba]]

Attachment: bug1405993-part4-unicode10-spidermonkey.patch

Update to Unicode 10 for SpiderMonkey, so we stay consistent with the Unicode version used in ICU.

Comment 11

[André Bargull [:anba]]

Attachment: bug1405993-part5-format-result-changed.patch

Looks like the CLDR data changed for "ar-MA-u-ca-islamicc", so we need to update this test.

Comment 12

[André Bargull [:anba]]

Attachment: bug1405993-part6-update-gecko-unicode.patch

Updated files after running genUnicodePropertyData.pl.

No changes for genSpecialCasingData.pl are present, except for bug 1407543.

Comment 13

[André Bargull [:anba]]

Attachment: bug1405993-part7-clobber.patch

ICU update requires a CLOBBER.

Comment 14

[Jan Beich]

Do you plan to support --with-system-icu + ICU 59.1 downstream[1]? If not bump version in build/autoconf/icu.m4.
Both --without-system-icu (default) and --with-system-icu (+ icu-60.1 package) built fine with the patches here on FreeBSD.

[1] Some from https://repology.org/metapackage/icu/versions

Comment 15

[André Bargull [:anba]]

(In reply to Jan Beich from comment #14)
> Do you plan to support --with-system-icu + ICU 59.1 downstream[1]? If not
> bump version in build/autoconf/icu.m4.

I don't think we need to bump the minimum required ICU version this time, because (at least currently) we don't use any new API from ICU60.

> Both --without-system-icu (default) and --with-system-icu (+ icu-60.1
> package) built fine with the patches here on FreeBSD.

Great, good to hear!

> [1] Some from https://repology.org/metapackage/icu/versions

Oh, that's a useful site! *bookmarked*

Comment 16

[André Bargull [:anba]]

The current plan [1] is to wait with updating ICU until 58 goes beta, because we're quite late in the cycle. So basically in two weeks [2] we can land these patches.

[1] https://mozilla.logbot.info/jsapi/20171101#c13794706-c13794770
[2] https://wiki.mozilla.org/RapidRelease/Calendar

Comment 17

[André Bargull [:anba]]

Attachment: bug1405993-part7-clobber.patch

Rebase CLOBBER patch to apply cleanly on inbound.

Comment 18

[André Bargull [:anba]]

Try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e338c45cccd3700ce8508efe0f080ea7f2de718b

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/53015dab6075
Part 1: Don't report warnings for sprintf calls in ICU. r=mystor
https://hg.mozilla.org/integration/mozilla-inbound/rev/d6daf71433d2
Part 2: Remove no longer used ICU patches. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/00c94696d5a3
Part 3: Update in-tree ICU to release 60.1. rs=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/cf8f20a4de1b
Part 4: Update to Unicode 10 for SpiderMonkey. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/8f65e64a3b9e
Part 5: Updated expected results when formatting dates in ar-ma locale. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/a917acc8147c
Part 6: Regenerate Unicode files after ICU update. r=jfkthame
https://hg.mozilla.org/integration/mozilla-inbound/rev/e74e4ec2cb39
Part 7: Updating ICU requires a clobber. r=clobber

Comment 20

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/53015dab6075
https://hg.mozilla.org/mozilla-central/rev/d6daf71433d2
https://hg.mozilla.org/mozilla-central/rev/00c94696d5a3
https://hg.mozilla.org/mozilla-central/rev/cf8f20a4de1b
https://hg.mozilla.org/mozilla-central/rev/8f65e64a3b9e
https://hg.mozilla.org/mozilla-central/rev/a917acc8147c
https://hg.mozilla.org/mozilla-central/rev/e74e4ec2cb39