Update our in-tree ICU to 62 and update to Unicode 11

RESOLVED FIXED in Firefox 63

Status

()

RESOLVED FIXED
10 months ago
6 months ago

People

(Reporter: anba, Assigned: anba)

Tracking

(Depends on: 1 bug)

Trunk
mozilla63
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox62 wontfix, firefox63 fixed)

Details

(URL)

Attachments

(7 attachments, 1 obsolete attachment)

(Assignee)

Description

10 months ago
From http://site.icu-project.org/download/62:
> ICU 62 upgrades to Unicode 11 and to CLDR 33.1 locale data.

That means we need to combine the ICU update with the Unicode update.

The ICU release is scheduled for June 13, 2018 and Unicode 11 is scheduled for June 5, 2018 according to https://www.unicode.org/versions/Unicode11.0.0/.


Try build with ICU 62 RC and Unicode 11 preview:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7015178ce7db30a0dc9654ecefd09bd6d1aa60af

Comment 1

9 months ago
http://site.icu-project.org/

"2018-06-20: ICU 62 (62.1) released. ICU 62 upgrades to Unicode 11 and CLDR 33.1. ICU 62 includes number format skeleton support in MessageFormat, ICU4C DecimalFormat wrapping the new NumberFormatter implementation for improved performance. For details, please see the ICU 62 download page."
(Assignee)

Comment 3

9 months ago
Updates one of our ICU patches to apply cleanly on ICU 62.
Attachment #8987029 - Flags: review?(jwalden+bmo)
(Assignee)

Comment 4

9 months ago
The actual update to ICU 62. (As always too large to attach as a plain diff file.)
Attachment #8987031 - Flags: review?(jwalden+bmo)
(Assignee)

Comment 5

9 months ago
Updates the expected values for currency and percent formatting after the CLDR update in ICU 62. I've also added tests for percent formatting with "de" for the differences between "en" and "de": "de" (among other locales) adds U+00A0 (NO-BREAK SPACE) before the percent sign, whereas "en" doesn't.
Attachment #8987033 - Flags: review?(jwalden+bmo)
(Assignee)

Comment 6

9 months ago
Update SpiderMonkey to Unicode 11 to stay consistent with ICU 62, which internally uses Unicode 11, too.
Attachment #8987034 - Flags: review?(jwalden+bmo)
(Assignee)

Comment 7

9 months ago
Update Gecko to Unicode 11 to stay consistent with ICU 62, which internally uses Unicode 11, too.

I've also updated "genUnicodePropertyData.pl" to use "https" instead of "http" links where possible.
Attachment #8987037 - Flags: review?(jfkthame)
(Assignee)

Comment 8

9 months ago
Clobber for ICU update.
Attachment #8987038 - Flags: review+
Attachment #8987037 - Flags: review?(jfkthame) → review+

Updated

9 months ago
Attachment #8987029 - Flags: review?(jwalden+bmo) → review+
Comment on attachment 8987033 [details] [diff] [review]
bug1466471-part3-update-test-expectations.patch

Review of attachment 8987033 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/tests/non262/Intl/NumberFormat/formatToParts.js
@@ +64,5 @@
>    };
>  var usDollarsCode = new Intl.NumberFormat("en-US", usdCodeOptions);
>  
>  assertParts(usDollarsCode, 25,
> +            [Currency("USD"), Literal("\xA0"), Integer("25")]);

This is a curious change in CLDR.  Hope we don't break anyone's over-expectations.
Attachment #8987033 - Flags: review?(jwalden+bmo) → review+

Updated

9 months ago
Attachment #8987034 - Flags: review?(jwalden+bmo) → review+
Comment on attachment 8987031 [details]
bug1466471-part2-icu62-update.zip

rs=me
Attachment #8987031 - Flags: review?(jwalden+bmo) → review+
(Assignee)

Comment 11

9 months ago
The V8 team found two ICU62 regressions [1,2] which probably affect us, too. I don't yet know if ICU will push a dot-release or if we'll need to apply the patches ourselves.

[1] https://ssl.icu-project.org/trac/ticket/13851
[2] https://ssl.icu-project.org/trac/ticket/13852
AFAICS, the https://ssl.icu-project.org/trac/ticket/13851 bug won't affect case conversions used by Gecko layout (e.g. CSS text-transform:lowercase), as we don't use the ICU u_strToLower API to implement these; but it does look like it will affect toLocaleLowerCase() in JS, so we'll still need the fix for that.
(Assignee)

Comment 13

9 months ago
https://ssl.icu-project.org/trac/ticket/13852 doesn't seem to affect us, because we always set the UNUM_SIGNIFICANT_DIGITS_USED flag before modifying UNUM_MIN_SIGNIFICANT_DIGITS/UNUM_MAX_SIGNIFICANT_DIGITS. So we can proceed with the update after pulling the patch for https://ssl.icu-project.org/trac/ticket/13851.
(Assignee)

Comment 14

9 months ago
Imports the case conversion patch from ICU.
Attachment #8990197 - Flags: review?(jwalden+bmo)
(Assignee)

Comment 15

9 months ago
Attachment #8987038 - Attachment is obsolete: true
Attachment #8990198 - Flags: review+

Updated

9 months ago
Attachment #8990197 - Flags: review?(jwalden+bmo) → review+

Comment 17

8 months ago
Pushed by apavel@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/289ff944eadc
Part 1: Update patches for ICU 62. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/2ba22d8e422a
Part 2: Update in-tree ICU to release 62.1. rs=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/35887b27bae5
Part 3: Update Intl tests after ICU update. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/68eca3cfb965
Part 4: Update to Unicode 11 for SpiderMonkey. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/dce28e8cfcd6
Part 5: Regenerate Unicode files after ICU update. r=jfkthame
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c84b1a609df
Part 6: Import patch to fix case conversion bug. r=Waldo
https://hg.mozilla.org/integration/mozilla-inbound/rev/4a310672bb05
Part 7: Updating ICU requires a clobber. r=Waldo
Keywords: checkin-needed
As always, a new version of ICU is making coverity lose his mind :)
181 new defect(s) introduced to Firefox found with Coverity Scan.
190 defect(s), reported by Coverity Scan earlier, were marked fixed in the recent build analyzed by Coverity Scan.
(even if we are now ignoring icu)
status-firefox62: affected → wontfix
See Also: → bug 1476304

Updated

6 months ago
Depends on: 1495244
You need to log in before you can comment on or make changes to this bug.