update Unicode character data to version 9.0

RESOLVED FIXED in Firefox 53

Status

()

defect
P3
normal
RESOLVED FIXED
3 years ago
2 months ago

People

(Reporter: epinal99-bugzilla2, Assigned: jfkthame)

Tracking

unspecified
mozilla53
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox53 fixed)

Details

(URL)

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
+++ This bug was initially created as a clone of Bug #1183209 +++

Unicode 9.0 is out (released 2016.06.21), so we should update our data.
Simon, are you up for this?

Also, no idea what priority this bug would have in the light of https://wiki.mozilla.org/Bugmasters/Process/Triage#How_Do_You_Triage
Flags: needinfo?(smontagu)
(Assignee)

Comment 2

3 years ago
We may want to wait until there's an ICU release with Unicode 9.0; we rely on their constants for things like script and line-break codes, so a mismatch between the version of ICU we're using and our other character-property data may be problematic.

Oh, on the other hand we may need to hold off on updating ICU, as there may be WinXP compatibility issues with that; last I heard, they were planning to drop support for XP in ICU58. So updating may be trickier than usual....
Waldo said that he'd backport ICU updates for XP for our in-tree copy. At least gandalf said that Waldo said .. ;-)
(Reporter)

Updated

3 years ago
Blocks: 1288961
(Reporter)

Updated

3 years ago
Blocks: 1305010
...yeah, backporting is an uncertain task.  If we're just copying backwards CLDR data files and the like, that's one thing.  But if the CLDR data files change format -- as I understand they're doing right now, or are likely to do very soon now, to include desirable localization info *that we want to use* -- it becomes a trickier proposition.  Backporting extra parsing code as well ups the risk/complexity level.

Given that bug 1215247 finally has indications of headway on getting ICU into Firefox on the last OS/platform we ship on, it may be worth waiting for that to happen, as much as we can.
:waldo, so when the new ICU comes out do you want us to hold back on upgrading to it until we have ICU in Android before we port it?
Flags: needinfo?(jwalden+bmo)
Oh, I see.

Ok, so basically - starting with Fx 53 we're dropping support for WinXP (bug 1305453), 53 opens up on 2016-11-07 which means that by the time we get ICU 58 (with no support for WinXP), we'll be able to merge it without any porting. Yay!
Flags: needinfo?(jwalden+bmo)
Depends on: xp-eol
Priority: -- → P3
(Assignee)

Comment 8

3 years ago
The ICU58 update (including Unicode 9 data) was landed in bug 1299615, so we should go ahead with the corresponding update here. It turned out to be not-quite-trivial because the format used for the UTR39 (security) data has changed.
Attachment #8810071 - Flags: review?(m_kato)
(Assignee)

Comment 9

3 years ago
The generated data tables. (The two parts will need to be folded together for landing.)
Attachment #8810072 - Flags: review?(m_kato)
Attachment #8810071 - Flags: review?(m_kato) → review+
Attachment #8810072 - Flags: review?(m_kato) → review+
(Assignee)

Comment 10

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/5d9a785a37c4cb6468ef5b3b610b8b84c4a337b7
Bug 1281448 - part 1+2 - Update character property table generator script for Unicode 9 (in particular, security/xidmodifications.txt is replaced by security/IdentifierStatus.txt and IdentifierType.txt), and adjust APIs to fit the new identifier-type property model; update the generated data files. r=m_kato
Backed out for Android bustage:

https://hg.mozilla.org/integration/mozilla-inbound/rev/f9f15852463a68cef4cfa7b36d82f1234de8f215

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=5d9a785a37c4cb6468ef5b3b610b8b84c4a337b7
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=39135150&repo=mozilla-inbound

[task 2016-11-14T09:37:07.348042Z] 09:37:07     INFO -  In file included from /home/worker/workspace/build/src/obj-firefox/intl/unicharutil/util/internal/Unified_cpp_util_internal0.cpp:47:0:
[task 2016-11-14T09:37:07.348143Z] 09:37:07     INFO -  /home/worker/workspace/build/src/intl/unicharutil/util/nsUnicodeProperties.cpp: In function 'const nsCharProps2& GetCharProps2(uint32_t)':
[task 2016-11-14T09:37:07.348223Z] 09:37:07     INFO -  /home/worker/workspace/build/src/intl/unicharutil/util/nsUnicodeProperties.cpp:70:5: error: narrowing conversion of '-1' from 'int' to 'unsigned char' inside { } [-Werror=narrowing]
[task 2016-11-14T09:37:07.348564Z] 09:37:07     INFO -       };
[task 2016-11-14T09:37:07.351145Z] 09:37:07     INFO -       ^
[task 2016-11-14T09:37:07.351237Z] 09:37:07     INFO -  /home/worker/workspace/build/src/intl/unicharutil/util/nsUnicodeProperties.cpp:70:5: error: large integer implicitly truncated to unsigned type [-Werror=overflow]
[task 2016-11-14T09:37:07.352461Z] 09:37:07     INFO -  cc1plus: all warnings being treated as errors
Flags: needinfo?(jfkthame)
(Assignee)

Comment 12

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/7cdecdce25a6ba2fa7fd1198bdd0233b057d259d
Bug 1281448 - part 1+2 - Update character property table generator script for Unicode 9 (in particular, security/xidmodifications.txt is replaced by security/IdentifierStatus.txt and IdentifierType.txt), and adjust APIs to fit the new identifier-type property model; update the generated data files. r=m_kato

Comment 13

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/7cdecdce25a6
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
Does this still block bug 1130266 (xp-eol) when http://site.icu-project.org/download/58 states 'This is the last ICU4C release that works on Windows XP and Windows Vista.' ?
(Assignee)

Comment 15

2 years ago
(In reply to Nick Thomas [:nthomas] from comment #14)
> Does this still block bug 1130266 (xp-eol) when
> http://site.icu-project.org/download/58 states 'This is the last ICU4C
> release that works on Windows XP and Windows Vista.' ?

I think this was intended to depend on (not block) the XP/Vista EOL, based on earlier proposals for dropping XP support from ICU, but as we've been able to take ICU 58 (and Unicode 9) without losing XP, we can forget that dependency.

Updating to ICU 59 (when the time comes for that) will presumably be dependent on the XP EOL.
No longer depends on: xp-eol
Flags: needinfo?(jfkthame)

Updated

2 years ago
Flags: needinfo?(smontagu)
Assignee: nobody → jfkthame
Duplicate of this bug: 1305010
You need to log in before you can comment on or make changes to this bug.