1221034 - Fix failures on unicode-bidi tests

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Xidorn Quan [:xidorn] UTC+11]

Please enter page given in the URL field, and scroll down to section "Results for prefixed property values". This is a test set from W3C L18N working group.

As of the new unicode-bidi values, we have passed all tests for "isolate" value, but fail on some of tests for "isolate-override" and "plaintext".

Richard Ishida told me that, Aharon Lanin, a bidi expert in Google, suggested us to move to use ICU's implementation of Unicode Bidirectional Algorithm, which may implement a more up-to-date Unicode spec.

I have no idea whether we should do that, or try to fix the current implementation. It seems this moving could be hard, so probably only fixing the current impl is fine.


This is the email:
On 19/05/2014 13:56, Aharon (Vladimir) Lanin wrote:
> BTW, Firefox has a couple of bugs in its unicode-bidi:isolate
> implementation that I suspect will preclude them from making the change
> in the default stylesheet. Those bugs are supposed to be fixed by FF's
> move to the ICU implementation of the UBA, which implement's Unicode
> 6.3's directional isolate formatting characters, but that move seems to
> be stalled for now.

Comment 1

[Simon Montagu :smontagu]

We are kind of halfway through updating our UBA implementation. Ted added support from ICU for the isolation *control characters* in bug 1157726, but that doesn't affect elements styled with unicode-bidi. I tried to add in the missing parts in bug 1160847, but didn't manage to complete the work at that time.

Comment 2

[Xidorn Quan [:xidorn] UTC+11]

It seems bug 1160847 doesn't fix this issue. We feed the bidi engine with the correct sequence, but the engine returns an incorrect fragmentation.

For example, for the following HTML fragment:
<div dir="rtl">a &gt; <span style="unicode-bidi: isolate-override; direction: rtl">&#x05D1; &gt; &#x05D2;</span> &gt; d</div>

We give the engine:
U+0061 U+0020 U+003e U+0020 U+2068 U+202e U+05d1 U+0020 U+003e U+0020 U+05d2 U+202c U+2069 U+0020 U+003e U+0020 U+0064

And the engine returns:
2: U+0061
1: U+0020 U+003e U+0020 U+2068
5: U+202e U+05d1 U+0020 U+003e U+0020 U+05d2
1: U+202c U+2069 U+0020 U+003e U+0020
2: U+0064

But I think the engine should really return:
2: U+0061 U+0020 U+003e U+0020 U+2068
5: U+202e U+05d1 U+0020 U+003e U+0020 U+05d2 U+202c
2: U+2069 U+0020 U+003e U+0020 U+0064

That says, our impl of the bidi engine may need update as well.

Comment 3

[Xidorn Quan [:xidorn] UTC+11]

Guess the engine has some issue with nesting override directly inside isolate.

Comment 4

[Xidorn Quan [:xidorn] UTC+11]

It seems with bug 1160847 fixed, we would pass all tests for plaintext, but not isolate-override.

Comment 5

[Xidorn Quan [:xidorn] UTC+11]

The latest ICU bidi engine outputs the following:
> level 2: U+0061 U+0020 U+003e U+0020 U+2068
> level 5: U+202e U+05d1 U+0020 U+003e U+0020 U+05d2
> level 2: U+202c U+2069 U+0020 U+003e U+0020 U+0064
which also makes sense as far as nothing is resolved to level 1 in this sequence.

That said, replacing our own bidi engine with that from ICU would also fill the gap here.

Comment 6

[Xidorn Quan [:xidorn] UTC+11]

Fixed by bug 1160847 and bug 1281988. We already passed all unicode-bidi tests on CSSWG repo (with the values unprefixed).