911849 - zero-width no-break spaces  and ⁠ not respected during line-breaking

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Alex Dixon]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.62 Safari/537.36

Steps to reproduce:

Added zero-width break (&#200b;) and no-break (&#feff;) spaces to help control line breaks in a Japanese string.


Actual results:

It looks like these are being ignored as this example shows: http://jsfiddle.net/DRydx/5/


Expected results:

zero width breaking spaces (&#200b;) should be used as a hint to break, as regular spaces are, and zero width non-breaking (&#feff;) should not be broken.  Chrome and IE10 both work correctly in the example, and Safari does if you remove the <span> tags used to help show what's going on.

Comment 1

[Alex Dixon]

I should have also mentioned that I don't speak Japanese, and this is an issue for other languages as well. This is pretty important for being able to translate responsive designs, where the width isn't fixed (and you can't hardcode the line breaks).

Comment 2

[Tiziana Sellitto [:tiziana]]

I can reproduce this on Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:26.0) Gecko/20100101 Firefox/26.0 ID:20130903030201 

maybe similar or related to bug 783180

Comment 3

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Attachment: 911849.html

Current contents of http://fiddle.jshell.net/DRydx/5/show/

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I'd like to see an analysis of the expected behavior here from somebody more familiar with the relevant line breaking rules and what these characters are supposed to do.

(In reply to Alex Dixon from comment #1)
> I should have also mentioned that I don't speak Japanese, and this is an
> issue for other languages as well. This is pretty important for being able
> to translate responsive designs, where the width isn't fixed (and you can't
> hardcode the line breaks).

Inserting control characters into the middle of text from another source really doesn't seem like a good way to address this problem.  CSS properties (e.g., those in css3-text) that control breaking seem better, though I don't remember the status of our implementation of them.

Comment 5

[Jonathan Kew [:jfkthame]]

While I agree that in most cases, it's probably better to use CSS properties to control line-breaking behavior, the fact that Firefox is allowing breaks within

  コミュニ

does seem like a bug. (Incidentally, this usage of U+FEFF is deprecated; the preferred Unicode character to use would be U+2060 WORD JOINER. But this doesn't change the Firefox behavior; it still allows a line break, which it shouldn't.)

AFAICS, the zero-width space (U+200B) does behave as expected; try changing the text in the example to

  <h1>supercali&#x200b;fragilistic&#x200b;expiali&#x200b;docious</h1>

and note how the ZWSP characters provide line-break opportunities.

Comment 6

[Jonathan Kew [:jfkthame]]

Attachment: Unicode characters U+FEFF (zero-width no-break space) and U+2060 (word joiner) should inhibit line-breaking.

I don't have a particularly deep understanding of all the line-break classes and rules, as implemented in nsJISx4051LineBreaker, but afaict simply assigning U+feff and U+2060 to CLASS_NON_BREAKABLE seems to improve the behavior here. If you agree this is a reasonable thing to do, we should also create some reftests for the behavior.

Comment 7

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Comment on attachment 799458 [details] [diff] [review]
Unicode characters U+FEFF (zero-width no-break space) and U+2060 (word joiner) should inhibit line-breaking.

Looks good to me, please add reftest for them.

r=masayuki

Comment 8

[Jonathan Kew [:jfkthame]]

Attachment: reftest for use of ZWNBSP/WJ to inhibit line-breaks in CJK text.

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Comment on attachment 800108 [details] [diff] [review]
reftest for use of ZWNBSP/WJ to inhibit line-breaks in CJK text.

>+
>+# test that ZWNBSP and WJ correctly inhibit breaks within CJK text (bug 911849)

I don't think these lines are necessary because every can find the bug# from annotation.

Comment 10

[Jonathan Kew [:jfkthame]]

Fair enough; pushed without the extra comment.

https://hg.mozilla.org/integration/mozilla-inbound/rev/2e7256075903 (patch)
https://hg.mozilla.org/integration/mozilla-inbound/rev/73be965472ba (test)

Comment 11

[Ryan VanderMeulen [:RyanVM]]

This reftest was failing on OSX due to one glyph not being positioned quite the same. I added fuzz for now, but you may want to investigate to see if that was the correct solution or not.
https://hg.mozilla.org/integration/mozilla-inbound/rev/4d2b2fac577f

Comment 12

[Jonathan Kew [:jfkthame]]

Thanks....yeah, it's still correctly showing the effect of the patch here; the glyph-positioning discrepancy is probably due to the glyph runs in the testcase being broken up and drawn separately due to font fallback for the zero-width no-break space characters.

I think we can fix this up so as to eliminate the fuzziness, by having the zero-width chars present in the reference as well, even though no attempt is being made to line-break there. I'll give this a whirl on tryserver, and push a followup if it works out.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/2e7256075903
https://hg.mozilla.org/mozilla-central/rev/73be965472ba
https://hg.mozilla.org/mozilla-central/rev/4d2b2fac577f

Comment 14

[Jonathan Kew [:jfkthame]]

Followup to eliminate the test fuzziness on OS X:
https://hg.mozilla.org/integration/mozilla-inbound/rev/76634fb95c41

Comment 15

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/76634fb95c41