Open Bug 1212668 Opened 4 years ago Updated 4 years ago

Text is clipped from the start of lines

Categories

(Core :: Graphics: Text, defect)

x86
Windows 10
defect
Not set

Tracking

()

People

(Reporter: karimarif, Unassigned)

Details

(Whiteboard: [gfx-noted])

Attachments

(7 files)

Attached image Firefox bug comparison
User Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.64 Safari/537.36

Steps to reproduce:

1) Install Jameel Noori Nastaleeq Font. Link here:
https://dl.dropboxusercontent.com/u/1294856/JNN.7z
2) Type a paragraph in simple html page


Actual results:

The text is not rendered as expected. All text lines are cut from their beginnings.


Expected results:

The font includes space kerning for every ligature. It kerns fine between the lines but not at the beginning of lines, causing all words to cut. I have attached the results from Internet Explorer, Microsoft Edge and Mozilla Firefox for comparison.
This font is used by millions of Urdu speakers across the globe, so we expect an urgent fix. Thanks
Severity: normal → major
OS: Unspecified → Windows 10
Hardware: Unspecified → x86
Attached file Test Page
Use this html page to reproduce bug if needed.
Component: Untriaged → Graphics: Text
Product: Firefox → Core
Summary: Space kerning is cutting text from the start of lines → Space kerning is clipping text from the beginning of lines
Summary: Space kerning is clipping text from the beginning of lines → Text is clipped from the start of lines
Version: 41 Branch → 44 Branch
Version: 44 Branch → Trunk
As you can see from the snapshots, none of the other known browsers have major issues with space kerning but Firefox.
Hei. Could some developer or moderator at least confirm the bug as new? I have checked and confirmed in all builds of Firefox.
Thank you for filing this bug report. Let me preface this with the fact that I do not understand how to read this language so I could be missing something that's absolutely obvious to you.

From what I can tell Firefox is wrapping the text earlier than it is in Chrome but no text is actually being "cut off" or lost. That's what I see when I test it locally and what I see in your screenshots, but perhaps I have misunderstood how you've explained the issue.

For future reference, we're a small team and get hundreds of bug reports every day. While I understand that you consider this to be an urgent issue, we have to prioritize this against things like crashes, security vulnerabilities, severe rendering errors, and serious performance regressions. I ask that you come to Bugzilla with a bit more understanding and a lot more patience. Your bug will be marked NEW when it has been confirmed by someone on the team. I suggest you read https://bugzilla.mozilla.org/page.cgi?id=etiquette.html before commenting in Bugzilla any further. 

Thank you.
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #4)
> Thank you for filing this bug report. Let me preface this with the fact that
> I do not understand how to read this language so I could be missing
> something that's absolutely obvious to you.
> 
> From what I can tell Firefox is wrapping the text earlier than it is in
> Chrome but no text is actually being "cut off" or lost. That's what I see
> when I test it locally and what I see in your screenshots, but perhaps I
> have misunderstood how you've explained the issue.

Thank you for your kind reply. I can understand that it could be a bit difficult to acknowledge this text clipping issue when someone is unaware of the said language. Anyhow, I have added another screenshot which proves that it's not a text wrapping issue. As you can see from the sample; the text is wrapping at exactly the same point in both FireFox and Edge. Yet in Firefox, from the second line onward, its being clipped from the right side. I hope this would make it easier to understand the issue.
Text clipping is visible in red box.
Okay, I can see this now. Thanks for explaining it more clearly.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Severity: major → normal
John, are you the right person to look into this?
Flags: needinfo?(jdaggett)
Whiteboard: [gfx-noted]
I think the question here is why the text is wrapping the way it does in Firefox with the Jameel Noori Nastaleeq font under Windows. The page is HTML output from Word.

Simon, could you take a look at this?
Flags: needinfo?(jdaggett) → needinfo?(smontagu)
Attached file Minimized testcase
This isn't primarily a wrapping issue, nor is it Windows only. The attached two-word testcase is cut off on the right, and the first letter of the second word overlaps the last letter of the first word (shown blown up on the second line). Neither of these things happens in Chrome.

(The wrapping differences are a knockon effect: the over-enthusiastic kerning affects the width of the text, and that in turn affects where the text wraps)
Flags: needinfo?(smontagu) → needinfo?(jfkthame)
The font uses a lot of (contextual) kerning with the <space> glyph, so it's plausible that bug 761442 (see range in comment 11) could have affected the behavior; John, can you look into that?

Having said that, I suspect the problems may run deeper. The 'kern' feature in this font actually gives the <space> glyph NEGATIVE advance width, and I'm not at all confident our text-run measurement/drawing/etc code will handle that at all nicely.

(FWIW, this seems to confuse Chrome somewhat, too; not as far as rendering is concerned, but when you try to select parts of the text something's a bit "off"...)
Flags: needinfo?(jfkthame) → needinfo?(jdaggett)
(In reply to blinky from comment #11)
> Testcase
> https://dl.dropboxusercontent.com/u/95157096/85f61cf7/11a0zoyad4.html
> 
> Regression range:
> https://hg.mozilla.org/mozilla-central/
> pushloghtml?fromchange=2949e808ed33&tochange=7b8ed29c6bc0

Note that prior to this range, we didn't have the clipping problem, but because kerning with inter-word spaces was not supported at all back then, we were not actually rendering the font entirely as designed.
(In reply to Jonathan Kew (:jfkthame) from comment #12)
> Having said that, I suspect the problems may run deeper. The 'kern' feature
> in this font actually gives the <space> glyph NEGATIVE advance width, and
> I'm not at all confident our text-run measurement/drawing/etc code will
> handle that at all nicely.

As the lead developer of this font, I can confirm that it is indeed the case. The 'kern' feature gives the <space> glyph NEGATIVE advanced width to produce desired kerning between glyphs with space. By doing so, it effectively ignore the spaces between glyphs. This is exactly how Farsi / Urdu Nastaliq (cursive script) should kern. Space 'kern' feature has therefore been long supported by OpenType for this purpose only as narrow, thin or even zero-width space is of no use when it comes to kerning in cursive scripts.

On a further note regarding how this was implemented in font; the GSUB table defines / decomposes all glyphs with space like this:
kaf -> space kaf
lam -> space lam

Combining that with <space> glyph's negative advanced width in GPOS table, we achieved kerning with and without space in a single table. Per current results, the font is rendering perfectly as designed, yes even better than Chrome actually, except of course this text clipping issue at the beginning of lines.
Jonathan, not quite sure what you're asking to investigate but this appears to be an extents problem. I'm guessing your concern was that we weren't catching whether there were interactions with spaces correctly? Even if I substitute 'true' for the condition on the line below, the testcase renders identically. So I don't think that's the cause.

https://dxr.mozilla.org/mozilla-central/source/gfx/thebes/gfxFont.cpp#2751

Attaching an extended testcase. Chrome and Webkit both display the complete string correctly with no clipping. Depending upon whether kerning is applied or not, glyphs overlap. Not sure why Safari never enables kerning, maybe that's something CoreText-related?
Flags: needinfo?(jdaggett)
(In reply to John Daggett (:jtd) from comment #15)
> Created attachment 8692805 [details]
> testcase, with kerning explicitly enabled/disabled
> 
> 
> Jonathan, not quite sure what you're asking to investigate but this appears
> to be an extents problem. I'm guessing your concern was that we weren't
> catching whether there were interactions with spaces correctly? Even if I
> substitute 'true' for the condition on the line below, the testcase renders
> identically. So I don't think that's the cause.

Right, it's not about detecting interaction with space; we get that right (otherwise we wouldn't kern at the inter-word spaces at all, and the problem wouldn't arise).

But this isn't primarily an extents issue. Note that even if the glyphs weren't getting clipped (which is what a purely extents problem could cause), the positioning of the entire line of text would be wrong.

Note that if you remove all the leading/trailing whitespace within the <p> elements in your test, the problem no longer happens to the "سال گزشتہ" text, even when kerning is enabled. But the presence of the leading space (which gets discarded for rendering) effectively shifts the origin of the following word (because during textrun construction/measurement, it was kerned back over the non-rendered space), and so the visible text is offset to the right of where it should be.

I think part of the issue may be something like this: if we have <glyph1><glyph2>, and there's a (negative) kern between them, we apply half the kern by reducing the advance of <glyph1> and half of it by shifting <glyph2> backwards (offsetting it back from its origin). This is the nicest way to apply kerning to glyph pairs in general, but when the first glyph is actually <space> and it then disappears at start-of-text or into a linebreak, we're left with <glyph2> shifted backwards.

In fact, we should not be applying kerning across linebreaks or against discarded whitespace. Perhaps that's really the root of the problem here. But at the point when we shape the text, we don't yet know what whitespace is going to disappear into linebreaks, and we don't re-shape after linebreaking. (That's a longstanding known bug, but it's hard to fix in general because of the potential perf cost.)
Aha, some further insight: I think the root of the problem here is actually cluster formation. Specifically, for some reason that I haven't dug into yet, harfbuzz is assigning the same cluster value to the <space> and the word-initial character (or ligature). So in the textrun, we end up with the glyphs for [space] and [seen-alef] in a single cluster associated with the <space> character, and the <seen> and <alef> characters are marked as non-cluster-start with no glyphs of their own.

Then, when we come to draw the run starting at the <seen> (i.e. skipping the initial <space>), we see that we're starting to draw 1/3 of the way through a cluster, and so we only draw 2/3 of it.

Evidence of the cluster formation happening in harfbuzz (rather than in our interface code, which I first suspected):

  $ hb-unicode-encode 20,633,627,644 | util/hb-shape JNN.ttf --cluster-level 1
  [lam.l=3+1107|blank=3+1|sa.l=0+1094|space=0+1]

Note that both 'space' and 'sa.l' are assigned cluster 0, while 'blank' (an artifact of the font's shaping lookups, I guess) and 'lam.l' are assigned cluster 3.

We can prevent harfbuzz doing this if we run it with cluster-level = HB_BUFFER_CLUSTER_LEVEL_CHARACTERS (instead of HB_BUFFER_CLUSTER_LEVEL_MONOTONE_CHARACTERS, which we currently use):

  $ hb-unicode-encode 20,633,627,644 | util/hb-shape JNN.ttf --cluster-level 2
  [lam.l=3+1107|blank=3+1|sa.l=1+1094|space=0+1]

Here, the <space> has remained separate from the <sa.l> ligature. And if we do this (in gfxHarfBuzzShaper), then suddenly the testcase appears correctly, with no clipping at the start of lines.

But I'm not sure whether this is a change we can safely make, or if it'll cause problems in other complex cases....
I think the problematic cluster formation happens because of stuff that's going on in the 'rlig' feature of the font. It looks like lookup index 45 inserts a <blank> glyph in front of each ligature (such as the <sa.l> ligature for the characters <seen, alef> here); that glyph will inherit its cluster value from the ligature to which it's being prefixed. Then, there's a lookup (index 52) that recognizes the sequence <space, blank> and ligates it to a simple <space>. This ligation, which crosses the cluster boundary between the <space> and <blank, sa.l>, causes the clusters to merge.

So I believe it would be possible to avoid the problem by restructuring the font lookups so as to avoid this ligation; either by not inserting the <blank> at all when there's a preceding <space>, or by using a different technique to remove it (can a Contextual Substitution lookup replace one glyph with zero?).

Whether we can work around this in Gecko (without changes to the font) just by using the higher harfbuzz cluster-level remains to be confirmed.
You need to log in before you can comment on or make changes to this bug.