70610 - emsp, ensp, thinsp entities don't display properly

Status: RESOLVED WORKSFORME

(Core :: Layout, defect)

Description

[Wayne Davison]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.17-21mdk i686; en-US; 0.9) Gecko/20010228
BuildID:    2001022808

The entity "emsp" is supposed to be rendered as a space character with the width
of an "M" in the current font.  The entity instead displays with no width at all.

Reproducible: Always

Comment 1

[Boris Zbarsky [:bzbarsky]]

I see this as well, in a CVS build from the evening of 2001-02-28.

This is not a problem on win2k apparently, so probably Unix-only.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Attachment: Testcase

Comment 3

[bsharma]

qa contact updated.

Comment 4

[tenthumbs]

Mozilla seems to treat U+2003 (em-space) and U+2002 (en-space) as
regular characters, at least partially.

The standard X fonts have neither glyph. Mozilla goes on its generic
glyph hunt, turns up nothing, and renders nothing.

If a font is available with U+2003, then mozilla will use that font,
even if it's different from the surrounding text font. It also appears
that once it finds U+2003 it doesn't look for U+2002. The display looks
wrong.

Obviously, mozilla should not use some random font but should fall back
to U+0020 in the current font.

BTW, Unicode says U+2003 is equal to the nominal font size in points,
more or less, and U+2002 is half that. This could be another solution.

Comment 5

[bsharma]

changing the component to 'Layout'.

Comment 6

[karnaze (gone)]

Reassigning to Erik.

Comment 7

[Frank Tang]

erik resign. reassign all his bug to ftang for now.

Comment 8

[Frank Tang]

mark all future new as assigned after move from erik to ftang

Comment 9

[Christopher Hoess (gone)]

Should this be added to the "HTML 4 named character tracker" bug?

Comment 10

[Boris Zbarsky [:bzbarsky]]

Yes, it should.  ccing bstell who seems to be working on Unix fonts...

Comment 11

[kill this account]

> The standard X fonts have neither glyph. Mozilla goes on its generic
> glyph hunt, turns up nothing, and renders nothing.

Sounds like it needs to be in the transliterator.

> Obviously, mozilla should not use some random font but should fall back
> to U+0020 in the current font.

Is this a request for special handling of U+2002 and U+2003?

Comment 12

[Christopher Hoess (gone)]

This is now WFM on a fresh Linux CVS build.  Can other people still see this?

Comment 13

[Boris Zbarsky [:bzbarsky]]

I'm still seeing this with the linux 2002-03-21-21 nightly.

Comment 14

[dcary]

I see this with Mozilla 1.1a (Build 2002061104) on Win98.
A quick hack to remap the entity "emsp" and the entity "ensp" to the entity
"nbsp". would be an improvement over what I'm seeing now. (Would the simple
space character U+0020 be even better ?). I'm seeing "entity. This" like a
single word (as if the "emsp" in the middle were completely deleted). When I
resize the window and force the words to reflow, it also acts like a single
word, moving to the next line all-or-nothing. (I think "emsp" is supposed to act
like whitespace and allow a line break ... or is it supposed to glue things
together like "nbsp" ?)

Apparently Mozilla is already treading those entites special -- with other
entities where it turns up empty (such as entity "hArr"), I see that Mozilla
displays a "?" question mark.

Comment 15

[Boris 'pi' Piwinger]

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4a) Gecko/20030331

Still seeing this problem.  ,  ,   and their numerical
equivalents ( ,  ,  ) show as squares with EMSP, ENSP, THSP
written into them, respectively.

As long as the real spaces are not available it would be an improvement to show
just normal spaces.

pi

Comment 16

[Roland Mainz]

smontagu:
Any comments/ideas ?

Comment 17

[Benjamin Smedberg]

Are the emsp, ensp, and thinsp entities supposed to mark word-wrap boundaries or
not?

Comment 18

[Hervé Cauwelier]

This bug has been opened for too long...

My experiement:

Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.7.5) Gecko/20041116 Firefox/1.0
(Ubuntu) (Ubuntu package 1.0-2ubuntu3)
Default font is Bitstream Vera Sans.

I see large spaces in the Test Case, not sure they're the size of a M or equal
to the current font-size. However both emsp and ensp renders the same.
As seen on the URL field, that test page gives the same results but I'm glad to
know that thinsp seems to work!

Together with #194498, the reign of entities for formatting and compositing rich
text is not over...

Comment 19

[Simon Montagu :smontagu]

The transliterator currently replaces both em space and en space by a single
ASCII space character. We could use multiple spaces (compare bug 264946).

Comment 20

[Boris Zbarsky [:bzbarsky]]

Multiple spaces wouldn't do the right thing, since they'd just get collapsed in
the rendering... the right thing to do would be several non-breaking spaces
followed by a single space, I think.

Comment 21

[Simon Montagu :smontagu]

If I'm not mistaken the transliterator is used in font code, later than any
space collapsing. Worth testing at least, no?

Comment 22

[Boris Zbarsky [:bzbarsky]]

Hmm.. yes, it is.

Note that in a current build I can't seem to reproduce this.  It may be that I
now have a font with these chars in it.

Comment 23

[Benoit]

Just for the record, the testcase does work correctly with Windows (2000 or XP). 

From these three characters, the one that we miss the most in French is "thinsp"
that should theorically be used before several punctuation signs (';', ':' and
quote marks). It renders quite well as intended on Windows, but breaks on Linux. 

If it is not possible to display it on Linux, could it at least be replaced by
"nbsp"? (it is not clear thinsp should really be non-breaking, but there is
currently no reasonable alternative. See the discussion on
http://www.cs.tut.fi/~jkorpela/html/french.html#spacing )

Comment 24

[Julian D. A. Wiseman]

1.5RC3 windows: thinsp doesn't work, rendering in my example (first bullet of Next Steps section of http://www.jdawiseman.com/papers/easymath/coin-stopping.html#next-steps ) as an extremely wide space. 

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/1.5

Comment 25

[Victor Engmark]

This problem is still present in 2.0.0.5 on Windows for /some/ files. Attaching SVG which doesn't work properly (all spaces are displayed as boxes). http://www.digitalmediaminute.com/reference/entity/, on the other hand, works OK.

Comment 26

[Victor Engmark]

Attachment: Minimal SVG test with literal characters and hex code

Comment 27

[Boris Zbarsky [:bzbarsky]]

roc, does new textframe or thebes now handle this for fonts that don't have em-space or en-space chars?

Comment 28

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Not explicitly. But they have changed the behaviour: those characters will now trigger the Unicode/Pango/ATSUI path, so if those layers have any special support for these characters, we'll get it. On my Mac, the testcase seems to work fine.

If there are still problems, we can either fix them with platform-specific tweaks, or we can put a solution in gfxTextRunWordCache similar to (but more complicated than) its handling of zero-width space. So basically we need people to test trunk builds on various platforms, including different versions of Pango, Uniscribe and ATSUI. The results may also vary depending on the fonts installed.

Comment 29

[Karl Tomlinson (:karlt)]

WFM on Linux, probably just due to font support.
DejaVu fonts (version 2.28 at least) support U+2002, U+2003, and U+2009.
Lucida Sans (from sun-jdk-1.6.0.13) also supports these characters.

Comment 30

[Boris Zbarsky [:bzbarsky]]

Right.  This bug is about the case when you don't have a font with these characters; to properly test it you'd probably need to uninstall some fonts...

Comment 31

[Jonathan Kew [:jfkthame]]

For completeness, we ought to support the whole collection of various widths of space; there must be a dozen or so in Unicode (in the U+200x row), and any of them could occur as a numeric character entity. Things like U+2006 may be less widely implemented in fonts.

Actually, if we give these any kind of special handling, perhaps it would be better to directly implement them as versions of space or   with overridden width, or something like that, rather than doing font fallback at all. In most cases, there's a clearly-specified width that they should be; and if there isn't, it doesn't really make any more sense to "borrow" some other font's idea of a "thin space" than to arbitrarily define it in terms of a fraction of an em.

Comment 32

[Karl Tomlinson (:karlt)]

(In reply to comment #31)
> For completeness, we ought to support the whole collection of various widths of
> space; there must be a dozen or so in Unicode (in the U+200x row), and any of
> them could occur as a numeric character entity. Things like U+2006 may be less
> widely implemented in fonts.

FWIW, DejaVu (except for Mono, for good reason, i assume) and Lucida Sans support U+2000 - U+200A.

> Actually, if we give these any kind of special handling, perhaps it would be
> better to directly implement them as versions of space or   with
> overridden width, or something like that, rather than doing font fallback at
> all. In most cases, there's a clearly-specified width that they should be; and
> if there isn't, it doesn't really make any more sense to "borrow" some other
> font's idea of a "thin space" than to arbitrarily define it in terms of a
> fraction of an em.

No more sense, no.  But for spaces defined in terms of em, it shouldn't matter which font is used, either.

(In reply to comment #30)
> Right.  This bug is about the case when you don't have a font with these
> characters; to properly test it you'd probably need to uninstall some fonts...

There must be plenty of characters that we could try to generate when the font has no support (pre-combined characters with marks, for example).

But if a readily available font that the user can install has support for the characters then I don't see a need to provide a special implementation,
especially when the most widely installed font on the platform has support for the characters.

Comment 33

[Boris Zbarsky [:bzbarsky]]

I'm not sure why you're treating this bug as Linux-specific...  This is almost certainly a problem on any platform with an insufficiently rich set of fonts installed, no?

Comment 34

[Karl Tomlinson (:karlt)]

(In reply to comment #33)
> I'm not sure why you're treating this bug as Linux-specific...

Mac has fonts with support for these characters as does Vista.  If there are problems on those platforms then that is a platform-specific font-fallback problem.

I don't know whether there is support on XP, but I don't see a report here of a problem with a recent build on XP.

> This is almost certainly a problem on any platform with an insufficiently
> rich set of fonts installed, no?

Yes.  But, looking at the present and to the future such platforms do not seem common.

Comment 35

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

(In reply to comment #34)
> Yes.  But, looking at the present and to the future such platforms do not seem
> common.

Mobile devices, perhaps?

Comment 36

[Mats Palmgren (inactive)]

The testcases works fine for me on Linux.