94319 - (symbolic-fonts) Symbolic fonts do not display properly, need generic solution rather than adding each new font to fontencoding.properties

Status: VERIFIED FIXED

(Core :: Layout, defect)

Description

[Alex Weyers]

Currently, when a symbolic font is utilized in a font face tag (ie. <font 
face="wingdings">), the text between the font tags will not be displayed using 
the symbolic font unless that particular font name has been appended to the 
fontencoding.properties file (indicating that the font should be treated as 
though it were utilizing a standard encoding (see Bug 77265)).

This solution is hardcoded and therefore not very adaptible.

If the user adds a new font symbolic font to their system, they will very 
likely expect to be able to utilize it in the browser or email.  Tediously 
adding each new font to this hardcoded list is unwieldy and inelegant.

Is there a way to specify a more generic situation?  Perhaps a flag for 
embedders to indicate that, when a symbolic font is encountered, they would 
just like to default to treating the font as though it utilizes windows-1252 
encoding?


The following is the list of fonts that were not properly displayed during a 
recent test:
Map Symbols 
Marlett 
Monotype Sorts 
MS Outlook 
MT Extra 
Symbol
Wingdings 2
Wingdings 3

Comment 1

[Frank Tang]

Attachment: test cases

Comment 2

[Frank Tang]

ok, more test cases, go to 
http://www.geocities.com/SiliconValley/Screen/6615/dingbat.htm download those 
fonts and try the following test cases. 

Comment 3

[Frank Tang]

Attachment: test page for fonts in http://www.geocities.com/SiliconValley/Screen/6615/dingbat.htm

Comment 4

[Frank Tang]

Attachment: propose fix ver1 based on 92 branch

Comment 5

[Frank Tang]

The patch basically said
If we got a Symbol encoded font and we don't know it's name, we will use 
"windows-1252" as the encoding. 

rbs and shanjian, can you code review this one ? Thanks.

Comment 6

[Frank Tang]

roy, and alex, can you try my patch in your build and verify it first?
I think we still have a sperate issue with "Symbol" font name itself. 

Comment 7

[Hixie (not reading bugmail)]

This whole issue is one that I believe should be WONTFIXed.

Letting fonts be used in this way is EXTREMELY platform-specific, and will
result in pages that are not compatible across different devices, thus breaking
the whole point of the web. We should not be encouraging platform-specific
practices like this.

If people want to use special glyphs, they should pick the relevant ones from
UNICODE. That will ensure the glyph will appear correctly on all platforms.

Comment 8

[Frank Tang]

Ian- In general, I agree with you. The problem is such idea issue block more 
users to use Gecko since it break compatability.  We have top customer ask us to 
fix that for them. 
That is why I put this patch here. 

Comment 9

[Marek Z. Jeziorek]

While the proposal may be bad for the web it is actually needed by our embedding
clients. Hence a proposal, ship this code contolled by a pref with a default
"good for the web" and allow the embedding client to set it to "bad for the web"
value.

Comment 10

[Tim Hill]

Bug 61188 is going to need some way to access the Marlett font from XUL (it
needs Windows-specific scrollbar arrow glyphs), so this fix would help. 
However, if there was some other easy way to insert these characters (apparently
there's a "private use" area in Unicode that might be useful for this
purpose..?), that'd be fine too.

Comment 11

[ronbdavis]

Could this be made a setting on the nsIWebBrowserSetup interface, passing a 
string as a default mapping, such as (to be reviewed and ok'd by the embedding 
team)

nsCOMPtr<nsIWebBrowserSetup> pIWebBrowserSetup(do_QueryInterface(pIWebBrowser));
if (pIWebBrowserSetup) {
   pIWebBrowserSetup->SetProperty
(nsIWebBrowserSetup::DEFAULT_FONT_MAPPING, "windows-1252");
}

Or something like that. That would be remove the hardcoded default, be 
consistent with other embedding properties, and allow embeddors to tailor their 
applications...which may have nothing to do with the web.

nsIWebBrowserSetup::SetProperty() would have to be able to set a string. And I 
don't know how far apart this interface is from the font mapping code.

Comment 12

[Frank Tang]

Is that necessary that it have to be control by nsIWebBrowserSetup? Is it 
acceptable that is controled by a value in the all.js file instead?
I don't see a need for run time configuration  here. Is it accept that is a 
build time configuration ? or if you really prefer a run time configuration, is 
that acceptable to change it through nsIPref instead. Also, I really dont' want 
to allow any one supply a string argument such as "windows-1252", I only want to 
expose a boolean flag. true or false in those configuration. 

Comment 13

[Judson Valeski]

nsIWebBrowserSetup should not be a dumping ground for stuff like this. prefs is
a dumping ground, and therefore, IMO, this belongs there.

Comment 14

[Ken]

I can live with a preference setting.

Comment 15

[Ken]

I can live with a preference setting. how would it be set up? As a bool?

Comment 16

[Frank Tang]

I will propose a new patch.

Comment 17

[Frank Tang]

Attachment: better patch, use a pref to make embeding client decide which one to pick

Comment 18

[Frank Tang]

Change 
pref("win.font.enablesymbol", false); 
to 
pref("win.font.enablesymbol", true);

will make it render these fonts. 

rbs- can you r= this one ?

Comment 19

[Hixie (not reading bugmail)]

Our "embedding customer" doesn't need this any more than Netscape does. If we
implement this, it will be a very serious bug.

Sites that "require" this behaviour should be evangelised. The correct way to
display characters like this is to use the correct UNICODE codepoint. If a font,
like Marlett, has special characters that are mapped to the PUA, then we should
allow them to be accessed from the PUA, but that is also to be strongly 
discouraged as it is also non-portable.

If you still think that we should "fix" this in some way that allows a Gecko-
based product to render web pages that use symbol fonts as if the symbol fonts
were Windows-1252 encoded, then please e-mail me explaining exactly why we need
this more than fixes to other parts of the product. (I am under AOL NDA.)

Comment 20

[Frank Tang]

This is a "paper standard" vs "de factor standard" issue, not a standard vs 
non-standard issue. 
The reality is every version of window browser support one behavior. I agree 
with you that our embeded clients have the same need as Netscape. So we probably 
should fix them for both. 

[e-mail] : can you try the patch

Comment 21

[Frank Tang]

Ian: I totally respect your opinion about "paper standard". However, I think we 
also need to face the reality and deal with de factor standard which current 
exist. 

"The correct way to display characters like this is to use the correct UNICODE 
codepoint."
First of all, Unicode define "character", it does not define "Glyph". Therefore,  
if a font define character "E" look like a "Elephane", then we should display it 
as an "Elephane". The shape of "Glyph" is not defined in Unicode, nor defined in 
HTML 4.0, it is defined in the TrueType Specification. And I think my 
implementation are confirm to the TrueType specification.

Comment 22

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

From section 1.2 of the Unicode Standard, version 3.1.1:

"The primary goal of the development effort for the Unicode Standard was to
remedy two serious problems common to most multilingual computer programs. The
first problem was the overloading of the font mechanism when encoding
characters. Fonts have often been indiscriminately mapped to the same set of
bytes. For example, the bytes 0x00 to 0xFF are often used for both characters
and dingbats."

This has nothing to do with the TrueType specification.  If you know the
characters aren't the ones you're supposed to get, then you're not conforming to
Unicode.  Period.

Comment 23

[Alex Weyers]

Frank - 

You latest patch works for me.  And since it is a windows preference with the 
default set to adhere to WWW standards, it really should not present much of an 
issue for other embedders.  

I'm still seeing the problem you mentioned with the Symbol font itself.  But I 
hope this code gets reviewed and checked in at some point.

Comment 24

[Frank Tang]

fixed and check into m92 with the pref turn on.

Comment 25

[Frank Tang]

fix for m92

Comment 26

[Alex Weyers]

Frank - 

Has there been any progress on the separate issue with the "symbol" font itself?

Comment 27

[Frank Tang]

reopen, we need to land this into m9.4 branch
I talk to marek in the pdt meeting. he said we got a pdt+ for it. 

Comment 28

[Frank Tang]

assign

Comment 29

[Alex Weyers]

Frank - 

Has there been any progress on the separate issue with the "symbol" font itself?

We will need any fix on the 0.9.4 branch.

Thanks.

Comment 30

[Frank Tang]

ok, we land it into m94 tree from yokoyama's account. close this bug. 

Comment 31

[Chris Petersen]

Based on last comments, marking verified with the
Oct 1 branch build.

Comment 32

[Pierre Saslawsky]

Apparently this bug was checked in on the 0.9.2 and 0.9.4 branches without proper 
reviews and approvals.  Neither the bug report nor the CVS comments mention a 
review/super-review and there is no trace of approval for 0.9.2 either (0.9.4 was 
approved by Marek).

The problem is that this fix works on Windows only while we would need an XP 
solution for bug 33127 if we decide to support the Symbol font.  Why wasn't the 
"fontEncoding.properties" file implemented on all platforms?

Comment 33

[Hixie (not reading bugmail)]

During a meeting I attended at Netscape back in August, it was decided that this
was never to be checked in on the trunk, but that an alternative would be
implemented on the long term. The alternative selected was to change the UI of
the mail compose window (in Mozilla as well as embedding customers) to have UI
specifically for picking dingbat characters. This UI would then insert the
appropriate UNICODE character, thus removing any need for special treatement of
Symbol or Wingdings fonts.

This UI is already present in some forms in many editors and some mail clients.
It would be similar to what we already have for smilies.

Comment 34

[Shanjian Li]

Attachment: patch for branch 1.0

Comment 35

[Shanjian Li]

many things have been changed, so the patch looks very different from the
original one. And a mistake in fontencoding.properties file have to be addressed
as well.

Comment 36

[Shanjian Li]

The branch patch works well on current trunk too. 
reopen the bug for consideration for trunk.

Comment 37

[Shanjian Li]

reassign to myself.

Comment 38

[Shanjian Li]

We constantly have embed product vendor asking for this feature, and the code in
gfx is constantly changing. In order to avoid working on this issue again and
again, I propose to check this feature to trunk but disable it through
preference. For those vendor who request this feature, they can enable it
themselves. 

Asking for r/sr.
frank, could you r=?

Comment 39

[Hixie (not reading bugmail)]

No. Please see comment #33 where the embedder in question comitted to finding an
alternative solution.

Comment 40

[Shanjian Li]

Ian, the suggestion in #33 will not solve all the problems. How about those
existing websites on the web? I understood your argument that this is a practice
against one of the initiatives of unicode. If you take a look at the existing
code, there is already some code there handling symbol font for windows. To make
this mechanism works better is not a step backward, and besides this feature
will not be enabled for mozilla. 

Comment 41

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I object to this for two reasons:

 * The embedder in question (if my memory is correct) has not demonstrated that
there is any need to change our behavior as a user-agent on the web, and is not
interested in our behavior as a user-agent on the web.  However, this patch
changes things in a way that does affect our behavior as a user-agent on the web
and is thus harmful to the forward progress of the web.  This is thus a much
larger (and more harmful) change than the one needed to satisfy the embedder's
needs.  (That said, since I would like Mozilla to be as close as possible to a
standards-compliant rendering engine, I would probably also object to the amount
(and visibility) of code necessary to make it a run-time option.)

 * The patch is Windows-only, and I'm getting tired of seeing Windows-only
patches go in (such as bug 76097 and bug 156943) with seemingly no intention of
fixing them for other platforms.

Comment 42

[Shanjian Li]

I don't think we have big disagreement againt the nature of this problem. The
disagreement is in how to handle this issue. Base on different standpoints and
different priority lists, a consesus sometimes just can't be reached without
some kind of comprimise. I don't want to further address my arguments, the
existing ones are clear enough. Think about why we have to put the patch to 092
branch, we will have to do the same thing again and again to branches. We agreed
to disable this patch on mozilla distribution, I don't know if there is other
alternative choice. As long as we can not eliminate such practice over the web,
the requirement will be raised again and again. If the patching of branch can't
be avoided, keep this from trunk will only make my life miserable and nothing
more. The user agent used by embedding vendors will still be patched as long as
they are not convinced to drop this requirement. (Since changing the world will
never be on vendors' priority list, I personally don't think they will change
their mind until the world has been changed.) So the end result is the same one
the web.  

Comment 43

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

My memory of the previous meeting discussing this was that the embedder
interested in this (as Ian noted in comment 33) was interested in it for a mail
application rather than for browsing the web.

Comment 44

[Hixie (not reading bugmail)]

This is a duplicate of bug 33127. I hadn't realised they were two separate bugs
until I looked for my comment describing the only possible compromise I would be
willing to make, namely (bug 33127 comment 119) limiting this to the <font> tag 
only (i.e. CSS wouldn't be affected). That would actually still be vaguely 
standards-compliant, would be a good way of weening people off Symbol, and would 
satisfy the Symbol-in-mail requirement. It would also mean we wouldn't have to 
implement it as a quirk. I imagine it would be relatively easy to implement 
(internally turm the Symbol font into -moz-Symbol if it is sot on the <font> 
element, and go from there).

*** This bug has been marked as a duplicate of 33127 ***

Comment 45

[Shanjian Li]

I investigate the possible solution for ian's suggestion in comment #119. I just
located the place when font tag is parsed. (CNavDTD.cpp, CollectAttributes, this
might not be the best place to do this though.) I have a feeling the such an
implemenation (hacking font name and later resolving it in font resolution code)
will not be easy, and it will have an impact on performance as well. In font
resolution code, we have to de-hack everyfont name and mark those hacking font
name with an additional flag. Then later when a font is determinted to be in
symbol encoding, this flag can be checked and proper action be taken. There are
many disadvantage in this approach, 
1, The implemenation will be hacky and it makes code difficult to understand and
maintenance. 
2, The behavior of font tag and CSS specification is inconsistence, that will
confuse people even more.
3, Font resolution is in performance critical code, string comparison and
de-hack operation may have some impact on performance. 
4, The change will involve multiple modules (gfx for sure, one in htmlparser or
content or both). This will be not an easy change. 

I would rather patch every branch we give to AOL instead of this one. 

Ian, I am not sure if you realize the system font handle for wingding was in the
trunk for a long time. Applying my patch in 94319 will not make mozilla worse in
terms of standard compatibility, but only let mozilla works well with other
symbol fonts. 

Please let me know you opinion. We need to reach a consensus quickly, no matter
what that is. 

Reopen this bug. 

Comment 46

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Hacking the parser is definitely the wrong thing to do.  You shouldn't need to
go any further from GFX than MapAttributesIntoRule in nsHTMLFontElement.cpp. 
That's the last point where we know whether something came from a font element
or not.  From there (and perhaps from other rule mapping functions, if you add a
new variable), you'd need to propagate a new variable through style data
calculation (nsRuleNode::ComputeFontData), out into nsStyleFont, and from there
to GFX.

Comment 47

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(Passing the information through as a string modification to the name is one
(hacky) way of passing the information from the rule mapping code to GFX, but
there are others.)

Comment 48

[Frank Tang]

>You shouldn't need to go any further from GFX than MapAttributesIntoRule in 
>nsHTMLFontElement.cpp. 
Agree. I reach the same conclusion when I study the code independently from
dbaron  and shanjian this afternoon.
Shanjian, please look at  
content\html\cpntent\src\nsHTMLFontElement.cpp
    231 static void
    232 MapAttributesIntoRule(const nsIHTMLMappedAttributes* aAttributes,
    233                       nsRuleData* aData)
    234 {
    235   if (!aData)
    236     return;
    237
    238   if (aData->mFontData) {
    239     nsCSSFont& font = *(aData->mFontData);
    240     nsHTMLValue value;
    241
    242     // face: string list
    243     if (font.mFamily.GetUnit() == eCSSUnit_Null) {
    244       aAttributes->GetAttribute(nsHTMLAtoms::face, value);
    245       if (value.GetUnit() == eHTMLUnit_String) {
    246         nsAutoString familyList;
    247         value.GetStringValue(familyList);
    248         if (!familyList.IsEmpty())
    249           font.mFamily.SetStringValue(familyList, eCSSUnit_String);
    250       }
    251     }
    252
I think adding code between line 248 and 249
familyList.Append(",--moz-from-html-font-tag-face-attribute");
before calling 
font.mFamily.SetStringValue(familyList, eCSSUnit_String);

could be the hacky way to pass this information from content to GFX

if we think this is not a good way to pass this type of information from content
to gfx, then shanjian please open a new bug to the "Style System" components
(assign to the default bug owner of "Style System") about the need of a
mechanism to carry this information in the nsCSSFont struct and make this bug
blocked by that. In that case, please also open a 2nd new bug that request a
mechanism to pass this kind of information from layout to gfx. That probably
need a nsIFontMetrics api changes. Also mark this bug blocked by that 2nd new one.
We probably won't need to open that two bugs if people think passing this
information in this hacky is "ok" for now.

Comment 49

[Frank Tang]

>internally turm the Symbol font into -moz-Symbol if it is sot on the 
><font> element, and go from there
hum.... I didn't see this before I put into my last comment.

Well. The problem is NOT JUST the font named "Symbol". The problem also happen
to TTF font which the 'cmap' encodingID == eTTMicrosoftEncodingSymbol

Comment 50

[Frank Tang]

> TTF font which the 'cmap' encodingID == eTTMicrosoftEncodingSymbol
including the following fonts:
Map Symbols 
Marlett 
Monotype Sorts 
MS Outlook 
MT Extra 
Symbol
Wingdings 2
Wingdings 3

so... what we need to pass a boolean information from the nsHTMLFontElement.cpp
to the gfx nsFontMetricsWin.cpp to indicate that the current font-family spec is
from a html <FONT FACE > instead from a font-family in css. If that boolean
information (it is a boolean information, but it does not mean the way to pass
need to be a PRBool type) is true, then we handle the font which has
eTTMicrosoftEncodingSymbol if the boolean information is false then we skip that
font in our fallback code. 

Comment 51

[Frank Tang]

>Has there been any progress on the separate issue with the "symbol" font itself?
shanjian- do we have the solution for that ?
Does your patch cover that part too ?

Comment 52

[Frank Tang]

>1, The implemenation will be hacky and it makes code difficult to understand 
>and maintenance. 
Agree. 

>2, The behavior of font tag and CSS specification is inconsistence, that will
>confuse people even more.
Agree, but that is not necessary a bad thing to do.

>3, Font resolution is in performance critical code, string comparison and
>de-hack operation may have some impact on performance.
disagree. we don't need to do a lot of string comparision. We need to append the
font name in the nsHTMLFontFamily.cpp which will only impact the performance in
the FONT tag
We then need to do one string search in the Init function of nsFontMetricsWin.
We may even lazy eval that untill we hit a font which are encoded in symbol
encoding int the TTF 'cmap'. That won't cause a lot of performance.  
>4, The change will involve multiple modules (gfx for sure, one in htmlparser or
>content or both). This will be not an easy change. 
It depened on how we going to pass the information. If we just try tp append a
special name in the fontFamily, then you only need to change the
nsHTMLFontElement.cpp and nsFontMetricsWin.cpp. If we decide to change the
nsCSSFont struct, then we definitely need to change more places. 

One possibility is to use the "variant" in nsFont to pass that information from
Layout to gfx. and for nsCSSFont, can we add
#define NS_STYLE_FONT_VARIANT__MOZ_SYMBOL_FONT_ENABLE 2 
#define NS_STYLE_FONT_VARIANT__MOZ_SYMBOL_FONT_ENABLE_MASK 2 
to pass that information? 

Comment 53

[Frank Tang]

Attachment: test cases according to ian's suggestion

Comment 54

[Frank Tang]

Attachment: prototype according to ian's suggestion

no font face hack. pass the information thorugh the "variant" field of the 
nsCSSFont and carried into nsFont.mVariant

use a bitmask for that 0x80

Shanjian- can you carry the prototype? with this patch on my system, I can show

http://bugzilla.mozilla.org/attachment.cgi?id=103289&action=edit
with the left hand side display with those font and right hand side 
display as [0-9][A-Z][a-z]

Comment 55

[Frank Tang]

notice that I remove the encoding of "Symbol" totally from the
fontEncoding.properties file. If I keep it as adobe, then both side will show me
as A-Z, if I change it to windows-1252 as shanjian's patch, then both side will
show me as Greek letters. If I remove it, then it will show as I expect. I can
still show ΑΒΔαβδ on my system as greek.
Probably it use other font. However, I am not sure what kind of dependency from
MathML on the Symbol . Will we break MathML if remove that entry? If that is the
case, then we need to add some logic inside nsFontMetricsWin.cpp to switch the
encoding according to the 0x80 bit of the mFont.variant. return windows-1252 if
the bit is set, and return Adobe-Standard-Encoding if that bit is clear. 

Comment 56

[Frank Tang]

ok, does people consider the way in my patch passing the boolean information
hacky or ok ?
No hacky string in font name. No string comparision. No new methods or data
field needed in api. Backward compatable with the current meaning of the api.
(except that I need to mask off the 0x80 in the nsTextFrame.cpp) 

Any opinion about the implementation, not the behavior, now (in term of passing
the information) ?

Comment 57

[Frank Tang]

Attachment: screenshot with my fix to view http://bugzilla.mozilla.org/attachment.cgi?id=103289&action=edit

The screen shot show you we display with the font in the
left hand side but not on the right hand side
the lefthand side use <FONT FACE="> the right hand side use font-family:

Notice that "Map Symbols" and "Monotype Sorts" are display the same in my
impage. This is because I don't have that two font installed on my system

Comment 58

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 103298 [details] [diff] [review]
prototype according to ian's suggestion

This isn't going to work right since:

 1) In the current style system code the variant isn't treated as a bitmask,
but rather as a value.
 2) Other rule mapping functions that sent the font family would need to set
something that says the symbol-font hack should be undone, so that the
CSS-specified fonts in the descendants of a FONT element wouldn't be treated
incorrectly.

Essentially, that means that while it would be possible to use the mVariant
value in the style struct (set in nsRuleNode::ComputeFontData), it's not
acceptable to use it in the rule struct (nsCSSFont), and you need to add a new
value.

Comment 59

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Bug 33127 comment 149 suggests that IE6 behaves correctly.  If that is the case,
I would be against changing our correct behavior.

Comment 60

[Michael Dunn]

Related: bug 167607

Comment 61

[Shanjian Li]

Frank's suggestion is much better than the one I am talking aobut in #45.

>> 1) In the current style system code the variant isn't treated as a bitmask,
>>but rather as a value.
>> 2) Other rule mapping functions that sent the font family would need to set
>>something that says the symbol-font hack should be undone, so that the
>>CSS-specified fonts in the descendants of a FONT element wouldn't be treated
>>incorrectly.
 I think add a new value should be fine. I added something like:
#define NS_STYLE_FONT_VARIANT_FROM_FONT_TAG    2
Since the only other variant besides normal is small cap, and font tag can not
be used to specify small cap font (is it right?). So we should be fine to just
use this value. 

>>Bug 33127 comment 149 suggests that IE6 behaves correctly.  If that is the 
>>case,I would be against changing our correct behavior.
That's not true. I just verified this on my winXP win IE 6.0.2800.1106. 
I totally agree not to change the behavior of mozilla. This patch is for
embedded client only, the code will not be enabled for trunk and
mozilla/netscape release.

Comment 62

[Shanjian Li]

Attachment: patch as suggested by frank.

Comment 63

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Adding a new bit and making font-variant a bitmask will not cascade correctly,
which will cause some CSS-specified fonts to be broken, as I said in my previous
comment.

Comment 64

[Shanjian Li]

*** Bug 167607 has been marked as a duplicate of this bug. ***

Comment 65

[Frank Tang]

open a new bug 175372 "need a new value in nsCSSFont to indicate the fontFamily
is set by HTML FONT element instead from css and pass from nsHTMLFontElement to
GFX" about how to pass information from nsHTMLFontElement.cpp to GFX (or layout).
What david said make sense. I think our style system need to be enahnced to make
the fix of this bug correct. Split that part to "Style System" as bug 175372.
Once that part is there, we can use that work to solve this issue in here. 

Jud, please help to make sure people working on Style System can support us to
address this issue. It take team work to make thing happen. :) 

Comment 66

[Hixie (not reading bugmail)]

Whatever is done for this should also remove the Wingdings hack, and should be
enabled in the Mozilla trunk as well as in the aformentioned embedded client.

Comment 67

[rbs]

Do people want to keep going with the patch or instead use a symbolic Character
Picker as alluded in comment 33? At the time that comment was entered, there
were no such picker. But now, the JavaScripted MathML Editor includes
cross-platform XUL palettes for picking symbolic characters, and interested
folks could possibly port them here. I am going to attach a screenshot for
illustration. The JS MathML Editor is at:
http://www.newmexico.mackichan.com/MathML/mathmled.htm
http://www.newmexico.mackichan.com/MathML/mathmled4.xpi

Comment 68

[rbs]

Attachment: screenshot - Character Picker

Comment 69

[Shanjian Li]

the picker could not solve the problem of compatibility with exist mail client,
right?

Comment 70

[rbs]

I see. It is a pain to emulate that broken backward compatibility.

Comment 71

[Michael Dunn]

Internal Reference:
http://bugscape.netscape.com/show_bug.cgi?id=19819

Comment 72

[Hixie (not reading bugmail)]

So is anything going to happen here? I thought this was an "urgent" issue last
month, which is why I agreed to the compromise solution, but if it's not
actually that important, then maybe we should just go back to WONTFIX.

Comment 73

[Frank Tang]

Ian: This IS an urgent issue. But this is also depend on 175372. We still need
to depend on 175372 to be fixed first in order to do anything with this. 

Comment 74

[Hixie (not reading bugmail)]

You could always take the patch in that bug, apply it to your tree, and work on
that before the other one was checked in...

Comment 75

[Frank Tang]

shanjian- 175372 is landed. Do you know how to fix this bug with that change now ?

Comment 76

[Shanjian Li]

Attachment: test case to include font webdings and wingdings

Comment 77

[Shanjian Li]

Attachment: new patch use the flag defined in CSS by dbaron.

Comment 78

[Shanjian Li]

dbaron, thanks a lot for your effort on 175372, it make my life much easier now. 

ian, "hack" for wingdings and webdings font has been removed in my new patch.
Now they will only work in font tag as other symbol fonts.

ftang, rbs/dbaron, could you r/sr?

Comment 79

[Hixie (not reading bugmail)]

Please don't make this Windows-only.

Also, please don't make this pref-controlled. We have too many hidden prefs, and
they are way too much work to QA. This should just be always enabled.

I'm glad we're making good progress on this though!

Comment 80

[rbs]

Agree with hixie about loosing the pref -- especially considering that
mFont.familyNameQuirks is now doing its filtering.

(I am in the process of reviewing the patch to double-check that it hasn't
regressed other legitimate Symbol glyphs that are referenced via their Unicode
values.)

Comment 81

[rbs]

I am observing a regression when referencing glyphs by their Unicode values.
This is likely due to the removal of "encoding.symbol.ttf = Adobe-Symbol-Encoding".

For the Symbol font, the character map that is displayed on the fly by Mozilla
should match, glyphIndex-by-glyphIndex, with the screenshot of the font's own
character map:
http://www.mozilla.org/projects/mathml/fonts/encoding/symbol-encoding.html
http://www.mozilla.org/projects/mathml/fonts/encoding/symbol.gif

Comment 82

[Shanjian Li]

rbs, can you verify your judgement in comment #81? If MATHML is referencing some
glyph in symbol font using Adobe-Symbol-Encoding, we have sacrifice one usage
(MATHML or font tag assuming symbol font in win-1252). I don't think symbol font
is heavily used and important to embedding customer. I can put that line back,
but need to confirm. 

Comment 83

[rbs]

Do you get a full match on your system with the encoding test page that I
showed?

Notice that it is not MathML specific. The test page is using 
  <span class="glyph">&#xUnicodeValue;</span> 
with the definition of the class as .glyph {font-family: Symbol;}. 
It is plain HTML. Also, it seems to me that the actual purpose of the
Adobe-Symbol-Encoding was to encode their Symbol font. Something else to bear in
mind is that bug 17962 relies on that as a fallback for other HTML4 characters.

It might be possible to further tweak your patch to yet again special-case the
Symbol font itself. And so when the quirks flag is set, the code uses the
windows-1252 encoding as you are doing now; if the flag isn't set, you use
Adobe-Symbol-Encoding.

Comment 84

[Hixie (not reading bugmail)]

rbs is correct, the following line is not a quirk:
   encoding.symbol.ttf = Adobe-Symbol-Encoding
...these are:
   encoding.wingdings.ttf = windows-1252
   encoding.webdings.ttf = windows-1252
The first should stay, the last two should be replaced by their real encoding.
(If we can work it out. There are some glyphs in these two, Wingdings
especially, that map quite well to real UNICODE glyphs.)

Comment 85

[Hixie (not reading bugmail)]

Comment on attachment 106018 [details] [diff] [review]
new patch use the flag defined in CSS by dbaron.

chenging review+ to review- since there are at least three issues with this
patch:
 1. It is Windows specific.
 2. It screws up the standards compliant case of Symbol usage.
 3. It is pref controlled.

Comment 86

[Shanjian Li]

>> 1. It is Windows specific.
This may or may not be a problem on MAC and *nix. They will be handled
separately even if it is a issue. I just filed bug 179945 for mac and 179946 for
linux. TTF on linux is still in infant stage, but it might be a problem in future. 

>> 2. It screws up the standards compliant case of Symbol usage.
Inconsistency is much more evil and confusing that missing functionality. Since
symbol font encoded characters that have been defined by unicode long time ago
and most people is aware of this fact, I　suggest to put the encoding for symbol
back there. In my testing, I found most of the glyphs in symbol font can be
covered by other fonts. I would like rbs to confirm the importance of symbol
font in MATHML. It is replacable, using my existing patch also make some sense.
I still prefer the first one.

>> 3. It is pref controlled.
Done

Comment 87

[Shanjian Li]

Attachment: new patch (removed hidden pref, put back adobe... for symbol font)

Comment 88

[Frank Tang]

Comment on attachment 106100 [details] [diff] [review]
new patch (removed hidden pref, put back adobe... for symbol font)

r=ftang + again. All the 3 issue raised by ian have been addressed:
issue 2 and 3 is no longer in the new patch
issue 1 is spin off to two other bugs for platform specific functionality
enhancement and should be tracked by that two other bugs. This bug is window
only in the beginning and still is.

Comment 89

[Hixie (not reading bugmail)]

Yep, that patch addresses my concerns.

dbaron?

Comment 90

[rbs]

>In my testing, I found most of the glyphs in symbol font can be
>covered by other fonts.

That's not really the point in this case. Ideally, what is needed is that for
<span class="Symbol">&alpha;</span>, it should pick alpha from the Symbol font
if given the class ".Symbol {font-family: Symbol}".

> I would like rbs to confirm the importance of symbol font in MATHML.
> It is replacable

Yep, it is critical for MathML in order to get a proper composition of stretchy
characters (e.g., large parenthesis). Indeed, for stretchy characters to work in
a cross-plaform manner, the MathML code precisely asks for "&#xUnicodeValue in
the Symbol font" (or some other font for that matter). If GFX doesn't honor the
request and instead pick a glyph in another font, then the rendering will suffer
from ugly mis-alignments that people will be quick to complain about. There is a
description of why this contract is needed at:
http://www.mozilla.org/projects/mathml/fonts/encoding/

> using my existing patch also make some sense.

I am alright with the updated patch since it doesn't break MathML, but notice
that it is now not going to make a difference for people who have been asking to
special-case the Symbol font in particular -- bug 33127). Are your alright with
that? (BTW, there was a mail that Pierre once circulated about the issue.)

=============
code-review comments (essentially nits):

+#define ENCODING_FOR_SYMBOL_FONT "windows-1252"
+GetDefaultConverterForSymbol(

Mind renaming to 'WINDOWS_CODEPAGE_ENCODING' and GetWindowsCodePageConverter?
(When I see the other names, I can't help think that there referring to the
Adobe's Symbol font.)

=============
+GetConverterCommon(nsString& value, nsIUnicodeEncoder** aConverter)

For uniformity with the naming convention being used in the file, use
'aEncoding' instead of 'value'.

Comment 91

[Shanjian Li]

Attachment: update patch

Comment 92

[Shanjian Li]

Comment on attachment 106156 [details] [diff] [review]
update patch

carry over review

Comment 93

[Shanjian Li]

rbs, 

Symbol font is a troublesome issue. We (me and ftang) discussed other possible
solutions this morning. Considering the fact that we current have one ccmap for
one font, and the ccmap is cached, it will be a big change to handle both
<font="symbol">a</font> and <font="symbol">&alpha;</font>. For the reason I
mentioned in #86, we decide to go with the current approach. 

I update the patch as you suggested, but with a modification in name. 

Comment 94

[rbs]

Comment on attachment 106156 [details] [diff] [review]
update patch

sr=rbs

Comment 95

[Shanjian Li]

fix checked in. 

Comment 96

[R.K.Aa.]

i built with this fix, but testing now: where the testcases say "This should NOT
display as abcdef" etc, it still displays as "abcdef". Etc.

I have at least marlett and wingdings installed. Was it decided to have a pref
for this after all, and what do i have to modify sin order to see/use the fonts
real face?

Comment 97

[rbs]

You need the patch for bug 175372 as well.

Comment 98

[R.K.Aa.]

Thanks. That was checked in already on the 12th, so there must be something
wrong with my build - will clobber.

Comment 99

[Katsuhiko Momoi]

There is a remaining issue of supporting the Symbol font, which
was not addressed by the final patch because suporting it would 
cause other problems.  I created a new bug to address this remaining 
issue: Bug 195038.

Comment 100

[Teruko Kobayashi]

Verified as fixed in 5-6 Win32 trunk build.

Comment 101

[Jon Roland]

This is to report I am unable to view <font face="symbol"> ... </font> in Moz
1.4 under Win 98SE and 1.5b under WinXP. It was previously working in 1.0, 1.1,
1.2, and 1.3. It seems to have re-emerged, so needs to be re-opened.

I have previously been reporting this to Bug #213702, which presumably needs to
be converged with this one.

It seems we need a general solution that can display any TTF or other compliant
font found in the system FONTS directory without having to do any further setup.

Comment 102

[rbs]

jdr, the fix didn't include Symbol for fear of breaking MathML. See comment 90
and comment 93.

What "works" is <font face="Wingdings"> ... </font> and friends, in quirks mode.

Comment 103

[Jungshik Shin]

rbs, you're quick :-) I was about to reply. I made a patch for GFX:GTK/Xlib with
Freetype2 that makes Symbol font work the same way as other 'symbol' fonts (bug
208213) without interfering with MathML's use of Symbol font. I'll see if it can
be ported back to GFX:Win. 

Comment 104

[ibell]

(In reply to comment #102)
> jdr, the fix didn't include Symbol for fear of breaking MathML. See comment 90
> and comment 93.

 There is a web legacy issue here.
 I have a website (www.ibell.oc.uk/maths) heavily reliant on FONT FACE tag 
invokation of symbols font. MathML should not be an issue when displaying 
declared HMTL 3.2. 

Comment 105

[rbs]

This was an old issue. The symbol font has been fixed as well a while back in
bug 195038.

Comment 106

[ibell]

(In reply to comment #105)
> This was an old issue. The symbol font has been fixed as well a while back in
> bug 195038.

 So next release of FireFox will support symbol font inclusion via FONT FACE tag?

Comment 107

[rbs]

It already does. Try, e.g., the first test cases attached to this bug:
https://bugzilla.mozilla.org/attachment.cgi?id=45324

Comment 108

[ibell]

(In reply to comment #107)
> It already does. Try, e.g., the first test cases attached to this bug:
> https://bugzilla.mozilla.org/attachment.cgi?id=45324

 OK yes. Symbol font inclusion mostly works if !DOCTYPE tag is either absent or
fully correct (I'd missed out 'Draft'). But, like MSIE 5+, FireFox annoyingly
fails for Symbol character 0xAD [up arrow] . A similar symbol is available using
uarr entity but not in HTML 3.2, uarr failing under Netscape 4.5-.
  

Comment 109

[rbs]

The DOCTYPE sniffing trickery is well documented here:
http://www.mozilla.org/docs/web-developer/quirks/doctypes.html

As to the fact that it "mostly works" or "annoyingly fails", it simply works
like pre-CSS browsers. Thus you have a common baseline that saves you from doing
something specific as you initially feared. Solution to all worries anyway:
HTML4 & Unicode (↑ for up arrow). You won't miss much because not that
many people still use Netscape 4.5-.

Comment 110

[ibell]

> As to the fact that it "mostly works" or "annoyingly fails", it simply works
> like pre-CSS browsers.

 Netscape 3 and 4.5 display symbol font uparrow OK. Furthermore the uarr entity
is not an entirely equivalent symbol. It descends below the baseline and has a
punier arrow head. 

> Thus you have a common baseline that saves you from doing
> something specific as you initially feared. Solution to all worries anyway:
> HTML4 & Unicode (↑ for up arrow).

 Fails under Nestcape 3.0 , still my favourite browser. My position is that I
have a site coded in HTML 3.2 that works under NS4.5- but fails under FF (and
MSIE6) and I can't make it FF compatible without abandoning 3.2 and loosing NS
4.5-. What's the problem with character 0xAD anyway?

Comment 111

[Hixie (not reading bugmail)]

If you want to use the particular up arrow from the Symbol font, specify the
Symbol font, as in (HTML first then accompanying CSS):

   <span>It's up there! &uarr; Wee.</span>

   span { font-family: Symbol, Verdana, sans-serif; }

...which will use the Symbol font for the arrow, and the Verdana font for the text.

Comment 112

[Florian Bender]

(This tricked my Bugzilla search, renaming alias to make it more distinct to harmony-symbol. Sorry for bugspam!)