251637 - Form input control fonts don't respect the language of the document

Status: RESOLVED WORKSFORME

(Core :: Layout: Form Controls, defect)

Description

[Seungbeom Kim]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040616
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040616

Mozilla seems to always show the form input controls with the system default
font for input controls, regardless of the language (or character encoding) of
the document. Under a Korean version of Windows XP, this mars the look-and-feel
of English pages, since the system default font for inputs controls goes well
with Korean but not English.

This is in contrast with MSIE, which respects the encoding and shows the input
controls in appropriate fonts.

Reproducible: Always
Steps to Reproduce:
See the reference URL and the attached screenshot.
Actual Results:  
The form input control text goes well with LANG=ko, but not with LANG=en.

Expected Results:  
It should respect the language (or character encoding) of the document and
display the input controls in appropriate fonts.

Comment 1

[Seungbeom Kim]

Attachment: screenshot of the reference URL

Comment 2

[rbs]

Is this a screenshot from what you see with Mozilla or IE? Care to attach both?

Comment 3

[rbs]

Attachment: patch to sanitize font stuff

1) make nsFormControlHelper::GetFrameFontFM() be language-aware.
2) remove GetFont() that used to be there for some quirks stuff (according to
an XXX comment).
3) remove GetTextSize() that nobody uses anymore.

Not sure if 1) actually fixes this bug. I cannot tell because my locale in EN.

Comment 4

[rbs]

Comment on attachment 153391 [details] [diff] [review]
patch to sanitize font stuff

Let see what reviewers think....

Comment 5

[Jungshik Shin]

Attachment: a better test case

This test case demonstrates a lot more conspicuous difference in the font
selection between non-form text and form-control-text than the test case at the
URL.

rbs, you can see the difference regardless of your locale, can't you? 

I'll apply your patch and see if it fixes the bug.

Comment 6

[Jungshik Shin]

escalating to normal and adding Richard Ishida of W3 to Cc.
this could be rather 'critical' in some situations (CJK distinction, Indic text
rendering, etc).

also assigning to rbs (why don't you take it? you have a patch although it has
yet to be tested)

Comment 7

[Jungshik Shin]

Unfortunately, the patch doesn't work ...

BTW, in the patch, you forgot to remove NS_RELEASE() corresponding to
|nsIFontmetrics *fontMet| you replaced by |nsCOMPtr ....|. 

Comment 8

[Seungbeom Kim]

(In reply to comment #2)
> Is this a screenshot from what you see with Mozilla or IE? Care to attach both?

The screenshot is from Mozilla. MSIE does not respect the LANG attribute, but
the document character encoding affects the whole document. I'll attach the
screenshots from MSIE too.

Comment 9

[Seungbeom Kim]

Attachment: screenshots of the reference URL from MSIE 6.0, with two different document character encodings

Comment 10

[rbs]

Attachment: testcase with: input { font-size: inherit; !important }

> BTW, in the patch, you forgot to remove NS_RELEASE() corresponding to
> |nsIFontmetrics *fontMet| you replaced by |nsCOMPtr ....|. 

Good catch, fixed in my local tree.

Debugging showed that form controls are given a smaller font-size (so that they
don't look too big on aggregate with their boxes), thus not everything is
equal. There will be differences. The attached testcase shows that things can
made identical by forcing: input {font-size: inherit; !important}.

Comment 11

[rbs]

...form inputs are given a different UI font-family too (-moz-field which is
ultimately resolved as "MS Shell Dlg" on my Win2K box). In general, in the case
where the UI font doesn't have CJK characters, a chance is given to gfx to look
up something else. And that may coincide with the CJK font used for the web page
if the user didn't set their own fonts on the containing element (i.e., allowing
gfx to apply the same search algorithm/order for both). 

Summary: there are going to be differences whether in font-family or font-size.
They are differences in the EN locale too. It is just that the default settings
are tuned to look good there.

Comment 12

[rbs]

> also assigning to rbs (why don't you take it?

I was scared of the debugging and didn't want to stop somebody else from taking
a stab :-) [see? It has taken 3h non-stop to come up with the conclusions above...]

Comment 13

[Jungshik Shin]

(In reply to comment #11)
> ...form inputs are given a different UI font-family too (-moz-field which is
> ultimately resolved as "MS Shell Dlg" on my Win2K box). In general, in the 

Thanks for the investigation. The question, then, is why '-moz-field' is always
resolved to a single font (that depends on the locale) regardless of 'lang'.
Users expect specifying 'lang' to influence the font selection in the form
control as well as in other parts. This has some implication for bug 91190 as
well, I guess. BTW, where does that mapping/resolution occur? A quick lxr search
didn't help me locate it.

Comment 14

[Boris Zbarsky [:bzbarsky]]

-moz-field is what is called a "system" font in CSS parlance.  That is, it
should be the font that's used for textfields by the OS.  At least that's the
theory.

Perhaps we shouldn't be using it for textfields, then?

See nsIDeviceContext::GetSystemFont (the id in this case is eSystemFont_Field)
and its various implementations.

Comment 15

[Jungshik Shin]

Attachment: screenshot of attachment 153399

In the screenshot, the font selection in non-form-control is 'controlled' by
'lang' while the font selection in form-control (button and 'input') is not
affected by 'lang'.  Regardless of 'lang', the locale-dependent resolved value
of '-moz-field' is used for buttons and 'input's. The shot was taken under
Korean locale on Linux so that the font used was Korean.

It appears that the resolution of '-moz-field' (and probably other '-moz-*') is
done 'globally' once (in the early stage of Mozilla process) and cached so that
'locally-specified' 'lang' doesn't have any effect on the resolution of
'-moz-field'.

Comment 16

[Boris Zbarsky [:bzbarsky]]

Jungshik, if you look at the api I pointed out in comment 14, you will see that
lang _can't_ possibly affect what -moz-field resolves to.

And again, the definition is that it resolves to whatever font the OS uses in
OS-native text controls.  Which is naturally dependent only on properties of the
OS....

Comment 17

[Jungshik Shin]

(In reply to comment #15)
Boris, thanks for the pointer to eSystemFont_Field.
 
> It appears that the resolution of '-moz-field' (and probably other '-moz-*') is
> done 'globally' once (in the early stage of Mozilla process) and cached so that
> 'locally-specified' 'lang' doesn't have any effect on the resolution of
> '-moz-field'.  

I hadn't read Boris' comment when I wrote the above. I also recall that
'-moz-fixed' is affected by 'lang' (in plain text mail rendering for which
'-moz-fixed' is used by default, the font used changes depending on the MIME
charset which we use to set 'lang'. the same is the case with 'textarea' which
also uses '-moz-fixed' as far as I can tell). Not all '-moz-*'s are equal...

In case of 'textfield', using '-moz-fixed' might (or might not) be a quick
'solution'(??)  What about buttons? Now we're entering a grey zone. We want
Mozilla to blend naturally with the rest of the desktop so that we use 'system
fonts' for 'UI' elements. With XUL and what 'WHATWG' is up to, the distinction
between what is 'UI' and what is not becomes murky...

Comment 18

[Jungshik Shin]

Comment on attachment 153495 [details]
screenshot of attachment 153399 [details]

this shot was taken with rbs' patch applied.

Now I'm confused (now I'm on a different computer with a stock mozilla 1.7
build) because the font selection for buttons and textfields in attachment
153399 [details] IS affected by 'lang'. On the other hand, the rendering in 'Edit
attachment' (that is to the right of textarea in which I'm writing this
comment)  is the same as attachment 153495 [details]...

Hmm, this needs a more careful look.

Comment 19

[rbs]

'-moz-fixed' is affected by lang. It comes from the Font prefs, which are
lang-aware. Whereas -moz-field comes from the OS locale. It is all clear, indeed.

> textfields in attachment 153399 [details] IS affected by 'lang'

That's also possible. If the font that is ultimately resolved for -moz-field
doesn't have the CJK glyphs (e.g., my "MS Shell Dlg" doesn't have them), then
gfx kicks its FindFont(), which as we know is lang-aware. So the font that is
finally used can be affected by the lang. Again, it is all clear to me.

But the other important point I mentioned is the font-size. Even the same fonts
won't look the same due to the difference of font-size between the textfield and
the non-textfield. Attachment 153489 [details] shows how to force identical sizes.

Comment 20

[Jungshik Shin]

Attachment: screenshot of attachment 153399 on Windows without the patch applied

Comment 21

[Jungshik Shin]

Attachment: screenshot of attachment 153399 on Linux(Xft) without the patch applied

Comment 22

[Jungshik Shin]

(In reply to comment #19)

> > textfields in attachment 153399 [details] IS affected by 'lang'
> 
> That's also possible. If the font that is ultimately resolved for -moz-field
> doesn't have the CJK glyphs (e.g., my "MS Shell Dlg" doesn't have them), then
> gfx kicks its FindFont(), which as we know is lang-aware. So the font that is
> finally used can be affected by the lang. Again, it is all clear to me.

 This doesn't explain why in attachment 153620 [details] 'LANG' (ja and zh-TW) in the
textfield and button is rendered with the same Korean font (used for KO line)
while the following Chinese characters are rendered with lang-dependent fonts.
The Korean font (that comes from the OS-wide-locale-dependent setting) does
cover all those Chinese characters so that FindFont shouldn't kick in. The fact
that 'LANG' is rendered with the OS-wide-font is explained by your explanation,
but I wonder why Chinese characters are rendered with lang-dependent fonts when
they're all covered by the OS-wide-font.

 Even more interesting is attachment 153621 [details](taken under ko_KR.UTF-8). For 'EN',
your explanation makes sense, but for zh-TW and ja it doesn't because
lang-dependent fonts are used for both 'LANG' and Chinese characters even though
the Korean font (the OS-wide-font) can cover them all. 

As for the size, I like the current behavior better(in textfields and buttons,
it should be a bit smaller....)

Comment 23

[rbs]

> As for the size, I like the current behavior better(in textfields and buttons,
> it should be a bit smaller....)

Sure, it is good as it is.

>but I wonder why Chinese characters are rendered with lang-dependent fonts when
>they're all covered by the OS-wide-font.

Which OS font is used in your case? I saw FindFont() do its font-switching
gymnastics on the string "LANG CJK-chars" with MS Shell Dlg. In fact, what I see
is simply equivalent to what one will see on a page with:

<span lang="ko">
  LANG CJK-chars
  <span style="font-family: MS Shell Dlg; font-size:90%">
    LANG CJK-chars
  </span>
</span>

Thus, the "font-family: MS Shell Dlg" alters the search. That's all. One could
get the same search order inside and outside with:

<span lang="ko" style="font-family: MS Shell Dlg">
  LANG CJK-chars
  <span style="font-family: MS Shell Dlg; font-size:90%">
    LANG CJK-chars
  </span>
</span>

[Note: don't give much importance to my patch, as the metrics comes from the
first ASCII font found anyway.]

Comment 24

[rbs]

>Which OS font is used in your case?

An easy way to check this is to use the DOM inspector:
- inspect a URL (attachment 153399 [details])
- click on the "inspect node" button (on the left of the binoculars button)
- then click in the textfield of interest to jump straight there
- then select "Object - Computed Style" instead of the default "DOM Node".

Comment 25

[Jungshik Shin]

Sorry for the confusion and thanks for the tip (I completely forgot about the
DOM inspector and was too lazy to have a debug session). In case of Xft, 'the
OS-wide' font (-moz-field got resolved to) was 'Sans', which is generic so that
it triggerd 'lang'-dependent font search in Gfx (and probably fontconfig
aliasing rule played a role, too). 

In case of Windows, it's a bit more complicated (the OS-wide font was 'MS
ShellDlg', the same as yours). A few factors  got into play in concert to get me
confused. They include 'font-linking' (not by Gfx:Win but seemingly by Win32
DrawString-like functions) that ties 'MS Shell Dlg' to 'Gulim' (which happened
to be my Korean font for Mozilla as well when I took attachment 153620 [details]), the
fact that Korean Windows core fonts share glyphs for Chinese characters
(although they have distinct glyphs for Korean characters) and that I didn't
include any Korean character in my test case. I still need to tinker more to
have a complete grasp of it (there are some aspects I don't understand).

Anyway, we have to think about what to do here (comment #17). To what extent do
we have to take into account 'lang' in selecting fonts? Do we have to give the
precedence to 'lang' even in 'UI-elements' (button, textfield, etc)? Depending
on our decision here, we may have to close this bug as 'WONT FIX'. 

Comment 26

[rbs]

It turns out that "MS Shell Dlg" is not really a font. It does the linking magic
that you mentioned:
http://support.microsoft.com/?kbid=282187
Hence, the exact appearance is bound to be not entirely predictable on non-EN
locales.

----------------------
As I was doing my earlier cleanup, I came to the opinion that _all_ font APIs in
direct contact with Gfx should really have the lang once for all. Currently, we
have, among others in nsI[Device|Pres]Context:

  NS_IMETHOD  GetMetricsFor(const nsFont& aFont, nsIAtom* aLangGroup,
                            nsIFontMetrics*& aMetrics) = 0;
  NS_IMETHOD  GetMetricsFor(const nsFont& aFont, nsIFontMetrics*& aMetrics) = 0;

The second one should just go away given that it is taking too much time to
continually debug these issues. On the very specific cases where the lang is not
important, users could (explicitly) pass aLangGroup as null.

Thus, I would think GetSystemFont() can be changed to have the lang, which can
be null for users who don't care. In other words, there can be a nice interface
(now), even if implemented later (or never).

Comment 27

[rbs]

jshin/bz, care to r/sr my patch, then? That will save me from facing potential
conflicts. The patch doesn't change the current status quo, but does a nice
cleanup. I might also remove the now redundant |aPresContext| as argument of
nsFormControlHelper::GetFrameFontFM() before check in.

Comment 28

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 153391 [details] [diff] [review]
patch to sanitize font stuff

sr=bzbarsky

Comment 29

[Jungshik Shin]

Comment on attachment 153391 [details] [diff] [review]
patch to sanitize font stuff

r=jshin

Comment 30

[Uri Bernstein]

*** Bug 167759 has been marked as a duplicate of this bug. ***

Comment 31

[Uri Bernstein]

(In reply to comment #26)

> As I was doing my earlier cleanup, I came to the opinion that _all_ font APIs in
> direct contact with Gfx should really have the lang once for all. Currently, we
> have, among others in nsI[Device|Pres]Context:
> 
>   NS_IMETHOD  GetMetricsFor(const nsFont& aFont, nsIAtom* aLangGroup,
>                             nsIFontMetrics*& aMetrics) = 0;
>   NS_IMETHOD  GetMetricsFor(const nsFont& aFont, nsIFontMetrics*& aMetrics) = 0;
> 
> The second one should just go away given that it is taking too much time to
> continually debug these issues. On the very specific cases where the lang is not
> important, users could (explicitly) pass aLangGroup as null.

See bug 100762 comment 7, which basically says the same thing (some 3 years
earlier :-)

Comment 32

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:emilio, could you have a look please?
For more information, please visit auto_nag documentation.