Closed Bug 185523 Opened 22 years ago Closed 21 years ago

Mozilla uses Western font settings for UTF-8 instead Unicode settings

Categories

(Core :: Internationalization, defect)

Product:
Core
Component:
Internationalization
Platform:
x86
BeOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: sergei_d, Assigned: sergei_d)

References

Details

Attachments

(1 file)

patch - excluding BeOS from hack
719 bytes, patch
beos
: review+
blizzard
: superreview+
Details | Diff | Splinter Review 
I have different fonts set for Western and Unicode in Font Preferences.
(Fonts for Western are BeOS-native and contain only ISO-8859-1 symbols)
But on some UTF-8 sites like user pages on www.livejornal.com Mozilla uses fonts
from western set, instead Unicode set - inspite autodetected or manually set
encoding is UTF-8. So i cannot, for example, read lot of russian pages there.
Example: http://www.livejournal.com/~maximov
Page uses CSS.
If i disallow other font usages by document in Appearance->Font preference, it
fails/falls back to western instead UTF-8 (<meta content="text/html;
charset=utf-8" http-equiv="Content-Type">) for text described by class

.entrytext {
padding: 2px;
text-align: justify;
font-size: 11px;
font-family: georgia,tahoma;
}


For simple without CSS or font settings all seems OK
This is a huge problem with several UTF-8 encoded Hebrew websites (and the only 
reasons why I still have to boot into Windows occasionally).

A couple of typical examples:
http://www.google.co.il
http://mac.plonter.co.il/plonwiki

See the screenshot here: http://snurl.com/UTF8vsLogicalHebrew
If needed, I can attach the HTML samples when I get back home.

By the way, does this bug depends on Bug 155374 ("nsLocalFile should use UTF-8 
as native charset for BeOS")?

Prog.
google is ok for me, only wrong thing was caption on submit/search button, but
this was fixed by setting plain system font to Hebrew-capable, e.g. Arial for
Windows.
But on plonter forum this is exactly just this bug, to fix it locally you should
set fonts for Western in Mozilla again to Hebrew capable.

About dependency. Yeah, it is possible, that something is wrong in "generic"
Mozilla code for BeOS case, BeZilla is in "Unix" build class, and unices are
well-known for its ancient tightening to poor-poor-poor 8-bit encodning, but
perhaps i should look more carefully into such files as nsFontMetricsBeOS to
determine if some important thing is still unimplemented.
Screenshots for Prognathous:
http://www.fi.tartu.ee/sergei_d/BeZillaHebrew2.jpg - google.co.il
http://www.fi.tartu.ee/sergei_d/BeZillaHebrew.jpg - forum
(after settings mentioned in previous comment)

>But on plonter forum this is exactly just this bug, to fix it locally you should
>set fonts for Western in Mozilla again to Hebrew capable.

Why? Doesn't the Hebrew font settings supposed to cover this?
>>But on plonter forum this is exactly just this bug, to fix it locally you should
>>set fonts for Western in Mozilla again to Hebrew capable.

>Why? Doesn't the Hebrew font settings supposed to cover this?

No, it does not. Forum page is set for charset "UTF-8", so, in theory, it should
be covered by fonts set for "Unicode". But it seems broken. I have feeling that
Mozilla (not just BeZilla) don't care about it, as every port does (unicode)
font mappings in it's own way, and "UTF-8" for simplicity  is somewhere handled
 as equivalent for "western", because it is ASCII-7 compatible.
Seems mozilla/intl/locale/ needs BeOS revisions
Ok, got problem roots.
mozilla/intl/locale/src/nsLanguageAtomService.cpp - 
nsLanguageAtomService::LookupCharSet
for pages with charset==utf-8 gets language group x-unicode.
It is absolutely OK.
But then, it handles x-unicode (ONLY!) case in very special manner - it calls
conversion of mLangGroup to ApplicationLocale lang-group
 if (langGroup.get() == mUnicode.get() /*"x-unicode"*/) {
   res = GetLocaleLanguageGroup(getter_AddRefs(langGroup));
   NS_ENSURE_SUCCESS(res, res);
  }
then GetLocaleLanguageGroup() calls GetApplicationLocale() from LocaleService.cpp
and for case
#if (defined(XP_UNIX) || defined(XP_BEOS)) && !defined(XP_MACOSX)
GetApplicationLocale calls POSIX locale functions.

Bummer. In BeOS, inspite people can set LC values manually, as rule it is "".
it results in fall-back case "us-EN" and "x-western"

I think, inspite i know workaround for UTF-8 (don't handle x-unicode so
"specially", let it stay as is for BeOS), i need advice of some Mozilla i18n
guru - approach, when we rely for unicode encoding on systam locale settings is
very suspicious:)

Personally for Prognathous - try to set/export hebrew locale variable in unix
manner, somewhere in UserSetupEnvironment for example, set non-hebrew fonts for
"Western" and see what will happen - maybe BeOS posix layer will report it for
Mozilla.

But it is BAD (TM) approach, because using multiple language without pain on one
page without pain is what UTF-8 for.

Sure, i should investigate MacOS and Win (UNICODE) approaches, but that's all today


NS_IMETHODIMP
nsLanguageAtomService::LookupCharSet(const PRUnichar* aCharSet,
  nsILanguageAtom** aResult)
*********
#if !defined(XP_BEOS)
  if (langGroup.get() == mUnicode.get()) {
    res = GetLocaleLanguageGroup(getter_AddRefs(langGroup));
    NS_ENSURE_SUCCESS(res, res);
  }
#endif
************
}
solved problem for utf-8 pages (now Mozilla takes user-defined fonts for
"Unicode") - but we still need work both on intl/locale and gfx/src/beos -
problem may be still actual for site-dictated fonts.
setting weak dependency
Depends on: 15374
> to fix it locally you should set fonts for Western in Mozilla again to Hebrew capable.Thanks a lot for this tip Sergei! viewing Hebrew UTF-8 pages is now possible, alas editing them is a different story... see this freely editable Wiki page:http://mac.plonter.co.il/plonwiki/_d7_90_d7_a8_d7_92_d7_96_20_d7_94_d7_97_d7_95_d7_9cAny suggestions?BTW, your workaround also works for another page with a similar problem, www.most.gov.il -  interestingly enough this one isn't encoded with UTF-8, it is defined to use the very common windows-1255 (identical to ISO-8859-8-i).Prog.
> to fix it locally you should set fonts for Western in Mozilla again to Hebrew
capable.

Thanks a lot for this tip Sergei! viewing Hebrew UTF-8 pages is now possible,
alas editing them is a different story... see this freely editable Wiki page:
http://mac.plonter.co.il/plonwiki/_d7_90_d7_a8_d7_92_d7_96_20_d7_94_d7_97_d7_95_d7_9c

Any suggestions?

BTW, your workaround also works for another page with a similar problem,
www.most.gov.il -  interestingly enough this one isn't encoded with UTF-8, it is
defined to use the very common windows-1255 (identical to ISO-8859-8-i).

Prog.

PS. sorry for the spam.
> the very common windows-1255 (identical to ISO-8859-8-i)

No, they are not identicle. windows-1255 is a superset of iso-8859-8, and has
some charachter that iso-8859-8 does not
Oh, did I say identical? I meant *virtually* identical... :-)

Prog.
Confirming for Linux also. The problem seems to be less noticeable because
fallback glyphs are used, but UTF-8 pages are definitely using the fonts set for
Western.
according to smontagu, problem appears also in Linux - it takes western font
settings for utf-8 pages instead Unicode font settings.
I think this issue is common for all unices
OS: BeOS → Linux
OS: Linux → BeOS
Now that I understand this issue, it turns out to be a dupe of bug 91190.

*** This bug has been marked as a duplicate of 91190 ***
Status: NEW → RESOLVED
Closed: 22 years ago
Resolution: --- → DUPLICATE
Mark as verified.
Status: RESOLVED → VERIFIED
BeOS don't need this hack, as it uses UTF-8 natively for widgets, and intl
users always set proper fonts as plain/bold/fixed to be able to work with file
names, menus etc.
Comment on attachment 117639 [details] [diff] [review]
patch - excluding BeOS from hack

review request
Attachment #117639 - Flags: review?(arougthopher)
Comment on attachment 117639 [details] [diff] [review]
patch - excluding BeOS from hack

Looks good to me Sergei.

You have CVS access now, correct?  If so, I'll let you check this in.  So you
know, however, the tree is only openned to driver approved checkins, so you
need to get permission before committing the change.  Plus, this needs
superreview.
Attachment #117639 - Flags: superreview?(ftang)
Attachment #117639 - Flags: review?(arougthopher)
Attachment #117639 - Flags: review+
removing "direct" duplication, reopening.
Maybe some kind of soft dependency is better in this case than duplication.
Status: VERIFIED → REOPENED
Resolution: DUPLICATE → ---
reassigning
Assignee: smontagu → sergei_d
Status: REOPENED → NEW
Comment on attachment 117639 [details] [diff] [review]
patch - excluding BeOS from hack

Transferring sr request: ftang is not a superreviewer
Attachment #117639 - Flags: superreview?(ftang) → superreview?(blizzard)
Comment on attachment 117639 [details] [diff] [review]
patch - excluding BeOS from hack

sr=blizzard
Attachment #117639 - Flags: superreview?(blizzard) → superreview+
Checking in nsLanguageAtomService.cpp;
/cvsroot/mozilla/intl/locale/src/nsLanguageAtomService.cpp,v  <-- 
nsLanguageAtomService.cpp
new revision: 1.20; previous revision: 1.19
done
Status: NEW → RESOLVED
Closed: 22 years ago → 21 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: