Closed Bug 2870 Opened 22 years ago Closed 20 years ago

@charset is not supported


(Core :: Layout, defect, P2, major)






(Reporter: momoi, Assigned: attinasi)




(Keywords: css2, Whiteboard: (py8ieh:reexamine tests))

(This bug imported from BugSplat, Netscape's internal bugsystem.  It
was known there as bug #105368
Imported into Bugzilla on 02/03/99 17:11)

** Observed with 4.05 Windows builds **
** This was originally reported by Dreamweaver-J international engineer
   at Macromedia **

In Style Sheets, some HTML editors (e.g. Dreamweaver-J) lets you insert
font names in Japanese. Apparently we don't support font names in anything
other than ASCII characters.
This is a serious problem for Dynamic HTML pages using Japanese fonts.
(In comparison, it is reported that IE4 can handle high-bit and multi-byte
font names.)
Per DP's request, here's a set of tests provided by Scott Richards, an
engineer working on Dreamweaver-J HTML editor.

He has more tests than just font & class names. This also includes a
comparison table with IE4.
Fonts on Japanese and other non-Latin systems use non-Latin font names.
Therefore this bug effectively prevents Japanese users from using style sheets.
The unicode JS engine is going in 4.06 This might make this possible. From our
end, david can evaluate the work involved.
JS 1.3 didn't make this problem go away.

Re-tested this problem with the above URL under NT4-Japanese
running Win32 4.06 5/5/98 build.
None of the problems mentioned on the test page were corrected
with the new JS engine. IE4 passed alll tests, however.

We really need to get this working for DHTML pages.
Can intl help us with this. Do we know is a simple test with a <FONT
FACE=japanesfontname> work.
Yes. A simple <FONT FACE="JPN_FontName">Strings_in-Japanese</FONT>
structure produces different font faces in Windows and Mac clients - 4.05 and
reassign djw -> dp.   thinking we need to move this to 5.0.
moving this to 5.0.   let me know if this is wrong.
There are lots of i18n test cases at the URL above, and none of them work:

  Double Byte Image Name
  Double Byte Frame Names
  Double Byte Layer Names
  Double Byte Class Names (CSS)
  Japanese Fonts

Frank, can you take a look at these and/or reassign as appropriate?  Thanks.
Assignee: ftang → peterl
Component: Internationalization → Layout
The font name inside <FONT FACE> tag should work on Gecko 5.0 Window version
now. I just fix that bug last week. I am not sure about Mac, probably still a
bug there. UNIX and PostScript do not use nonASCII font name at all so it should
not be a problem.

As for Style Sheet, we have to divide it into two cases, the case the style
sheet is embedded in HTML and the cases the style sheet is in external file.

Here is how the CSS 1 said about this "Appendix B: CSS1 grammar" (see )


                   : unary_operator?
                     [ NUMBER | STRING | PERCENTAGE | LENGTH | EMS | EXS
                     | IDENT | hexcolor | URL | RGB ]
 the only possible toke which make sense here for font name token is STRING

2. and STRING is {string}
{string}                        {BEGIN(0); return STRING;}

3. and
string          \"({stringchar}|\')*\"|\'({stringchar}|\")*\'

4. and stringchar      {escape}|{latin1}|[ !#$%&(-~]
which mean string char could be either
   a. ANY char in ASCII except two characters - " and '
   b. 0xa1 - 0zff (but becaureful, it really mean U+00a1 - U+00FF, not any byte
combination, it really mean the LATIN 1 0xa1- 0xff)
   c. escape (see below)
5. escape          {unicode}|\\[ -~¡-ÿ]
mean escape could be two thing
   a. a leading \ char plus ANY ASCII or 0xa1 - 0xff (same as above)
   b. unicode (see below)
6. unicode         \\[0-9a-f]{1,4}
 mean a \ char plus one to four chars in the following set "0123456789abcdef"
(notice even ABCDEF is not in the list)

What does this spec mean, it mean when you want to put font name which have non
ISO-8859-1 characters there, you have to use \045f for to encode it. Therefore,
it is highly possible the "Dreamweaver-J HTML editor" does not apply such
encoding process whenhey have Japanese font name. According to the spec, any
0xa1-0xff byte will be interprete as ISO-8859-1 instead of the chars in *some*
charset. Therefore, I have to said this bug is invalid for CSS 1.

However, for CSS 2, thing are a little different-

CSS 2 allow non ISO-8859-1 non ASCII characters in CSS 2- Read the D.3 section
of CSS2

"CSS1 style sheets could only be in 1-byte-per-character encodings, such as
ASCII and ISO-8859-1. CSS2 has no such limitation. In practice, there was little
difficulty in extrapolating the CSS1 tokenizer, and some UAs have accepted
2-byte encodings. "

So here is the action to fix this-
1. CSS parser should use the charset it specified in the beginning of CSS 2 to
interpret the byte. Currently, cata already hookup To
Unicode converter into nsIUnicharInputStream and we believe the CSS parser are
using nsIUnicharInputStream, but CSS parser should pass the correct charset to
create the nsIUnicharInputStream.
2. We should make sure we handle the *chaged from CSS1 to CSS2" Unicode escape
as mention in D.3. of CSS2

Reassign this to CSS parser folks. after the parser do the right thing, we
should verify weather the Mac GFX do the right thing. But for now, we can use
Window to debug it.

Change the component to Layout since there are no Style Sheet componment.
Reassign this to perterl since he own the new Style system and work on CSS

peterl: do you support "@charset" in CSS2 parser ? as today
(2/5/1999) nsIUnicharInputStream know about "ISO-8859-1" , "ISO-8859-7",
"Shift_JIS" and "windows-1253" as charset, later, we will add 80+ charsets
support. But you can use the charset I mention above to test your code for now.
Closed: 21 years ago
Resolution: --- → WONTFIX
marking resolved won't fix in MozillaClassic codebase.
If someone thinks this bug is still relevant in new seamonkey codebase, please
open a new bug under Browser, or re-open this one and change the product to
Browser. All open MozillaClassic bugs are being closed. Thanks.
OS: Windows NT → All
Priority: P1 → P2
Product: MozillaClassic → Browser
Hardware: PC → All
Resolution: WONTFIX → ---
Target Milestone: M4
reopen and clear the wontfix. and mark the product to browser and set teh Target
to M4 (just make sure it is not M3). Mark the platform to All and os to all.
Priority change to p2. Please reset the Target if M4 is not reasonable. Please
notice my comment on 02/05/99 13:51 . This is reoepn for SeaMonkey/Gecko.
Target Milestone: M4 → M7
Blocks: 7228
Deferring to M10
Pushing off non-beta 1 issues
Reassigning peterl's bugs to myself.
Accepting peterl's bugs that have a Target Milestone
Pushing my M15 bugs to M16
Tao and Ray,
Won't this affect our ability to localize for Japanese?  If so, we need this
soon, right?  Don't we specify the font in files like global.css?
  window {
    background-color: white;
    font: 3mm arial,helvetica,sans-serif;
    padding: 0px;
(BTW, how do we make the *.css files localizable?  Shouldn't things like font
names be externalized?)
Localizable style info shall be in locale/[lang-country]/*.css . As I said, it might worth a while to allocate
 resources to look at the contributed Janpanese translation and log bugs aganist non-conforming UI.
But in my example, will we try to replace arial and helvetica with equivalent
Japanese fonts which on Japanese Win/Mac systems will use Japanese
(non-ASCII) names?  If so, this bug would become a stopper for the M14 JA
Do we have the list of fontnames that are locale-sensitive or platform specific? Who might know? Erik?
On Win and Mac, most fontnames are localized.  So Japanese fonts have Japanese
names and Chinese fonts have Chinese names on localized OS releases.  (On
US releases, the same fonts will have ASCII names).  On Unix systems, fontnames
are ASCII as for any fonts I've seen.
Summary: Font names in Style Sheets cannot support high-bit characters → [BETA]Font names in Style Sheets cannot support high-bit characters
Could we retest this one?
OK. I re-tested this (Thanks, Scott Richards at DreamWeavers for
the test cases!).
** 12/21/99 Win32 M12 build **

For the Style Sheets,

Failed: Double-Byte Class name test
Passed: Double-Byte Font Name test


Failed: Double-Byte Layer Name.
        (But we also failed Single-byte Layer name and so
        probably Layer is not implemented yet?)

Though not related to this,

Partial failure: Doyble-Byte Frame test (The 2nd "click test did not
                   reload the frame correctly with Auto-Detect on.)

Passed: Double-byte image name.

So, the font names are now supported.
Shall we file separate bugs for the other failures on
Double-Byte Class name and Layer name?
Yes, please and mark the dependency to relate them. Thanks!
And close this bug as WORKSFORME?
Good news about the font names!
I'm not sure it can be closed as WorksForMe: '@charset' is not yet implemented
and the comments from ftang on 02/05/99 suggest that it may be required.

I recommend giving ftang a chance to look at the problem and decide whether it is
fixed or if we need to improve the current testcases.
CCing ftang
Due to Beta indication in Summary, putting beta1 into keyword field.
Keywords: beta1
hold down a second. The font name test momoi try is css embedded inside HTML. 
Since the HTML is tag with meta tag, the character convert correctly into 
unicode. However, it is still a problem for stand along css file. Without the 
@charset support, we cannot ship any stylesheet which have localized name in it.

what is the default encoding for CSS2 when the @charset info is missing ?
ftang: It looks like there is no default encoding. The spec at
TR/REC-CSS2/syndata.html#x64 says:
Note that reliance on the @charset construct theoretically poses a problem since 
there is no a priori information on how it is encoded. In practice, however, the 
encodings in wide use on the Internet are either based on ASCII, UTF-16, UCS-4, 
or (rarely) on EBCDIC.

I changed the summary line to "@charset is not supported" and removed the [BETA] 
tag which was added by tao because of the original problem with font names 
(problem that was fixed in December).

If someone thinks that this issue is still necessary for beta1, please tell me: 
Do we have any localized names in stylesheets? Can't we translate these names 
using escape sequences?
Keywords: beta1css2
Summary: [BETA]Font names in Style Sheets cannot support high-bit characters → {css2} @charset is not supported
do you support the CSS2 escape ?
Yes. Be aware that the CSS2 escape sequences don't follow the same rules as HTML. 
The CSS syntax (see requires 1 
to 6 hex digits after the '\'.

What kind of localized names do you have in style sheets? Font names only? Or 
something else?
Summary: {css2} @charset is not supported → @charset is not supported
Whiteboard: (py8ieh:reexamine tests)
See some comments about '@charset' support in bug 28500.

Cowardly reassigning i18n/style issues to attinasi. On my list, they are likely 
to stay buried for a very long time.
Assignee: pierre → attinasi
Blocks: 14744
I have added support for the @charset directive into the CSSLoader and 
CSSParser. I need to test this code with some charset other than "ISO-8859-1" 
but I do not know how yet. I'll be getting some assistance and reviews and 
checking this in shortly... If anybody would like the changes to help me test 
I'd be happy to send them out, just ask (6 files).
rchen, momoi or teruko,
  Can you provide any test cases to attinasi?
cc'd teruko
How about the test page pointed by the URL field of this bug report?
Try the style sheet test one.

You'll need to have Japanese font installed on your system.
Depends on: 35800
change the qa contact to teruko. attinasi- please mark this bug close after you 
check in so teruko can QA the daily build for you. 
QA Contact: momoi → teruko
Fix checked in.
Closed: 21 years ago20 years ago
Resolution: --- → FIXED
I tried to verify this, but I found the bug 37448.  I will not verify this until 
37448 is fixed.
I verified this in 2000042809 Win32.I created the test cases in 
http://babel/tests/browser/css/css2/jFonts-sjis.html, jFonts-eucjp.html, and 
jFonts-sjis-euc.html This works fine in Win32 build.  I will create the 
different test cases for Mac and Linux.
I verified this in 2000-05-16-11 Mac.
You need to log in before you can comment on or make changes to this bug.