401803 - Russian characters (textfields, bookmarks etc) shown as "?????"

Status: RESOLVED WONTFIX

(Core Graveyard :: GFX: Mac, defect)

Description

[Roland Anderson]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8

bookmarks written in Russian changed to "?????" after updating OS to 10.5. I am not able either to write or paste Russian text into the "properties" field. Even in that case the Cyrillic text is changed to question marks. This was no problem when using OS 10.4.8

Reproducible: Always

Steps to Reproduce:
1. tried to write in the Properties field
2. tried to paste in the Properties field
3.
Actual Results:  
Just try to add bookmarks in Russian and it happens

Expected Results:  
The question marks replace the Cyrillic text

The Russian text should have been in Russian

Comment 1

[Jesse Ruderman]

Based on "I am not able either to write or paste Russian text into the 'properties' field", I'm guessing your bookmarks data is intact, and Firefox is just failing to display it in the correct font.

Comment 2

[Alexey Melchakov]

Yes, this is a font problem.

Problem can be fixed in two ways:

1. You may change default system language to russian in Leopard
2. You may install localized ru_RU build of firefox.

But thats really annoying, many people who use english ineterfaces now are forced to switch to russian one only due to this bug in firefox.

Comment 3

[Alexey Melchakov]

Displaying of russian is broken on most XUL interface textfields and also on some sites. You cant see russian in searchbox on google.com, for example.

Comment 4

[Eugene Piven]

Just to confirm - cyrillic text is properly passed on to the server, for example, at google.com, just like Jesse Ruderman noted, however locally it is displayed incorrectly.
Same problem exists in other XUL-based applications, like Thunderbird - message subject is displayed as question marks, although in the preview field it is displayed correctly.

Comment 5

[Roland Anderson]

Everything worked alright before I updated the OS to 10.5! I had a mixture of English, Russian and Swedish bookmarks. The Swedish ones are still OK, but the Russian ones are messed up.

Comment 6

[Eugene Piven]

Another note - this problem does not persist in the most recent Firefox 3 build (i.e. characters on tabs and in bookmarks display correctly).

Comment 7

[Dirtblade]

Experiencing the exact same problem. Followed up recommended solution and replaced Firefox english version to russian and the issue gone away. Although, I'm not in favor of using russian localized Firefox version and hoping that the bug will be confirmed and fixed.

Comment 8

[Alexey Melchakov]

Bug is still unconfirmed, does anyone kno how to attract attention of firefox core developers to this page?? People, please vote for this bug, i hope this matters )

Comment 9

[Dirtblade]

There is a new 2.0.0.9 version was released. Anyone tried?

Comment 10

[Dirtblade]

Just tried 2.0.0.9 -- same result ;(

Comment 11

[Roland Anderson]

I updated Firefox to 2.0.0.9, but the problem remains. I tend to think that it's something with Apple that's the core of the problem. But there should be some kind of international standard that stops this kind of problem.

Comment 12

[Alexey Melchakov]

2.0.0.9 is a bugfix release to 2.0.0.8 http://developer.mozilla.org/devnews/index.php/2007/10/22/firefox-2008-update-to-be-updated/

I guess they was not fixing anything else in this build.

While this bug is unconfirmed we may not hope for any fix even in 2.0.0.10, please vote!

Comment 13

[Alexey Melchakov]

(In reply to comment #2)

> Problem can be fixed in two ways:
> 
> 2. You may install localized ru_RU build of firefox.
> 
> But thats really annoying, many people who use english ineterfaces now are
> forced to switch to russian one only due to this bug in firefox.
> 

Another hint:

You can use ru-RU firefox with usual english interface by installing locale pack for en-GB ( http://releases.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.9/mac/xpi/en-GB.xpi ), navigating to about:config and changing general.useragent.locale property from ru to en-GB.

Thanks Alexander Slovesnik for this solution.

Comment 14

[Dirtblade]

That worked!!! Great, thank you!

Comment 15

[sashk]

Here is another hint, which worked for me.
Download en-US version of Firefox. Download ru version of Firefox.
Install ru version of Firefox to /Applications, then select show Package contents. Open image of en-US version, and select show Package contents. 
Navigate to Contents/MacOS/chrome in en-US package, and copy following files into the same location of ru version: en-US.jar and en-US.manifest. 
Change general.useragent.locale property from ru to en-US and restart Firefox.
You've got no question marks in the Bookmarks and in text fields...

Comment 16

[Alexey Melchakov]

Macoshints.com has <a href="http://www.macosxhints.com/article.php?story=20071107200650912">another solution</a>. And this bug stays unconfirmed? 

Comment 17

[Stefan [:stefanh]]

Confirming based on comments. Someone need to find a better component for this if it affects all xul-based apps. Does not seem to happen on trunk, so switching to the branch version.

So, this happens only if you have choosed a non-russian language in OS X? Based on the macosxhints article ("rename en.lproj to ru.lproj) it looks like Leopard need to be fooled to think that the localization is russian...

Comment 18

[Stefan [:stefanh]]

Note also that russian is new for Leopard - afaik Tiger didn't included a russian localization.

Comment 19

[Alexey Melchakov]

People on Russian Apple Community say russian localization was available from 10.4.6, leopard just improved localization in some ways (new keyboard layouts, spellchecker for russian language, maybe something else)

Comment 20

[Stefan [:stefanh]]

Switching component, adjusting summary a bit

Comment 21

[John M Fuller]

I have the opposite case where I downloaded Firefox beta 3.0.1b and, after installing Leopard, had the browser start up as if I had been told to begin displaying all its menus and language items in Russian! I have tried starting it from a different volume, fixing preferences, everything, and it still chooses to act as if the entire computer cum user(me) were speaking Russian! 

Help! How do I stop THIS insanity???

Comment 22

[Ben Elkin]

I have filed a Thunderbird bug 405624, which is related to this one.

No problems with Camino.

Comment 23

[Ben Elkin]

Stefan wrote:
>Confirming based on comments.

To those without any knowledge in Russian, how to reproduce the problem:

1. Open a new tab
2. Go to some Russian web site, e.g. <http://www.yandex.ru/>.
The tab name is displayed as "??????",  the window name is "Яндекс"
3. Save the page as bookmark - the bookmark name gets "??????"

Comment 27

[Stefan [:stefanh]]

See also bug 402671 and bug 404197.

Comment 28

[Alexey Kuznetsov]

Attachment: example of russian gmail account in firefox

Comment 29

[Samuel Sidler (old account; do not CC)]

I can reproduce this using Firefox 2.0.0.11 and the steps in comment 23.

This seems pretty critical for 10.5-compatibility and it seems suspicious that we're showing characters wrong in one place and right in another.

Requesting blocking1.8.1.12 for this branch-only bug and CCing a few Firefox people (though this happens in Thunderbird as well).

Comment 30

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Same applies fur Russian and Ukrainian version of Mozilla-Europe:

http://www.mozilla-europe.org/ru/
http://www.mozilla-europe.org/uk/

There are not only affected buttons and bookmarks. Also normal text is shown as question marks.

Comment 31

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I forgot to mention that also Tiger is affected. Updating summary accordingly.

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12pre) Gecko/20071216 BonEcho/2.0.0.12pre ID:2007121605

Comment 32

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This bug was fixed on trunk between the builds 2006-11-20-07-trunk and 2006-11-21-07-trunk.

As far as I can see two bugs were checked-in at this time which belongs to fonts: bug 360244 and bug 352174. CC'ing the assignees.

Is there a chance to get this fixed on branch 1.8.1?

Comment 33

[Stuart Parmenter]

No -- we've completely redone text rendering in 1.9.  No way can we really backport it.  We don't normally keep bugs around for things like this in old releases...  I suggest using Firefox 3 betas.

Comment 34

[Samuel Sidler (old account; do not CC)]

(In reply to comment #33)
> backport it.  We don't normally keep bugs around for things like this in old
> releases... 

... Unless it's part of us supporting a new operating system, which is what we're doing. Sometimes we fix these bugs to support the OS.

Comment 35

[rob sullivan]

when i Get Info on Firefox.app and go to Languages i see only English listed and checked. Safari has about 18 languages loaded and checked. i want to add Russian but don't know where to browse to add it. anyone know??? thanks!

Comment 36

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That has nothing to do with this bug. Please use our support forums to ask these questions in future.

Firefox is delivered in a specific language. If you want another language you have to install the Russian build or use a language pack. For the latest 2.0.0.11 release you can find it here:

ftp://ftp.mozilla.org/pub/firefox/releases/2.0.0.11/win32/xpi

It's easier to handle different languages when you install the Locale Switcher: http://benjamin.smedbergs.us/switch-locales/

Comment 38

[Matthew (lilmatt) Willis]

FlockBug 11539 is related to this.

https://bugzilla.flock.com/show_bug.cgi?id=11539

Comment 39

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #29)
> I can reproduce this using Firefox 2.0.0.11 and the steps in comment 23.
> 
> This seems pretty critical for 10.5-compatibility and it seems suspicious that
> we're showing characters wrong in one place and right in another.

It's not suspicious at all; the window title is being rendered by the OS but everything else is being done by Gecko.

We actually do see this in Camino in the language-switching <select> at the bottom of http://www.mozilla-europe.org/ru/.  

After noticing that, I played around with that <select> a little bit in DOMi.  AFAICT, it's being given a font-family of -moz-list (which is supposed to map back to Lucida Grande, and Computed Style says it does); when I replace that "-moz-list" with an explicit "Lucida Grande", ta da, I get Cyrillic glyphs in "Times CY".

This suggests that somewhere font-fallback or font mapping is screwed up (which also syncs with comment 32).  This comes as no surprise, given the number of weird 10.5 vs 10.4 font bugs jdaggett's fixed on the trunk and the number of weird font bugs that have existed on the branch, including some where non-ASCII glyphs would render incorrectly when "mixing" OS locales and app localizations.

Since there are (admittedly ugly) work-arounds available, I think the best course of action here is for people to either use the work-around or to do as Stuart suggests in comment 33 and use trunk builds/Firefox 3 betas.  (If there's another Firefox 2.0.0.x release before Firefox 3, MoCo might want to consider choosing one of comment 2, comment 13, or comment 17 as the "blessed" work-around and relnote it).

Comment 40

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As I said before. This is not only 10.5 related and also exists on Tiger.

Comment 41

[Stefan [:stefanh]]

(In reply to comment #40)
> As I said before. This is not only 10.5 related and also exists on Tiger.
> 

Are you sure? By looking at comment #0, comment #5 and the duplicates; bug 404206, bug 401441 and bug 405624 it looks like this started to happen after people upgraded to 10.5.

Comment 42

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Screenshot under OS X 10.4.11

This screen shot shows the behavior of Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12pre) Gecko/20080130 BonEcho/2.0.0.12pre ID:2008013003 under OS X 10.4.11 with latest patches. Using a trunk build all characters are correctly shown.

Comment 43

[Stefan [:stefanh]]

So, this wasn't an issue on 10.4.8, but it is on 10.4.11.

Comment 44

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I did a search and found some really useful information which shows that we have a  problem within our builds. That doesn't seem to be related to a specific OS X version.

http://www.macosxhints.com/article.php?story=20071107200650912

After changing en.lproj to ru.lproj within Resources of the package contents and restarting the Firefox it is working fine. Do we have a failure in packaging?

Comment 45

[Stefan [:stefanh]]

(In reply to comment #44)
> I did a search and found some really useful information which shows that we
> have a  problem within our builds. That doesn't seem to be related to a
> specific OS X version.
> 
> http://www.macosxhints.com/article.php?story=20071107200650912
> 
> After changing en.lproj to ru.lproj within Resources of the package contents
> and restarting the Firefox it is working fine. Do we have a failure in
> packaging?
> 

See comment #16. I don't think it's a failure in packaging. On branch, afaik we've always done like this. afaik the xx.lproj is a way to tell the OS what locale the app should run in (if you re-name en.lproj to ru.lproj, you'll notice that the majority of the menuitem labels in the Firefox menu is in russian - that menu is drawn by the OS). So, it looks more to me that Apple have changed something.

Comment 46

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

If you got a late build of 10.4 with a new Mac, you could have ended up with OS-wide Russian resources (I forget which late version of 10.4 added them; our Russian translator told me at one point); otherwise, you got Russian when you upgraded to 10.5.

At whatever point you got Russian OS resources (either a new Mac with a late 10.4 build, or upgrading to 10.5), the presence of these resources started triggering a bug related to font-fallback/font selection, probably in the old Mac gfx code's ancient crufty font code.

A Camino user who reported this problem (with html content, like the <select> on http://www.mozilla-europe.org/ru/) confirmed to me today that adding something like 

select {
font-family: 'Lucida Grande' !important;
}

in your userContent.css can also fix this. Obviously you'd have to repeat this for any other html form controls that are experiencing this problem and do something similar with userChrome.css to fix XUL application UI, but this might be a less-invasive change than hacking your app's .lproj folder or using a Russian build, etc.

Comment 48

[tobiwei]

The problem still occurs with Firefox 2.0.0.14 on Mac OSX 10.5

Fonts on google.ru are not shown correctly. Firefox is not able to display some russian strings correctly.

Comment 49

[Samuel Sidler (old account; do not CC)]

We won't be fixing this in Firefox 2.0.0.x. WONTFIX.