5402 - [PP]Mac menus DON'T display Unicode correctly

Status: VERIFIED FIXED

(SeaMonkey :: UI Design, defect, P2)

Description

[Fergus Sullivan]

Take navigator.xul and change a couple of strings to Japanese or to European
extended characters (using Unicode).  On Win, all on-screen widgets look good,
on Mac, the menus and any HTML buttons or Menu popups are corrupted.

It looks like MacFE still expects either Shift-JIS or MacRoman, as appropriate.
It should be looking for and displaying Unicode characters.

Logging as XP Misc, although maybe it should be XUL.

Comment 1

[Fergus Sullivan]

Changing component to XPApps.

Comment 2

[Fergus Sullivan]

And assigning to Peter Trudelle.  CC-ing L10n & I18n people.  Changing to P2
also.  Maybe it should be higher.

Comment 3

[Fergus Sullivan]

Setting TFV to M5.  You'd think I'd make all these changes in one go, wouldn't
you?

Comment 4

[Peter Trudelle]

Clearing target milestone and resetting priority to p3. These fields are owned by
engineering, please do not touch them. Also, this should have been filed as three
separate bugs, one for each widget, not lumping everything together.  Could you
split it up?

Comment 5

[Peter Trudelle]

also, the severity of this bug is not critical, which is defined as "crashes,
loss of data, severe memory leak". Reassigning to fergus to split up and refile
with appropriate severities.

Comment 6

[Fergus Sullivan]

Okay.  Changing to severity to major.  And calling this one the Menu bug.
Others will follow for popup html menus and for html buttons.

Also assigning back to trudelle as split is done.

Comment 7

[msanz]

Peter, Fergus is also an engineer. Priority is high because we cannot pseudo
localize the product if this does not work correctly, and we have a pseudo
localization task for M5 (inherited from M4) that is blocked by this now.
Please, reset this bug, 5438 and 5439 accordingly

Comment 8

[Peter Trudelle]

Okay, but we may not get to these for M5.  Chofmann has dictated that we reduce
our individual bug lists to the top 2 bugs as of next Tuesday.  In general,
bugs that affect everyone will be triaged with higher priority than those that
just affect I18N.
reassigning to saari as p2 for m5

Comment 9

[Peter Trudelle]

Also, when I said 'engineering, I meant only the development team responsible
for fixing the bug. QA professionals are also engineers, but they own the
severities, not the priorities.

Comment 10

[saari (gone)]

So how do I test this, being that I don't run an internationalized OS version?

Comment 11

[Fergus Sullivan]

Saari, try using generic extended characters.  é and such like.  Be aware,
though, that they need to be entered as UTF8 into the navigator.xul file.

Here's how to produce a UTF8 character on any Mac.  Open a new Doc in composer.
Select View>Character Set>UTF-8

Type some extended characters--try using option-a for a 'å'.  Save HTML.  copy
the strings concerned and their encoding into navigator.xul and watch behaviour.

Note that many widgets work.  Menus is one that doesn't.

Comment 12

[msanz]

saari, you can find the Japanese XUL file at the URL mentioned above.

Comment 13

[leger]

Adding to QA Blocker radar.

Comment 14

[chris hofmann]

and we get fergus over to saari's cube to help on this one?

Comment 15

[saari (gone)]

I can definitely use some help. Everything I've tried so far hasn't had any
affect.

Comment 16

[Fergus Sullivan]

On my way to talk to Saari now.

Comment 17

[saari (gone)]

So if anyone has insight on how to fix this, I'd love to hear it...
SetMenuItemTextEncoding is new with Appearance, and takes a script ID, which I
don't have at runtime anyway.

What is the right magic to get the menu manager to do this correctly?

Comment 18

[tague]

Chris :- If you point me at some code, I can take a look at this and see if we
can get things up and running correctly.

Comment 19

[tague]

Update: i have this partially working - there is still a problem with
hierarchical menus and Unicode.  More later.

Comment 20

[saari (gone)]

Anything I can help with? I can probably figure it out if I see the code that
does the right thing. In other words, is this checked in yet?

Comment 21

[tague]

Nothing checked in yet.  I fixed the problem with hierarchicals, but I'm having

problems with Japanese in the file menu at the moment.  I'm going to send mail to

some ex-colleagues at Apple to get some more information.



There may need to be some re-thinking of the mac implementation of the menu code

- we can talk about that later, after I get information back from Apple.

I'll let you know if there is anything I need help with.

Comment 22

[saari (gone)]

Ok, well one change I may check in is setting the menu item text with
SetMenuItemText(). That may help you.

Comment 23

[saari (gone)]

Also, Mac menus arn't dynamic yet, so there will be more substantial changes to
them in the future.

Comment 24

[chris hofmann]

is there enough work done to remove the block status?  doing that now.
if this is wrong remark it blocker..

Comment 25

[tague]

I don't think this can be considered a QA blocker - I don't think it is going to
be technically possible to use Unicode in Mac Menu drawing.

I talked to some ex-coworkers at Apple about ATSUI, the new Menu Manager and
Unicode Menus.  According to them, the Menu Manager doesn't use ATSUI and
doesn't support Unicode drawing.  The support we will be able to do for Unicode
menus on the Mac is going to be limited.

The code needs to convert the Unicode data into the encoding of the system
Script; however, we won't be able to have Japanese menu's on a US system.  The
Unicode data will have to be limited to characters that can be displayed by the
system script.

Comment 26

[saari (gone)]

moving to M7

Comment 27

[Erik van der Poel]

Montse, when do you need to have this fixed?

Saari, do you have enough info now to fix it? I.e. convert from Unicode to
system script, and then pass to menu. If not, can you get the info from Tague,
please?

Comment 28

[saari (gone)]

I know we have calls to do this on Windows, how about Mac? Or should I be using
the OS stuff (is there a Unicode to Shift JIS converter in the OS now?)

Comment 29

[Erik van der Poel]

Tague and Frank, please comment on whether we should use Mozilla's converters
or Mac converters (from Unicode to system script).

Comment 30

[tague]

I think in cases like this, it makes much more sense for us to use the platform
encoding converters.  Both Frank and I are using the platform encoding
converters in platform specific parts of the code (Frank for ATSUI drawing, me
for key/IME input into Unicode strings).

For purely platform-specific parts of the code, it is much easier to use the
platform encoding converters (if they exist), because they have knowledge about
the kind of conversions that need to be done at that level (system encoding to
Unicode).  That knowledge isn't really built into the Mozilla encoding
converters.

Comment 31

[Erik van der Poel]

OK, Saari, please use the Mac converter to convert from Unicode to the system
script. If you don't know how to do this, please contact tague or ftang.

Comment 32

[saari (gone)]

Ok, what API should I be looking at? (sorry, not up to speed on TSM)

Comment 33

[tague]

Attachment: bug fix patch

Comment 34

[tague]

I have a patch for this which I sent to various Mac people to look at and
approve.  I will check this in, once I get approval.

Comment 35

[tague]

checked in a fix today, 6/2/98 - 5:30pm.

Note: we are only going to support drawing the menu bar (i.e. the menu label) in
the system script.  This means that you will only be able to display a Japanese
menu bar on a Japanese system.  The actual menu text will be multi-lingual.

Comment 36

[bobj]

Temporarily reopened, so I can reassign to Tague.
I'm doing my anal-rententive management thing and trying to track what bugs
my group is fixing. :-}

Comment 37

[Fergus Sullivan]

Verifyied against June 18 build.

Note that difference between native widgets and macos means that proper display
is only possible under KanjiTalk.  JLK isn't enough for once.

Comment 38

[bobj]

What doesn't work on JLK.  Just the menu bar or also the menu entries?
We know the menu bar is not multilingual, but do multilingual menu entries
display on JLK?

Comment 39

[Fergus Sullivan]

Just the menu bar.  If we go down the route of registering it as JA application
(using language register hooks) then this would be rectified.

Comment 40

[tague]

the menu items are "mono"-multilingual.  any menu item can be displayed in any
macintosh script, but you can't mix two scripts in the same menu item.  you can
have Japanese, German, Indic, Chinese menu items in the same XUL file, and they
will be displayed fine, but you couldn't do something like Save as... where
"save" was in Japanese and "as" was in chinese.

The actual menu bar is special.  It will only draw in the system script, so you
can only support Japanese or Chinese in the menu bar.  This won't work with just
a language kit, unless as fergus mentions, the user uses the lanugage kit
registration application and registers the application as a japanese or a
chinese application.  whatever the application is registered as determines what
script can be displayed in the menu bar.

the only way around this limitation is to write a custom MDEF which does Unicode
drawing.  that would be incredibly painful and too time consuming.

Comment 41

[bobj]

It's good to clarify what is implemented.  Thanks.

The current implementation supports 99.99% of the cases that matter.
The menu bar will normally be mono-lingual for the current localization.
When the Mac app is localized, we should set the language setting appropriately.
And it will be very rare that a single entry (e.g. What's Related, document
title, bookmark) will contain characters from more than one script.

I think there is no reason to do a custom MDEF.