10816 - Add Unicode clipboard support to Mac

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P3)

Description

[tague]

add support for Unicode clipboard items to the Mac clipboard code

Comment 1

[Frank Tang]

Here is more info
http://developer.apple.com/techpubs/mac/TextEncodingCMgr/TECRefBook-154.html
Naoki, since tague is pretty doom, is that possible you can also help him to
take care this issue ?

Comment 2

[nhottanscp]

Who owns Mac clipboard code?

Comment 3

[Simon Fraser [no longer active]]

How do you propose to do this? There are no native unicode clipboard types on
Mac.

Comment 4

[nhottanscp]

The document mentioned by ftang says 'utxt' has been added.

Comment 5

[Frank Tang]

Change OS to Mac System 8.5
The URL have been change to 
http://developer.apple.com/techpubs/macos8/TextIntlSvcs/TextEncodingConversionMa
nager/TEC1.5/TEC.b4.html 
File Types

The file type 'utxt' has been registered for UTF-16 plain text documents. The 
(optional) scrap type 'utxt' is
also registered for UTF-16 Clipboard text.

Whether it is useful to register a file type or scrap type for UTF-8 text is 
currently under discussion. As do other
documents and text that use WorldScript encodings, plain UTF-8 documents could 
use the file and scrap type
'TEXT'. UTF-8 is compatible with the assumptions that govern WorldScript 
encodings; these encodings are not
specifically identified in 'TEXT' files and Clipboard contents.

Comment 6

[Frank Tang]

For sample application support 'utxt' scrape type- download
ftp://dev.apple.com/devworld/Sample_Code/Text/TypeServicesForUnicode.sit
You need MacOS 9 to run that application. But the code in SampleWindow.cp show 
you the implementation.

Comment 7

[nhottanscp]

Move to M15.
24010 is for more generic (non unicode) clipboard bug. Although this bug does 
not depend on 24010, I think it's simpler to wait for that to be implemented 
before doing this unicode specific case.

Comment 8

[nhottanscp]

Is this supported by other apps?
My concern is that implement this may complicate the testing. And if not used by 
other apps then no benefit to do this.
So I am considering to move this to M20.

Comment 9

[nhottanscp]

moving to M20

Comment 10

[Leif Halvard Silli]

Isn't it time to upgrade it's priority? And fix this bug?!

This bug *prevents* -- in both mail/news, composer and browser -- (perhaps 
"editor" is the right module to target?) transport via the clipboard to/from 
external programs of any other text than *pur ascii*. I.E. if you try to copy 
some cyrillic text from any Mozilla module and paste it into a any mac editor, 
the cyrillic text is pasted in as a lot of empty spaces. 

The only "workaround" is to convert any text you want to import in Mozilla to 
html first. Or to save the Mozilla document as html first and then open the html 
document in for example IE or iCab and copy things from there.

Actually, I *think* the clipboard worked in Netscape6. But it has never worked in 
the post n6 Mozilla builds. At least not on MacOS 9.0, 9.0.4 or 9.1.

Comment 11

[Karl Ove Hufthammer]

Changing component:

"XP Toolkit also accepts bugs relating to copy/paste, drag & drop, key bindings,
the clipboard [...]"

Comment 12

[Karl Ove Hufthammer]

Bug #79864 may be a dup of this (it deals with copying ISO-8859-1 to the
clipboard).

Comment 13

[Peter Trudelle]

->pink for triage, clearing future res.

Comment 14

[Mike Pinkerton (not reading bugmail)]

back to i18n

Comment 15

[nhottanscp]

Sorry I did not respond earlier.

>Actually, I *think* the clipboard worked in Netscape6. But it has never worked in
>the post n6 Mozilla builds. At least not on MacOS 9.0, 9.0.4 or 9.1.
NS6 supports text in system charset for inter application copy/paste, i.e. you
need to have an localized OS.
Is the problem you mentioned above same as the one reported as bug 79864?

Comment 16

[Leif Halvard Silli]

No -- unfortunately the behaviour fo Netscape 6.01 is the same as Mozilla of
todays build... 

And yes, I first filed a comment here before making my own bug report because I
supposed it could the same bug. And I still am inclined to think it is the same
bug. I think the answer laysl in what Frank Tangs has said, perhaps.

Comment 17

[nhottanscp]

I modified my local build to put 'utxt' to clipboard code but nobody seems to
support it on MacOS 9.1. Other applications supports 'styl' (sytled text), I
think mozilla does not support 'styl'. With 'styl', text can be associated with
fonts then fonts can be mapped to scripts, so they can copy/paste without
depending on a system charset unlike mozilla.

I also tried MacOS X with 'utxt' enabled Carbon build. Apple's Mail supports
'utxt' also TextEditor (replacement of simpletext?). IE does seem to support it
but you can paste through TextEditor.
So I think it's worth to do it at least for OS X.

Comment 18

[nhottanscp]

Attachment: Patch, added 'utxt' to MimeMapper.

Comment 19

[Mike Pinkerton (not reading bugmail)]

yup, that's all i was gonna do. r=pink.

Comment 20

[Leif Halvard Silli]

>I modified my local build to put 'utxt' to clipboard code but nobody seems to
support it on MacOS 9.1.

I don't think that is correct. WorldText support it, as far as I now. So does
the free Simple Unicode Editor (SUE), soe does Style. And there is also a
TextEncodingConverter OSAX (TEC.OSAX) that support it.

And even if WorldText uses utxt internally [it at least saves text as utxt],
there is no problem copying to e.g. NisusWriter or SimpleTExt from WT and
vice-versa. I suppose the clipboard is converted when one move between 'styl'
and 'utxt' apps. And I suppose the same thing could/should happen in MOzilla?

I think it's worth to do it also for OS 9.1 because a) I believe apple's
WorldText support it (you find it on the 9.1 CD though perhaps not on the update
CD). And b) because the current situation is so bad. Mozilla is fine as a
browser in the literal meaning of the word. But without clipboard support, it
really ain't working.

Comment 21

[nhottanscp]

Thanks for the info. It's a generic patch, not risticted for OS X.

Comment 22

[Leif Halvard Silli]

I am here inserting some info I just received from my programmist friend from
Poland. He has some info which might be relevant, but he was to busy to log into
the Bugzilla system (He wrote this text about Bug #7986, but due to its
technical nature I include it here instead):

[snip]
Hello,

Leif asked me to take a look at this bug [Bug #7986] and I can confirm he is right. 
Text conversions in Mac OS do not depend on what localized version of 
Mac OS you use. The general engine for conversions is called Text 
Encoding Converter (TEC). You may take a look at my program "Cyclone" 
which enables conversions of text from many encodings. I have never 
heard that it is not working properly in Russian, Polish, Hebrew or 
American English Mac OS. It is simply unrelated. Proper display of the 
text is another story and requires language kits.
Whether you use TEC in Mac Mozilla, or you use some internal converter 
engine you should not assume that the only language available to user is 
the main language of operating system and you should not base the 
conversion on this information. When you know the input encoding, let's 
say it is a ISO Latin 2 (Central European) you are supposed to convert 
it to Mac CE regardless of Mac OS localization. You say you use Unicode 
text for internal operations, that is great, why don't you put 'utxt' 
clipboard flavor in addition to 'TEXT' flavor?. This way Unicode-enabled 
applications would be able to use Unicode text directly and not 
reconvert it.
My programs are available at:
<http://homepage.mac.com/tkukiel/>
SUE (Simple Unicode Editor) project is available with sources. SUE is 
able to import and export text with help of TEC. I do not deal with 
clipboard in SUE because it is all done by Multilingual Text Engine on 
which SUE is based. You may use the sources for text conversion if you 
want. If you need the sources for clipboard conversion from Cyclone, I 
may open it as well, but it is very similar to file conversion which is 
available in SUE.

Regards,

Tomasz Kukielka
[e-mail]

Comment 23

[Leif Halvard Silli]

Sorry, I meant referring his note to Bug 79864

Comment 24

[Leif Halvard Silli]

I am including yet some other comment about how some programs handle utxt on the
clipboard. From Tomasz:

On Friday, May 11, 2001, at 02:17 , Leif Halvard Silli wrote:

> Perhaps you could tell me (or better: THEM) if/that WolrdText, SUE and 
> Style are using utxt on the clipboard?
Regarding WorldText and SUE - I am not sure what Apple does in MLTE. I 
know there was a bug in early version of MLTE that prevented from 
reading 'utxt' clipboard format. About Style - it would be the best to 
ask Marco, but I guess it tries to read 'TEXT' with 'styl' first, and 
then if it is not present, it reads 'utxt' (which is without style 
information). If I may guess, MLTE does the same. Actually I think it 
should be done the other way: 'utxt' should be "preferred" since we do 
not know if 8-bit text carries correct, but this is not acceptable for 
WASTE an MLTE because you lose style information.
Cyclone supports 'utxt' in a sense that if you convert clipboard from 
Unicode it takes 'utxt' flavor first and when you convert to Unicode 
(16-bit of course) it puts 'utxt'.

Tom

Comment 25

[nhottanscp]

I forgot about WASTE, yes that is used in 4.x. I heard about MLTE that replacing
TE in OS X. Since mozilla has its own text widget, I don't think we going to
replace it by MLTE. Anyway, in case there is an enhancement request, please file
as a separate bug, thanks.

Comment 26

[Leif Halvard Silli]

WASTE <URL http://www.merzwaren.com/ > is free for freeware apps. WASTE 2.0 is
moving towards MLTE functionality, btw. It is used in iCab, and mac OE/IE but
was not especially successfull in Communicator I think... I'll consider filing a
bug for *real* style support but WASTE as such is not important. However, to get
it right: MLTE is allready here, in 'Mac OS 9.x': SUE uses it. As does
WorldText. As does MLTE-demo from merzwaren.com.

Comment 27

[Christopher Blizzard (:blizzard)]

sr=blizzard

Comment 28

[nhottanscp]

checked in

Comment 29

[Andreas Becker]

Changing QA contact to [e-mail].  Yuying, can you verify this?  Please 
let Frank know in case you need more information regarding verifying this bug.

Comment 30

[Leif Halvard Silli]

Tested Build 2001051508 (don't know if the fix is included there?) trying to
copy cyrillic text in Mozilla and paste into WorldText [WT]. If I select and
copy *only* cyrillic text (no ASCII including *no* spaces, commas, dots etc),
then the cyrillic text will be pasted into WT. But as soon as I also include
ascii text the behaviour is as befor. However, I think I have not tested
selecting only cyrillic characters in earlier builds, so it could be that this
is nothing new.

Comment 31

[Leif Halvard Silli]

I now tested the same thing in a build from 9th of may. And there not even
copying just Cyrillic worked. Hence I draw the conclusion that the fix has been
checked in in todays test build but that it ain't working... - yet.

Comment 32

[nhottanscp]

There is a regression caused by this checkin bug 81028. And I think paste from
other app will not work without that fix. So please wait for that to be fixed
for verification.

I don't have WorldText, does it supports 'utxt'? From the comment of 2001-05-11
12:21, not clear if it supports it or not.
Anyway, verification can be done by using MacOS X with TextEdit or MacMail.

Comment 33

[Leif Halvard Silli]

Yes, WorldText support 'utxt'. Select "view Clipboard" and you will see info
telling you if the content is unicode.  If you have OS 9.1 full install CD, you
will find WorldText in CD Extras folder. Else, try MLTE demo from
<http://www.merzwaren.com/bin/mltedemo-10a3.sit>. SUE unfortunately does not
have "view clipboard".

Comment 34

[Leif Halvard Silli]

You refer to bug 81028, but drag and drop works... as long as the text is from
iso-8859-1 range.

Also, WorldText tells me that:
- cyrillic text only is copied as Unicode Text (unformatted). This works.
- cyrillic + ascii is copiead  as formatted text *and* formatted unicode text.
  But this does not work. Only ASCII part is copied.
  This is in my view a logical consequense of Mozillas lack of styl support.
  I suppose that if also the ASCII part is copied as unformatted unicode text,
  this could work. So, please: either turn off formatted text copyiing or hurry
  up and fix bug 79864. 
- ascii/iso-8859-1 text is copied as formatted text and formattd unicode text
  And this works. And this is also logical and obvious.

Comment 35

[marina]

reassigned QA contact

Comment 36

[nhottanscp]

*** Bug 78039 has been marked as a duplicate of this bug. ***

Comment 37

[marina]

I am using BBEdit Lite 6.1 on Mac OS 9.0 : the copy/paste work for non-ascii
Latin-1 chars. Cyrillic chars re turning into question marks (???) Does BBEdit
Lite support "utxt"? I'll reopen. Leif, please comment on this , nhotta is on
vacation.

Comment 38

[Leif Halvard Silli]

Hi Marina, I'm not sure about unicode status of BBEdit *lite*.
According to my testing with Build ID 2001083110
one *cannot* copy *from* Mozilla *to* unicode aware editor (like WorldText). But
the reason for this is ??as it was before-- that WorldText prefers styled text
over Unicode text. And since copying inside Mozilla (still) places both a layer
of  UNicode text and styled text on the clipboard even if the styled text is not
usable, WorldText receives the  cyrillic text as question marks. While editors
which prefers unicode text over styled text, has no problems whatsoever. An
example on a such editor is Pepper and also --I think-- BBEdit full version.

Comment 39

[marina]

Leif, thanks for the prompt reply. So my understanding is that this "fix' that
i've  undone and resolved the bug "reopen" will fix only cases when editor
prefers styled text over Unicode text, all other scenarios will fail ( and
display ???), right?

Comment 40

[Leif Halvard Silli]

Hi, Marina.I belive that it is the opposite. When the editor prefers *unicode
text* over styled text, things will work. When the editor prefers *styled text*
we will get questionmarks, even if the editor is Unicode savvy. And the reason
for this is to be find in another bug: the lack of support for (multilingual
text via) styled text.In fact, the question marks is something I believe nhotta
fixed. He made it so that non-ascii text is copied to the *styled text* layer
(of the clipboard) as question marks, as a kind of warning that there is an
encoding problem. The unicode layer (of the clipboard) is untouched by this, so
that if you paste the same text into a different unicode aware editor that
prefers utxt, things will work.

In my opinion the styled text layer is totally useless. Hence in my view Mozilla
should only skip the styled text layer on the Mac platform. At least any text
with non-ascii text should not be copied as styled text. It just makes it more
incompatible that it in needs to be. Just think about the WorldText case.
WorldText is a unicode aware editor. But it also understand styled text. And it
does so because it needs to be compatible with old editors. Mozillal doesn't
have the ability to work with old multilingual editors *at all* until it gets
styled text support. So why make oneself imcompatible just by that useless
styled text layer?

Comment 41

[marina]

Leif, thanks very much for this comprehensive clarification. Using TextEdit on
Mac OSX i was able to paste into the Editor cyrillic strings and display them
correctly as well as japanese.Even those strings that had non-ascii and ascii
chars were pasted just fine. And i agree with you that the styled text layer is
useless..Now it looks like BBEdit lite prefers styled text over unicode and my
undersatnding is with old editor copy/paste from mozilla won't work. So i am
marking this bug as fixed for now ( even though there will cases when paste
won't work)

Comment 42

[marina]

verifying with 2001-09-12 build on Mac OSX