56010 - Can't copy some international characters into the clipboard.

Status: VERIFIED FIXED

(Core :: XUL, defect, P1)

Description

[Joseph Elwell]

Mozilla is having problems copying/pasting the ' character. (Note: I can't not
seem to type in the ' character that is causing problems. Use the URL given to
reproduce!) Also note that the Html source does not have any peculiar escaped
characters. I'll post the source as an attachment.

Steps to Reproduce:
1) Load URL
2) Highlight the part that says, "Netscape's lack of Lightweight Directory
Access Protocol support, will be a deathblow to the browser"
3) Paste contents to another application or to Mozilla's URL field.

Actual Results:
Only "Netscape" get's pasted.

Expected Results:
"Netscape's lack of Lightweight Directory Access Protocol support, will be a
deathblow to the browser" is pasted.

Comment 1

[Joseph Elwell]

I'm testing on linux 2000100909 mn6 build. I notice in vi that the ' character
in the file shows up as "~R".

Comment 2

[Joseph Elwell]

Attachment: testcase.

Comment 3

[Simon Fraser [no longer active]]

-> brade. possibly more entity issues.

Comment 4

[Akkana Peck]

Cc jst, jfrancis: noxif entity issues.

Comment 5

[Simon Fraser [no longer active]]

On Mac when I paste into composer, I see the ’ (curly apostrophe) character in th 
raw, i.e. we're not doing any encoding. The page source contains a raw ’ too.

An example of a page with encoded entities:
http://abcnews.go.com/sections/world/DailyNews/mideast_001012.html
like:
“anguish”
When I copy-paste this into composer on Mac, I again get unencoded “”, rather 
than the entities.

Comment 6

[Akkana Peck]

Are people still seeing this on today's build?  I don't see it -- I get the
entire line pasted, including the apostrophe.

Comment 7

[Kathleen :Brade]

I think the NOXIF landing fixed this bug (or one of the other subsequent 
patches).  Resolving as works for me.

Regarding Simon's comments, I think that's a separate issue (not related to 
clipboard stuff) but I think that's also been fixed with today's builds.  (jst 
fixed some entity issues that were appearing with the NOXIF landing).

JElwell--please reopen if you can reproduce the problem you reported.

Comment 8

[Joseph Elwell]

I still see this on linux build 2000101309. I hope you're not trying to test
with anything other than the text at the zdnet url or the testcase attached to
this bug. Everything else is the plain ole apostraphe that I can type in on my
keyboard.

Comment 9

[Blake Ross]

(the ZDNet headline has changed)

With the testcase, although the whole line is properly copied for me, I see 
what Simon saw -- a ` instead of a ' 

win32 branch

Comment 10

[Johnny Stenback (:jst)]

So, I actually looked into this and it turns out this is unrelated to the NOXIF
change, we had the same problem before we landed any of the NOXIF changes.

The problem is the the textarea control (and input controls, including the URL
bar) don't seem to handle unicode at all on linux (works fine on win and mac)
and that's why we're unable to paste the weird quote on linux. I'm reassigning
this to rods in hope that he'll know where this problem is, or knows who this
belongs to.

Comment 11

[Johnny Stenback (:jst)]

Well, I talked to Pavlov about this and he whipped up a fix in less than a
minute and he agreed to take this bug. Reassigning. Thanks Pavlov!

Comment 12

[Peter Trudelle]

This also affects other I18N characters, so it is a common data loss scenario.
Fix is just reordering the types we look for to prefer UTF-8, one line, should
be very safe. rtm+ need info

Comment 13

[Stuart Parmenter]

Attachment: patch to fix problem

Comment 14

[Stuart Parmenter]

this patch solves the problem by asking for UTF8_STRING before asking for
COMPOUND_TEXT.  Compound text requires us to convert the data to the current
locale, and if that locale does not understand some characters, then you'll get
what we have here.  UTF8_STRING on the other hand, won't do this conversion, and
therefore we won't lose any data.

Comment 15

[Jason Eager]

Updated the summary based on one comment saying that it affects most
international characters.

Comment 16

[Stuart Parmenter]

someone please review this and super review this.

Comment 17

[Brendan Eich [:brendan]]

r=[e-mail].

/be

Comment 18

[Akkana Peck]

r=akkana, though I hope someone from I18n also looks at it.

Note, the only place where the problem shows up for me in today's builds is
pasting into the urlbar.  I can paste from the attached testcase into rxvt just
fine even before the patch (shows up as a quote).  But the patch does fix it in
the case of pasting into our own urlbar.

Comment 19

[Erik van der Poel]

Pav, when it tries UTF8_STRING first, is it actually asking the destination
program whether it supports UTF8_STRING? My concern is that UTF8_STRING is
relatively new, so there are still many programs out there that don't support
it. If we use UTF8_STRING without confirming that the destination program
supports it, then we could have problems.

Comment 20

[Stuart Parmenter]

No, that won't be a problem.  If it returns no data to us when we ask for
UTF8_STRING, then we will ask for COMPOUND_TEXT, then STRING, etc until
(hopefully) we get something.

removing the need info

Comment 21

[Erik van der Poel]

OK, thanks for the clarification. Yes, we should try UTF8_STRING first, then
COMPOUND_TEXT, and finally STRING. So, I agree with the patch. sr=erik

Comment 22

[Jim Roskind]

RTM double plus
Please land on branch asap.
The risk seems really low.. and pasting into the URL bar seems like a Good Thing
(TM) to get dead right.

Comment 23

[Christopher Blizzard (:blizzard)]

sr=blizzard

Comment 24

[Stuart Parmenter]

checked in fix on branch and trunk

Comment 25

[Stuart Parmenter]

i meant to mark this fixed

Comment 26

[lchiang]

I've verified on:
win32, linux, and mac 2000-11-04-09 builds

Able to paste in "Netscape 6?s lack of Lightweight Directory Access Protocol
support, will be a deathblow to the browser?s corporate appeal." in both URL
field and here, in this bug report field. Also, tried the testcase.

verified for branch.  marking vtrunk in keyword for blake to verify on trunk.

Comment 27

[lchiang]

hmmm... could this be a new problem now?  After I posted, I see that the ' has
turned into a question mark (in my previous comments).  

Comment 28

[Colin Blake]

Hmmm, this "low risk" fix broke pasting on OpenVMS. See bug 65070 for details. 
If anyone knows WHY it broke OpenVMS, please reply to bug 65070. Thanks!

Comment 29

[Terri Preston]

Verified fixed linux build 2001102910