38016 - – or ” breaks pasting as plaintext

Status: RESOLVED FIXED

(Core :: XUL, defect, P4)

Description

[R.K.Aa.]

M16 linux 2000-050215
if text contains the signs – or ” (or equivalent "plaintext") it's
impossible to copy "beyond" them. Text gets selected, but when you paste it,
it's cut off.

Comment 1

[R.K.Aa.]

Attachment: try copy whole lines

Comment 2

[Akkana Peck]

Cc'ing nhotta, who might know something more about this.

Incidentally, I don't see the problem on linux when pasting from one composer
window into another (which is the only app I have which understands html paste
or html entities).  What are you pasting into?  What are you seeing cut off?

Comment 3

[Akkana Peck]

Turns out that this happens when pasting as *plaintext*, at least into some
apps, like xterm apps.

If you load the attachment in the editor and use the debug menu to show the
selection (Debug->Test Selection), you see that the output system is providing
the whole line, but it has a funny unprintable character in the middle, and
Linux apps get confused by this and won't deal with the rest of the line.

Naoki and I had a discussion about this a while back with regard to html output,
and the upshot was that the default behavior is to keep entities in the output
stream rather than encoding them to a form like – or  .  Unix systems
can probably live with that since we don't have many apps that handle html paste
anyway.  But the issue is still unresolved for plaintext: keeping the raw
characters does cause the problem described in this bug, but we can't turn them
into the html entity representation either.

Perhaps the best solution is that in the paste code, if we're ultimately going
to ascii rather than unicode (as is the case most of the time on Unix systems),
we need to strip out any characters which aren't legal ascii.  The clipboard
could do that as a postprocess, or it could set
nsIDocumentEncoder::OutputEncodeEntities and I could make the output sink do
it.  Cc'ing clipboard owners -- Pav, which solution would be easier?  Naoki and
Frank, does this solution sound acceptable for I18n users on Unix?

Comment 4

[John Morrison]

I was seeing one of two things happen when pasting into other apps:

1) initially, I was getting:
      ->>>>>>>>>>>>>> Read Clipboard from memory
      Transferable didn't support data flavor text/unicode (type = 31)
   but then this automagically stopped happening.

2) when pasting into (e.g.) gnotepad or a term, the paste would stop at 
   the location of the first entity in the selection

Comment 5

[Stuart Parmenter]

I would expect nsPrimitiveHelpers::ConvertUnicodeToPlatformPlainText()
(http://lxr.mozilla.org/seamonkey/source/widget/src/xpwidgets/nsPrimitiveHelpers.cpp#134)
which uses a charset converter to do this... Maybe there is a bug in the charset
encoder code?

Comment 6

[nhottanscp]

I have a similar bug for inter application copy/paste 31797.
I am going to fix that by using one of the encoder function which substitute not 
converted characters.
Here's my patch for 31797.

Index: nsPrimitiveHelpers.cpp
===================================================================
RCS file: /cvsroot/mozilla/widget/src/xpwidgets/nsPrimitiveHelpers.cpp,v
retrieving revision 1.10
diff -c -r1.10 nsPrimitiveHelpers.cpp
*** nsPrimitiveHelpers.cpp	2000/04/17 08:21:12	1.10
--- nsPrimitiveHelpers.cpp	2000/05/03 20:11:45
***************
*** 164,169 ****
--- 164,170 ----
    if ( *outPlainTextLen ) {
      *outPlainTextData = NS_REINTERPRET_CAST(char*, 
nsAllocator::Alloc(*outPlainTextLen + sizeof(char)));
      if ( *outPlainTextData ) {
+       rv = 
encoder->SetOutputErrorBehavior(nsIUnicodeEncoder::kOnError_Replace, nsnull, 
'?');
        rv = encoder->Convert(inUnicode, &inUnicodeLen, *outPlainTextData, 
outPlainTextLen);
        (*outPlainTextData)[*outPlainTextLen] = '\0';          // null 
terminate. Convert() doesn't do it for us
      }


The other possibility is to use transliterate option of the entity encoder. 
Which translates a smart quote to double quote (0x22) for example.
http://lxr.mozilla.org/seamonkey/source/intl/unicharutil/idl/nsIEntityConverter.
idl

Comment 7

[nhottanscp]

I checked in the fix for 31797.
Those characters which cannot be converted to ISO-8859-1 will be substituted 
with '?'.

Comment 8

[Peter Trudelle]

reassigning to pavlov as p4 for m18

Comment 9

[Peter Trudelle]

mass-moving all bugs to m21 that are not dofood+, or nsbeta2+

Comment 10

[Stuart Parmenter]

 nhotta's fix basically fixes the problem of us not pasting enough... but I
think we should convert "smart quotes" in to normal quotes for ascii instead of
leaving them as question marks.  We should open another bug on that issue.

Comment 11

[Stuart Parmenter]

bug 46729 filed