Closed Bug 1547595 Opened 5 years ago Closed 4 years ago

GTK puts \r\n line endings into the Wayland Clipboard on Linux

Categories

(Core :: Widget: Gtk, defect, P3)

Product:
Core
Component:
Widget: Gtk
Version:
66 Branch
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
81 Branch
Tracking Flags:
Tracking Status
thunderbird_esr78 --- wontfix
firefox-esr78 --- wontfix
firefox79 --- wontfix
firefox80 --- wontfix
firefox81 --- fixed

People

(Reporter: stefan, Assigned: evilpie)

References

(Blocks 1 open bug,
URL
)

Details

Attachments

(3 files)

Bug 1547595 - Work around brokeness in GTK with CRLF copying. r?stransky
47 bytes, text/x-phabricator-request
Details | Review
Bug 1547595 - browser_removeUnsafeProtocolsFromURLBarPaste has stopped failing now that \n isn't changed to \r\n CLOSED TREE
47 bytes, text/x-phabricator-request
Details | Review
Bug 1547595 - Work around brokeness in GTK with CRLF copying. r?stransky
47 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

I use the sway window manager with Firefox on arch linux. I have set GDK_BACKEND=wayland to enable wayland support.

I copied a file from github (with \n line endings) in the wayland clipboard. Firefox seems to always put \r\n windows line endings into the wayland clipboard.

Actual results:

When you copy this file 1 into the wayland clipboard using CTRL+C, firefox seems to always use \r\n line endings. This can be confirmed with:

$ wl-paste | hexdump -C | grep 0a | tail
00005990  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
000059e0  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005a30  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005a80  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005ad0  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005b20  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005b70  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005bc0  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005c10  20 20 7d 0d 0a 20 20 2d  20 7b 20 6b 65 79 3a 20  |  }..  - { key: |
00005c60  20 20 7d 0d 0a 0a

vs

$ curl -Ls https://raw.githubusercontent.com/jwilm/alacritty/master/alacritty.yml | hexdump -C | grep 0a | tail
00005730  20 20 20 20 20 20 20 20  20 20 20 20 20 7d 0a 20  |             }. |
00005780  20 20 20 20 20 20 20 20  20 20 20 20 7d 0a 20 20  |            }.  |
000057d0  20 20 20 20 20 20 20 20  20 20 20 7d 0a 20 20 2d  |           }.  -|
00005820  20 20 20 20 20 20 20 20  20 20 7d 0a 20 20 2d 20  |          }.  - |
00005870  20 20 20 20 20 20 20 20  20 7d 0a 20 20 2d 20 7b  |         }.  - {|
000058c0  20 20 20 20 20 20 20 20  7d 0a 20 20 2d 20 7b 20  |        }.  - { |
00005910  20 20 20 20 20 20 20 7d  0a 20 20 2d 20 7b 20 6b  |       }.  - { k|
00005960  20 20 20 20 20 20 7d 0a  20 20 2d 20 7b 20 6b 65  |      }.  - { ke|
000059b0  20 20 20 20 20 7d 0a 20  20 2d 20 7b 20 6b 65 79  |     }.  - { key|
00005a00  20 20 20 20 7d 0a                                 |    }.|

Expected results:

I would expect to see \n line endings in the wayland clipboard, since the original data also consists of \n line endings.

Component: Untriaged → Widget: Gtk
Product: Firefox → Core
Blocks: wayland
Priority: -- → P3

I confirm this bug on Firefox 67.0.4. Wayland activated by setting MOZ_ENABLE_WAYLAND=1 in ~/.profile. Using sway as WM on Arch.

Experiencing the same issue on Firefox 68.0.1, but I am using KDE Plasma with kwin on X.

I can confirm this using Firefox 69.0, using the sway window manager on Arch Linux.

Can confirm this on X11 KDE in Arch. This is not a wayland specific issue. Copying on firefox and pasting to vim prompts "Other unprintable character (\xd)". Pasting to Kate, and the ending is converted to ^M^M. Is this bug given a lower priority because of the wayland keyword?

(In reply to danielee0707 from comment #4)

Can confirm this on X11 KDE in Arch. This is not a wayland specific issue. Copying on firefox and pasting to vim prompts "Other unprintable character (\xd)". Pasting to Kate, and the ending is converted to ^M^M. Is this bug given a lower priority because of the wayland keyword?

On my system, when unsetting MOZ_ENABLE_WAYLAND and using the X11 version of firefox, copying does NOT include windows line endings.

Are you sure you are actually using X11?

(In reply to theferdi265 from comment #5)

(In reply to danielee0707 from comment #4)

Can confirm this on X11 KDE in Arch. This is not a wayland specific issue. Copying on firefox and pasting to vim prompts "Other unprintable character (\xd)". Pasting to Kate, and the ending is converted to ^M^M. Is this bug given a lower priority because of the wayland keyword?

On my system, when unsetting MOZ_ENABLE_WAYLAND and using the X11 version of firefox, copying does NOT include windows line endings.

Are you sure you are actually using X11?

I think so. I didn't change or include anything in .profile or anything for firefox. Unless Firefox now defaults to Wayland? I'm on X for other things for sure.

Blaz, thanks for your investigation here! Do you mind to create a patch for it?
Thanks.

So I went ahead and tested out the change locally (my first time building firefox, ./mach bootstrap is very convenient!) and it doesn't work. The codepath I looked at is for pasting, not copying.

I'm doing some further debugging now though so I hope to have a patch ready soon. From what I can tell (MOZ_LOG="WidgetClipboard:5"), the codepath that's used for copying is nsClipboard (nsClipboardWayland is only used on pasting) so it should be broken for both X and Wayland. It's possible that X/xclip contains some workaround that fixes the line ending.

I think this is also the same bug as https://bugzilla.mozilla.org/show_bug.cgi?id=1572104 so they should be merged.

After some debugging, it's a GTK3 issue, Firefox correctly passes over \n line endings. I've opened an upstream issue with more information: https://gitlab.gnome.org/GNOME/gtk/issues/2307

As I can see the original bug was reproduced in alacritty(This is a slice from config, at least), so I can say, that we've fixed this issue in the recent version of alacritty (v0.4.0).

The thing you observe is a GTK behavior and not a Firefox bug in general. GTK normalize clipboard content for some mime types. So on every "store" GTK transforms every \r and \n to \r\n and on every load it transforms everything back to \n. This is true for some mime types, like text/plain;charset=utf-8 , I guess on X11 the mime type is UTF8_STRING(X11 specif one iirc), which ends with just \n and most things are fine.

So the question is whether it's a bug or not. If you paste from Firefox to other apps, then it's unlikely a Firefox bug. If you see the same thing when pasting from Firefox to other GTK apps, then I guess its something related to Firefox. The code in GTK isn't new, it's there for decade, at least, however I'd still recommend to ask them about such behavior, however I kind of understand the idea behind this normalization...

Also other clients couldn't work with UTF8_STRING mime type, like X11 apps. So if you paste from ANY Wayland GTK app to mostly all X11 apps ,like WAYLAND_DISPLAY= alacritty (it should launch alacritty over Xwayland), you'll still paste new lines into it. That being said its unclear who should handle such conversions(compositor?).

That's all I can say for now, I'm not a Wayland expert, etc, etc. I was just fixing the same bug in alacritty (paste from firefox to alacritty) and sharing the information I have.

Bugs you might be interested in
https://github.com/swaywm/wlroots/issues/1839
https://github.com/jwilm/alacritty/issues/2844

(In reply to Kirill Chibisov from comment #11)

The thing you observe is a GTK behavior and not a Firefox bug in general. GTK normalize clipboard content for some mime types. So on every "store" GTK transforms every \r and \n to \r\n and on every load it transforms everything back to \n. This is true for some mime types, like text/plain;charset=utf-8 , I guess on X11 the mime type is UTF8_STRING(X11 specif one iirc), which ends with just \n and most things are fine.

Yeah, that's why I opened that upstream GTK issue https://gitlab.gnome.org/GNOME/gtk/issues/2307

The code in GTK isn't new, it's there for decade, at least, however I'd still recommend to ask them about such behavior, however I kind of understand the idea behind this normalization...

I think the main problem is just a difference in behavior between X11 and Wayland. X11 had a bunch of extra code for X integration outside that file that would fix these line endings so it doesn't happen. The same code is applied for XWayland code paths in Wayland, but not for normal paste.

https://github.com/jwilm/alacritty/issues/2844
(sidenote) The fix in alacritty didn't work for me, I still get ^M when pasting to vim.

Is this still valid? I'm seeing the following on FF 73:

000041e0  20 20 20 20 20 20 20 20  20 7d 0d 0a 20 20 23 2d  |         }..  #-|
00004230  7d 0d 0a 0d 0a 20 20 23  2d 20 7b 20 6b 65 79 3a  |}....  #- { key:|
00004280  20 20 20 20 7d 0d 0a 20  20 23 2d 20 7b 20 6b 65  |    }..  #- { ke|
000042d0  20 20 20 20 20 20 7d 0d  0a 20 20 23 2d 20 7b 20  |      }..  #- { |
00004320  20 20 20 20 20 20 20 20  7d 0d 0a 20 20 23 2d 20  |        }..  #- |
00004370  20 20 20 20 20 20 20 20  20 20 7d 0d 0a 20 20 23  |          }..  #|
000043c0  20 7e 41 6c 74 20 20 20  20 20 20 20 7d 0d 0a 20  | ~Alt       }.. |
00004420  0a 20 20 23 2d 20 7b 20  6b 65 79 3a 20 48 6f 6d  |.  #- { key: Hom|
00004470  7d 0d 0a 20 20 23 2d 20  7b 20 6b 65 79 3a 20 45  |}..  #- { key: E|
000044c0  20 20 7d 0d 0a 0a                                 |  }...|

If you're testing it in Alacritty with our clipboard it should be fixed, since v0.4.0, but it was fixed on Alacritty side and not on Firefox, I guess. If you're using wl-paste, I think it should be an issue. I'm talking about Alacritty, since you're pasting our config here. You can try to repro on Alacritty v0.3.3 to test on it though.

I used wl-paste, just as described in comment 0
On GS, not on Sway though.

So results here look like:

$ wl-paste | hexdump -C | grep 0a | tail
000041e0  20 20 20 20 20 20 20 20  20 7d 0d 0a 20 20 23 2d  |         }..  #-|
00004230  7d 0d 0a 0d 0a 20 20 23  2d 20 7b 20 6b 65 79 3a  |}....  #- { key:|
00004280  20 20 20 20 7d 0d 0a 20  20 23 2d 20 7b 20 6b 65  |    }..  #- { ke|
000042d0  20 20 20 20 20 20 7d 0d  0a 20 20 23 2d 20 7b 20  |      }..  #- { |
00004320  20 20 20 20 20 20 20 20  7d 0d 0a 20 20 23 2d 20  |        }..  #- |
00004370  20 20 20 20 20 20 20 20  20 20 7d 0d 0a 20 20 23  |          }..  #|
000043c0  20 7e 41 6c 74 20 20 20  20 20 20 20 7d 0d 0a 20  | ~Alt       }.. |
00004420  0a 20 20 23 2d 20 7b 20  6b 65 79 3a 20 48 6f 6d  |.  #- { key: Hom|
00004470  7d 0d 0a 20 20 23 2d 20  7b 20 6b 65 79 3a 20 45  |}..  #- { key: E|
000044c0  20 20 7d 0d 0a 0a                                 |  }...|

and

$ curl -Ls https://raw.githubusercontent.com/jwilm/alacritty/master/alacritty.yml | hexdump -C | grep 0a | tail
00003ff0  67 6c 65 46 75 6c 6c 73  63 72 65 65 6e 20 7d 0a  |gleFullscreen }.|
00004000  0a 20 20 23 2d 20 7b 20  6b 65 79 3a 20 50 61 73  |.  #- { key: Pas|
00004050  7d 0a 20 20 23 2d 20 7b  20 6b 65 79 3a 20 43 6f  |}.  #- { key: Co|
000040a0  20 7d 0a 20 20 23 2d 20  7b 20 6b 65 79 3a 20 4c  | }.  #- { key: L|
000040f0  20 20 7d 0a 20 20 23 2d  20 7b 20 6b 65 79 3a 20  |  }.  #- { key: |
00004140  20 20 20 7d 0a 20 20 23  2d 20 7b 20 6b 65 79 3a  |   }.  #- { key:|
00004190  20 20 20 20 7d 0a 20 20  23 2d 20 7b 20 6b 65 79  |    }.  #- { key|
000041e0  20 20 20 20 20 7d 0a 20  20 23 2d 20 7b 20 6b 65  |     }.  #- { ke|
00004230  20 20 20 20 20 20 7d 0a  20 20 23 2d 20 7b 20 6b  |      }.  #- { k|
00004280  20 20 20 20 20 20 20 7d  0a                       |       }.|

A decent workaround is to pipe wl-paste to dos2unix (this could have side-effects though if you copied real \r\n line-endings and wanted to preserve them but the chances of that are rare):

$ wl-paste | dos2unix | hexdump -C | grep 0a | tail
00003ff0  67 6c 65 46 75 6c 6c 73  63 72 65 65 6e 20 7d 0a  |gleFullscreen }.|
00004000  0a 20 20 23 2d 20 7b 20  6b 65 79 3a 20 50 61 73  |.  #- { key: Pas|
00004050  7d 0a 20 20 23 2d 20 7b  20 6b 65 79 3a 20 43 6f  |}.  #- { key: Co|
000040a0  20 7d 0a 20 20 23 2d 20  7b 20 6b 65 79 3a 20 4c  | }.  #- { key: L|
000040f0  20 20 7d 0a 20 20 23 2d  20 7b 20 6b 65 79 3a 20  |  }.  #- { key: |
00004140  20 20 20 7d 0a 20 20 23  2d 20 7b 20 6b 65 79 3a  |   }.  #- { key:|
00004190  20 20 20 20 7d 0a 20 20  23 2d 20 7b 20 6b 65 79  |    }.  #- { key|
000041e0  20 20 20 20 20 7d 0a 20  20 23 2d 20 7b 20 6b 65  |     }.  #- { ke|
00004230  20 20 20 20 20 20 7d 0a  20 20 23 2d 20 7b 20 6b  |      }.  #- { k|
00004280  20 20 20 20 20 20 20 7d  0a 0a                    |       }..|
See Also: → 1572104

Following up from bug 1572104. I don't see how this is a Firefox issue. It's not our fault when GTK is explicitly converting LF to CRLF here.

To make matters worse, it's even inconsistent! This is only done for text/plain;charset=utf-8, but not for UTF8_STRING, which Firefox used to use. I would consider this a GTK bug.

I wonder it we can (as a workaround) use "UTF8_STRING" text target instead of "text/plain;charset=utf-8" if that makes such difference.

Matthias Clasen https://gitlab.gnome.org/GNOME/gtk/-/issues/2307#note_681240

It is hard to fix in GTK3, since the data conversion machinery is not cleanly separated from the GTK frontend. In GTK 4, this problem will solve itself

Matthias Clasen https://gitlab.gnome.org/GNOME/gtk/-/issues/2307#note_832398

By all means, use what makes you happy. Firefox is not a GTK app, really

Yeah... Unfortunately using UTF8_STRING as a workaround for now is probably the best solution. Even if everything goes well and GTK4 does land this autumn, integrating it into Firefox is still gonna take a long time I guess.

(In reply to Martin Stránský [:stransky] from comment #19)

I wonder it we can (as a workaround) use "UTF8_STRING" text target instead of "text/plain;charset=utf-8" if that makes such difference.

We could do that, but I think I prefer to just use gtk_selection_data_set for this case and thus bypass the bad GTK code.

Assignee: nobody → evilpies
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Summary: Firefox puts \r\n line endings into the Wayland Clipboard on Linux → GTK puts \r\n line endings into the Wayland Clipboard on Linux

(In reply to Tom Schuster [:evilpie] from comment #21)

(In reply to Martin Stránský [:stransky] from comment #19)

I wonder it we can (as a workaround) use "UTF8_STRING" text target instead of "text/plain;charset=utf-8" if that makes such difference.

We could do that, but I think I prefer to just use gtk_selection_data_set for this case and thus bypass the bad GTK code.

I ended up implementing what Martin suggested before, because I am not confident that some other GTK code relies on the clipboard for text/plain being normalized to CRLF.

Can you re-review, please?

Flags: needinfo?(stransky)

(In reply to Tom Schuster [:evilpie] from comment #24)

Can you re-review, please?

Sure. I'm still thinking about the gtk_target_list_add_text_targets -> gtk_target_list_add change. In that case we're missing "text/plain;charset=utf-8" and perhaps some localized variants here. I wonder that can break html pasting (where "text/plain;charset=utf-8" is used) and if we may also break some exotic locales...

See your patches at Bug 1497580 which introduced the gtk_target_list_add_text_targets() change.

Flags: needinfo?(stransky)

Why would HTML pasting break? That should use text/html. Otherwise we will now usually fallback to UTF8_STRING instead of text/plain;utf-8, which should not really make a difference.

This basically just reverts https://hg.mozilla.org/integration/autoland/rev/80b93ae22c27, except that we continue to use gtk_selection_data_set_text.

Pushed by evilpies@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/372abda7c202
Work around brokeness in GTK with CRLF copying. r=stransky
Attachment #9157941 - Attachment description: Bug 1547595 - browser_removeUnsafeProtocolsFromURLBarPaste has stopped failing now that \n isn't changed to \r\n → Bug 1547595 - browser_removeUnsafeProtocolsFromURLBarPaste has stopped failing now that \n isn't changed to \r\n CLOSED TREE
Pushed by evilpies@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/44e820d9be26
browser_removeUnsafeProtocolsFromURLBarPaste has stopped failing now that \n isn't changed to \r\n CLOSED TREE

https://hg.mozilla.org/mozilla-central/rev/372abda7c202
https://hg.mozilla.org/mozilla-central/rev/44e820d9be26

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
status-firefox79: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla79
Regressions: 1647689
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

This change was backed out in bug 1647689

So do we have some way forward here? I suggested using gtk_selection_data_set instead of gtk_selection_data_set_text for text/plain;charset=utf-8. However it's unclear to me if bypassing the \r\n "normalization" is going to cause issues for other apps that would depend on that.

What do you think we should do?

Flags: needinfo?(stransky)

(In reply to Tom Schuster [:evilpie] from comment #34)

What do you think we should do?

Please try the solution suggested at comment 33.

Flags: needinfo?(stransky)

I tried the latest patch and it solve the problem perfectly with KDE Plasma on X11

Thanks for testing. Jan can you review? Martin is away.

Flags: needinfo?(jhorak)

Looks good, thanks for the patch.

Flags: needinfo?(jhorak)
Pushed by evilpies@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/ee44f7dc5f2b
Work around brokeness in GTK with CRLF copying. r=jhorak

Backed out changeset ee44f7dc5f2b (bug 1547595) for browser_removeUnsafeProtocolsFromURLBarPaste.js failures

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&searchStr=linux%2C18.04%2Cx64%2Casan%2Copt%2Cmochitests%2Ctest-linux1804-64-asan%2Fopt-mochitest-browser-chrome-e10s-5%2Cm%28bc5%29&fromchange=fc7a06e1632f7dca6b8e0017c680fd4ccafa190d&selectedTaskRun=c8qxZIrNQEKJoO3PaO2g1A.1&tochange=e22536c8901ae0746050e891b57f6ee4a3e8fe2c

Backout link: https://hg.mozilla.org/integration/autoland/rev/e22536c8901ae0746050e891b57f6ee4a3e8fe2c

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=309352390&repo=autoland&lineNumber=3072

[task 2020-07-10T17:34:48.198Z] 17:34:48     INFO - TEST-START | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js
[task 2020-07-10T17:34:53.556Z] 17:34:53     INFO - TEST-INFO | started process screentopng
[task 2020-07-10T17:34:53.960Z] 17:34:53     INFO - TEST-INFO | screentopng: exit 0
[task 2020-07-10T17:34:53.960Z] 17:34:53     INFO - Buffered messages logged at 17:34:48
[task 2020-07-10T17:34:53.964Z] 17:34:53     INFO - Entering test bound test_stripUnsafeProtocolPaste
[task 2020-07-10T17:34:53.965Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:' - 
[task 2020-07-10T17:34:53.965Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript: strips relevant bits. - "" == "" - 
[task 2020-07-10T17:34:53.965Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:1+1' - 
[task 2020-07-10T17:34:53.965Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript:1+1 strips relevant bits. - "1+1" == "1+1" - 
[task 2020-07-10T17:34:53.966Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:document.domain' - 
[task 2020-07-10T17:34:53.966Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript:document.domain strips relevant bits. - "document.domain" == "document.domain" - 
[task 2020-07-10T17:34:53.966Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: ' 	javascript:document.domain' - 
[task 2020-07-10T17:34:53.967Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering  	javascript:document.domain strips relevant bits. - "document.domain" == "document.domain" - 
[task 2020-07-10T17:34:53.967Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'java
[task 2020-07-10T17:34:53.967Z] 17:34:53     INFO - script:foo' - 
[task 2020-07-10T17:34:53.968Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering java
[task 2020-07-10T17:34:53.968Z] 17:34:53     INFO - script:foo strips relevant bits. - "foo" == "foo" - 
[task 2020-07-10T17:34:53.970Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'java	script:foo' - 
[task 2020-07-10T17:34:53.971Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering java	script:foo strips relevant bits. - "foo" == "foo" - 
[task 2020-07-10T17:34:53.971Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'http://
[task 2020-07-10T17:34:53.971Z] 17:34:53     INFO - example.com' - 
[task 2020-07-10T17:34:53.972Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering http://
[task 2020-07-10T17:34:53.972Z] 17:34:53     INFO - example.com strips relevant bits. - "http://example.com" == "http://example.com" - 
[task 2020-07-10T17:34:53.972Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'http://
[task 2020-07-10T17:34:53.972Z] 17:34:53     INFO - example.com
[task 2020-07-10T17:34:53.973Z] 17:34:53     INFO - ' - 
[task 2020-07-10T17:34:53.975Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering http://
[task 2020-07-10T17:34:53.975Z] 17:34:53     INFO - example.com
[task 2020-07-10T17:34:53.975Z] 17:34:53     INFO -  strips relevant bits. - "http://example.com" == "http://example.com" - 
[task 2020-07-10T17:34:53.976Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'data:text/html,<body>hi</body>' - 
[task 2020-07-10T17:34:53.977Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering data:text/html,<body>hi</body> strips relevant bits. - "data:text/html,<body>hi</body>" == "data:text/html,<body>hi</body>" - 
[task 2020-07-10T17:34:53.977Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javaScript:foopy' - 
[task 2020-07-10T17:34:53.979Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javaScript:foopy strips relevant bits. - "foopy" == "foopy" - 
[task 2020-07-10T17:34:53.979Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javaScript:javaScript:alert('hi')' - 
[task 2020-07-10T17:34:53.980Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javaScript:javaScript:alert('hi') strips relevant bits. - "alert('hi')" == "alert('hi')" - 
[task 2020-07-10T17:34:53.984Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:javascript:alert('hi!')' - 
[task 2020-07-10T17:34:53.984Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript:javascript:alert('hi!') strips relevant bits. - "alert('hi!')" == "alert('hi!')" - 
[task 2020-07-10T17:34:53.985Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'data:data:text/html,<body>hi</body>' - 
[task 2020-07-10T17:34:53.985Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering data:data:text/html,<body>hi</body> strips relevant bits. - "data:data:text/html,<body>hi</body>" == "data:data:text/html,<body>hi</body>" - 
[task 2020-07-10T17:34:53.985Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:data:javascript:alert('hi!')' - 
[task 2020-07-10T17:34:53.986Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript:data:javascript:alert('hi!') strips relevant bits. - "data:javascript:alert('hi!')" == "data:javascript:alert('hi!')" - 
[task 2020-07-10T17:34:53.988Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'javascript:data:text/html,javascript:alert('hi!')' - 
[task 2020-07-10T17:34:53.988Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering javascript:data:text/html,javascript:alert('hi!') strips relevant bits. - "data:text/html,javascript:alert('hi!')" == "data:text/html,javascript:alert('hi!')" - 
[task 2020-07-10T17:34:53.988Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: 'data:data:text/html,javascript:alert('hi!')' - 
[task 2020-07-10T17:34:53.989Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering data:data:text/html,javascript:alert('hi!') strips relevant bits. - "data:data:text/html,javascript:alert('hi!')" == "data:data:text/html,javascript:alert('hi!')" - 
[task 2020-07-10T17:34:53.990Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Clipboard has the given value: '
[task 2020-07-10T17:34:53.990Z] 17:34:53     INFO - 
[task 2020-07-10T17:34:53.990Z] 17:34:53     INFO - 
[task 2020-07-10T17:34:53.990Z] 17:34:53     INFO - javascript:foo' - 
[task 2020-07-10T17:34:53.991Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering 
[task 2020-07-10T17:34:53.991Z] 17:34:53     INFO - 
[task 2020-07-10T17:34:53.991Z] 17:34:53     INFO - 
[task 2020-07-10T17:34:53.992Z] 17:34:53     INFO - javascript:foo strips relevant bits. - "foo" == "foo" - 
[task 2020-07-10T17:34:53.992Z] 17:34:53     INFO - Buffered messages logged at 17:34:50
[task 2020-07-10T17:34:53.993Z] 17:34:53     INFO - Console message: [JavaScript Error: "Unknown Collection "main/messaging-experiments"" {file: "resource://services-settings/RemoteSettingsClient.jsm" line: 159}]
[task 2020-07-10T17:34:53.994Z] 17:34:53     INFO - UnknownCollectionError@resource://services-settings/RemoteSettingsClient.jsm:159:5
[task 2020-07-10T17:34:53.994Z] 17:34:53     INFO - sync@resource://services-settings/RemoteSettingsClient.jsm:468:13
[task 2020-07-10T17:34:53.994Z] 17:34:53     INFO - 
[task 2020-07-10T17:34:53.994Z] 17:34:53     INFO - Buffered messages finished
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Timed out while polling clipboard for pasted data, got: java
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - script:foo - 
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - Stack trace:
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - chrome://mochikit/content/browser-test.js:test_ok:1299
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:putAndVerify:1312
[task 2020-07-10T17:34:53.995Z] 17:34:53     INFO - Not taking screenshot here: see the one that was previously logged
[task 2020-07-10T17:34:53.996Z] 17:34:53     INFO - TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | Failed to copy string 'java
[task 2020-07-10T17:34:53.996Z] 17:34:53     INFO - script:foo' to clipboard - false == true - JS frame :: chrome://mochitests/content/browser/browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js :: paste :: line 72
[task 2020-07-10T17:34:53.996Z] 17:34:53     INFO - Stack trace:
[task 2020-07-10T17:34:53.996Z] 17:34:53     INFO - chrome://mochitests/content/browser/browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js:paste:72
[task 2020-07-10T17:34:53.998Z] 17:34:53     INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | entering java
[task 2020-07-10T17:34:53.998Z] 17:34:53     INFO - script:foo strips relevant bits. - "foo" == "foo" - 
[task 2020-07-10T17:34:53.998Z] 17:34:53     INFO - Leaving test bound test_stripUnsafeProtocolPaste
[task 2020-07-10T17:34:53.999Z] 17:34:53     INFO - GECKO(2674) | MEMORY STAT | vsize 20975828MB | residentFast 1719MB
[task 2020-07-10T17:34:53.999Z] 17:34:53     INFO - TEST-OK | browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js | took 5400ms
Flags: needinfo?(evilpies)

Not sure why that patch doesn't fix browser_removeUnsafeProtocolsFromURLBarPaste.js on Linux 18.04, the original patch fixed that test. I can't debug this at the moment.

Flags: needinfo?(evilpies)

So what happens in browser_removeUnsafeProtocolsFromURLBarPaste.js is that we call something like

Cc["@mozilla.org/widget/clipboardhelper;1"].getService(
  Ci.nsIClipboardHelper
).copyString("java\rscript:alert(1)")

Note the \r (Carriage Return). We are calling gtk_selection_data_set so the \r is preserved, but it seems like when calling mContext->GetClipboardText(aWhichClipboard) to retrieve the text from the clipboard, \r is being replaced with \n.

The test using \r is already disabled on Windows: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/browser/browser_removeUnsafeProtocolsFromURLBarPaste.js#53.

Honestly not sure what the correct way forward is.

Flags: needinfo?(stransky)

Sorry, I don't have time to investigate it right now, but I can review any patches here.
Thanks.

Flags: needinfo?(stransky)
Pushed by evilpies@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/16268141400a
Work around brokeness in GTK with CRLF copying. r=stransky,jhorak

https://hg.mozilla.org/mozilla-central/rev/16268141400a

Status: REOPENED → RESOLVED
Closed: 4 years ago → 4 years ago
status-firefox81: --- → fixed
Resolution: --- → FIXED
Target Milestone: mozilla79 → 81 Branch

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

status-firefox80: --- → ?

Can we bring this fix to ESR 78 as well? This would make Thunderbird users like me happy :)

Should I backport the patch or is this done by the Firefox/Thunderbird ESR release team?

@Tom can you uplift those three patches to the ESR branch? So its fixed in Thunderbird as well? TB91 is still half year away.

Flags: needinfo?(evilpies)

I will leave that decision to Martin. I personally don't think this is something we would usually backport. I have seen bug reports with the clipboard in Wayland recently as well.

Flags: needinfo?(evilpies) → needinfo?(stransky)

I don't think we should backport it, the Wayland port is still too raw and it's not aimed to ESR line. Also Thunderbird / Wayland support is quite experimental. If any distro ships TB on Wayland it should backport the patches there (Fedora does that so I can take backported patches there).

Flags: needinfo?(stransky)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: