335783 - Support attaching image screenshot/files/anything from clipboard (copying, cutting, pasting, Ctrl+C, Ctrl+X, Ctrl+V to add as attachment)

Status: NEW

(Thunderbird :: Message Compose Window, enhancement)

Description

[[:nickname]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2

If you press the key "Print" to take a screenshot, Thunderbird should detect that the clipboard contains an image paste it as attachment or inline into the message.

Reproducible: Always

Comment 1

[Mike Cowperthwaite]

Pasting the image *to the message body* is bug 223909 -- and only applies to an HTML-composed message.  That feature is implemented in TB 1.5 and 2a1, by the way, but not on the trunk.

I like the idea of turning an image on the clipboard into an attachment.  An extra item on the Attach toolbutton menu (and on the File|Attach submenu, and the context menu for the attachment panel) would be a good start, and adding Paste support to the attachment panel -- these would work for both HTML and plain text composition.

I'm not so sure about Paste into the message body.  For an HTML message, paste of an image creates an embedded image in the message body; but pasting an image to a plain-text message does nothing.  This is similar to bug 113435 (about drag&drop).

It wouldn't necessarily need to be limited to images; Windows Explorer allows 
a "Copy" action on a file, and the file could be attached that way as well.  
I don't know about making this a general attach-anything-from-clipboard, altho it could be done.

Comment 2

[[:nickname]]

Indeed you were right, it works in HTML mode. I wasn't aware of it because I'm not using it.

Comment 3

[Thomas D. (:thomas8)]

Attach-anything-from-clipboard-option for mail composition, as described by Mike, sounds really cool stuff to me! My vote for this one.

Please update summary of this bug to reflect Mike's broader idea:
Support attaching image (screenshot)/files/anything from clipboard

Comment 4

[[:nickname]]

Updated summary.

Comment 5

[[:nickname]]

If this ever gets supported, renaming of attachments probably is important (Bug#427354).

Comment 6

[rsx11m]

The most intriguing use case for such a feature is obviously pasting an image (e.g., screendump) directly from the clipboard into an attachment while avoiding the additional step of saving it into a file first with some imaging application. While it is already possible to paste into an HTML message body, this may not always be optimal, be it to avoid the overhead of HTML encoding or the desire to make it a "true" attachment due to its size (bug 452092). I'm not sure how useful this would be for plain text or HTML/RTF-formatted code on the clipboard, one may argue that if of sufficient size, creating an attachment rather than pasting such as part of the message body may be preferred depending on size and content. Attaching a file by copy-pasting (bug 379186) can be performed by drag-and-drop, URLs from web pages copy-pasted into Attach > Web Page, thus there would be somewhat less gain in these cases.

(In reply to comment #1)
> extra item on the Attach toolbutton menu (and on the File|Attach submenu, and
> the context menu for the attachment panel) would be a good start, and adding
> Paste support to the attachment panel -- these would work for both HTML and
> plain text composition.

Yes, it would be preferable to have this as an additional item in the Attach menu/panel rather than introducing ambiguities in the Paste function, which is strictly speaking "owned" by the editor backend at the time of composition.
The code implementing communication with the clipboard widgets through nsIClipboard and nsITransferable could be adapted from the HTML Editor.
I'm giving some pointers below from bug 384116 for respective functions in editor/libeditor/html/nsHTMLDataTransfer.cpp:

(1) An adaption of nsHTMLEditor::CanPaste() could be used to identify pastable content on the clipboard, thus deciding whether or not Attach > From Clipboard is visible or disabled. This could also be used to determine upfront already whether we are dealing with an image, text, or a simple reference to a (set of) files or a web page.

(2) As done in nsHTMLEditor::PrepareHTMLTransferable(), an nsITransferable object could be initialized and should obey the "clipboard.paste_image_type" setting for images. Then, the content of the clipboard could be retrieved in the desired "flavor".

(3) For URI references, those could be just applied to the nsIMsgAttachment object for attaching, any content would have to be written into a file first. This is done in nsHTMLEditor::InsertFromTransferable() for the HTML Editor. Rather than creating HTML code with a reference to the temporary file, the respective URI would be used for nsIMsgAttachment, SetUrl() instead.

(In reply to comment #5)
> If this ever gets supported, renaming of attachments probably is important

This was implemented by bug 190298 already. I agree that "moz-screenshot.png" is not very descriptive as attachment name.

Comment 9

[rsx11m]

(In reply to comment #1)
> It wouldn't necessarily need to be limited to images; Windows Explorer allows 
> a "Copy" action on a file, and the file could be attached that way as well.  

I just had a brief look at the Windows clipboard to see what's happening there when a file is copied onto the clipboard. A reference (file path) is stored and marked as a drag-and-drop object, thus it should be feasible to use the same mechanisms to retrieve it as used for a drag-and-drop operation (basically just translating the path into a valid "file://" URI for the attachment).

> (comment #6, pasting of images) I agree that "moz-screenshot.png"
> is not very descriptive as attachment name.

This no longer applies after bug 490879, the pasted content is now directly translated into a "data:" URI. Thus, that part would need to be revived in order to paste the image data into a file, or the corresponding recoding part identified where the "data:" content is converted into an attachment (also see related bug 578104 on defining a useful name for that attachment).

Comment 10

[Thomas D. (:thomas8)]

rsx11m, thank you for moving this forward, sounds really promising...

1.) which program are you using to inspect clipboard?
2.) can you attach a screenshot / output file of clipboard inspection?

Comment 11

[rsx11m]

On Windows XP, there is a nice system utility called CLIPBRD.EXE which can be invoked through the Start > Run menu or the command line. In its View menu, you can see the different flavors the clipboard content is offered, and switch between them to see the representation. It would be a rather dull screenshot, showing just the "C:\..." path for a copied file.

On Linux, there is klipper on KDE, but unfortunately it is much less detailed on the contents. There may be a better tool. I don't known about Mac OSX.

Comment 12

[Thomas D. (:thomas8)]

I am trying to think of this bug in terms of unifying TB routines and behaviour for handling of attachments during composition regardless of the way they were attached. Iow, when implementing this bug, please follow the lines of the proposed solution for:

Bug 378046 - Mail composition: opening/editing attached file sometimes unexpectedly opens/edits original file (only if attachment was added via TB OR drag-and-drop AND draft has never been closed yet)
There's a lot of noise on that bug, read Bug 378046 Comment 60 for a concise and updated summary.

In short: To avoid dataloss and make the behaviour of composition attachments more predictable (e.g. upon viewing/editing them from attachment pane), the default procedure for attaching data must be this:
-> immediately upon attaching (a file, or anything from clipboard that will become an attached file), take a snapshot copy of the data and keep it as an attachment file in TB's tmp directory (to which we link from attachment pane, until the tmp-file is merged with the mail, e.g. when you close the draft).

Notwithstanding the future implementation of *another* way of attaching files/data in whatever state they have *upon sending* of the mail, the above needs to be the default behaviour because
- user can expect that we attach the file/the clipboard data in exactly the state they are in *at the time of attaching* (and not some unpredictable future state, e.g. web page content (copied via clipboard) might look completely different upon sending).
- we need the immediate tmp snapshot file because otherwise the original files  might be removed (removable media), or altered, or renamed, or deleted (dataloss). The same applies for clipboard content (this bug), e.g. web content identified by URI from clipboard etc.
- we need to be consistent about what we show when user views/edits the attachment from attachment pane: either always the original file, or always the tmp file, but not sometimes the former and sometimes the latter (hence bug 378046). It's obvious the *default* must be viewing/editing *the attachment* (i.e., the tmp file), not the original source file (notwithstanding that we might want to provide access to the original file as a courtesy).

Relevance for this bug

Bug 378046 proposes a two-step routine for attaching files:
1) get the full file path of original source files that are to be attached (different routines for Mapi, non-Mapi cases)
From then on, unified routine:
2a) take immediate snapshot copy (tmp file)
2b) merge with message when required (e.g. upon closing the draft)

I am not sure if exactly the same can be applied to this bug, as I don't have enough knowledge about clipboard, its flavors, and relevant code like nsIClipboard. I am sure that we need immediate snapshot files of anything that might have changed if it were attached only later upon sending (like files, or full web page content from clipboard).

(In reply to comment #9)
> (In reply to comment #1)
> > It wouldn't necessarily need to be limited to images; Windows Explorer 
> > allows a "Copy" action on a file, and the file could be attached that way
> > as well.  
> I just had a brief look at the Windows clipboard to see what's happening there
> when a file is copied onto the clipboard. A reference (file path) is stored,
> marked as a drag-and-drop object, thus it should be feasible to use the same
> mechanisms to retrieve it as used for a drag-and-drop operation (basically
> just translating the path into a valid "file://" URI for the attachment).

As explained above, we need to take an immediate snapshot copy of the file(s) instead of linking to the original source file(s). We have to do that anyway in cases where the user used "Cut" from Windows Explorer instead of "Copy".

(In reply to comment #6)
> (3) For URI references, those could be just applied to the nsIMsgAttachment
> object for attaching, any content would have to be written into a file first.
> This is done in nsHTMLEditor::InsertFromTransferable() for the HTML Editor.
> Rather than creating HTML code with a reference to the temporary file, the
> respective URI would be used for nsIMsgAttachment, SetUrl() instead.

I am not sure that I fully understand this use case. Is it this?
- copy/cut a URI (e.g. from Firefox's location bar)
- use Attachment > Paste (implemented by this bug)
In that case, we should also take an immediate snapshot of the current web content identified by URI (as content can change by the time user is sending the mail or closing the draft).

Something similar is currently possible by dragging an URI into attachment pane, and it currently does the wrong thing (just adds a link to the web page and gets the actual content later upon sending, instead of taking an immediate snapshot of the current content at the time when it is attached).

Comment 13

[[e-mail]]

If Thunderbird had this feature it would improve my quality of life.

Comment 14

[rsx11m]

After also drag-and-drop of image files is now using a data: URI to take a snapshot of the file's content per bug 609632, this appears to be a generic enough mechanism to apply to all data pasted from the clipboard without the
need for a temporary file.

A little test case for current trunk builds:
1. Open a composition window;
2. select Attach > Web Page and copy-paste the following as the URL:
   data:text/plain;7bit,Hello World
3. close the Attach dialog, send the message or save as draft.

The message will contain an attachment with the base64-encoded content:

> Content-Type: text/plain;
>  name="data:text/plain;7bit,Hello World"
> Content-Transfer-Encoding: base64
> Content-Disposition: inline;
>  filename="data:text/plain;7bit,Hello World"
> 
> SGVsbG8gV29ybGQ=

The attachment name would be subject to bug 578104, but in general this mechanism works (at least for small items, I don't know if there is any absolute limit of the size of a data: URI).

(In reply to comment #14)
> > (3) For URI references, those could be just applied to the nsIMsgAttachment
> > object for attaching, any content would have to be written into a file first.
> I am not sure that I fully understand this use case. Is it this?
> - copy/cut a URI (e.g. from Firefox's location bar)

If you copy an image from a web page, it will put both the encoded pixel data as well as the URL of the image location onto the clipboard (embedded in HTML). Going with a snapshot solution, the pixel data should be preferred over the reference and can be encoded in a data: snapshot, so this should resolve the issue. Care has to be taken though to get the right flavor off the clipboard if it's available in multiple versions. For example, you may want to attach the actual HTML as a file if it contains the actual content, and not just an image of the content an application may put there as well.

So, this may need some more thinking to ensure that all cases are accurately considered, or just produce a "Paste As" dialog as done in similar cases to let the user pick the desired representation from the flavors offered (I know that clarkbw doesn't like that, but an explicit choice may be most useful rather than the fixed order of flavors currently probed on the clipboard).

Comment 16

[Thomas D. (:thomas8)]

adding some relevant search words to summary

Comment 17

[Thomas D. (:thomas8)]

Similar bugs for other UI parts:

Keyboard ways of copying/moving things using
- Ctrl+C, Ctrl+X, or
- context menu "Copy to" / "Move to":

Bug 512942 for messages (Ctrl+C, Ctrl+X only)
Bug 404955 for folders
Bug 339227 for address book items
Bug 335783 for attaching imgs/msgs/files/anything from clipboard (this bug)

Comment 18

[rsx11m]

Strictly speaking, this bug would only involve Ctrl+V as it is on the receiving end on whatever is on the clipboard. If so desired, copying an attachment back onto the clipboard using Ctrl+C (or copy+remove with Ctrl+X) would be separate bugs to introduce such mechanisms as they equally aren't supported by the backend yet.

Comment 19

[Jens Müller (:tessarakt)]

If it is ok that it might take some months, I'd be willing to take this.

Comment 20

[rsx11m]

Jens, thanks for looking into this further. Given that this bug has been pending for a couple of years, a few more months more certainly shouldn't be a problem.

Comment 21

[[:nickname]]

(In reply to rsx11m from comment #18)
> Strictly speaking, this bug would only involve Ctrl+V as it is on the
> receiving end on whatever is on the clipboard. If so desired, copying an
> attachment back onto the clipboard using Ctrl+C (or copy+remove with Ctrl+X)
> would be separate bugs to introduce such mechanisms as they equally aren't
> supported by the backend yet.

I agree; I don't know why I also wrote the CTRL+C/CTRL+X stuff back then. I think it's reasonable to clarify that the subject is only about the pasting option.

Jens: thanks for you interest, it's much appreciated.

Just to sum up things:
Pasting an image to a Thunderbird message which is a HTML message already works.

This bug is about creating text-only message and the effect what happens when a user tries to paste image-type data into it. In this case the image data from the clipboard should appear as an attachment to the message.

Does everyone agree with that? If so, I'd update the summary.

Sidenote: Microsoft Outlook solves this differently: when trying to paste an image to a text-only message, the user is prompted with a dialog to convert the text-only message to an HTML message. If the user accepts, the message is converted the image pasted inline. If the user declines, nothing happens.

Six years ago when I created this bug, I didn't specify this in detail as I was reminded later then pasting images to HTML messages in TB already works. Using Outlook on a daily basis too, this behavior is very practical. However, Outlook has another feature TB currently has not: to convert a message from/to text-only/HTML and back (with loss of information of course). But that's a separate bug.

Comment 22

[[:nickname]]

Actually, I was wrong on the conversion thing: TB is able to convert a message from/to text-only/HTML . My bad.

Comment 23

[Thomas D. (:thomas8)]

(In reply to Markus Fischer from comment #21)
> (In reply to rsx11m from comment #18)
> Jens: thanks for you interest, it's much appreciated.
> 
> Just to sum up things:
> Pasting an image to a Thunderbird message which is a HTML message already
> works.
> This bug is about creating text-only message and the effect what happens
> when a user tries to paste image-type data into it. In this case the image
> data from the clipboard should appear as an attachment to the message.
> Does everyone agree with that?

Not quite. Since comment 0 is lacking a lot of detail and structure (STR, Actual Results, Expected Results), this bug has evolved into something along the following lines (starting from Mike's comment 1):

STR:
- Copy (or cut?) "anything" (files; raw image data like screenshot; plaintext selection; etc.) from somewhere outside TB (from inside TB mostly needs new bugs).
- Right-click on attachment-pane, "Paste as attachment" (or Ctrl+V with focus in attachment pane, or respective options from menu)

Actual Result:
- n/a (TB does not offer that option)

Expected Result:
- "anything" (files, non-file selections) should be added as a file attachment in attachment pane (for non-file stuff: in an appropriate file format, e.g. .txt for plaintext selection; I'm not sure about the details for other types of raw data like image raw data)

We may or may not agree to deal with "pasting anything into body of plaintext message" here, but imho that looks like an added-value edge case that could and should be done in a separate bug; that's very similar to Bug 113435 (and could be bundled there).

This is a tentative description; we need to clarify desired UI and behaviour and to delimit what we can/want to do here or not. I strongly suspect that this is more like a meta bug (or perhaps focused on pasting image raw data as a file attachment into attachment pane) and we need to break out some pieces to be done in separate bugs.

Comment 24

[Thomas D. (:thomas8)]

(In reply to Thomas D. from comment #23)
> We may or may not agree to deal with "pasting anything into body of
> plaintext message" here, but imho that looks like an added-value edge case
> that could and should be done in a separate bug; that's very similar to Bug
> 113435 (and could be bundled there).

Grr, more accurately, that's "pasting 'anything' non-plain-text onto plaintext body, where 'anything' needs to or could be added as a file attachment". Per comment 6, that doesn't look like "the most intriguing usecase" for this bug, which is pasting image raw data (e.g. from screenshot) into attachment pane and autoconverting that into a real file attachment without need for interim saving as image file using image software.

Comment 25

[rsx11m]

Just to clarify: This shouldn't be restricted to plain-text messages, a user may equally want to attach content on the clipboard onto an HTML message where it also could be included inline (such as images or text). However, any ambiguities need to be taken care of, and the associated user action should be orthogonal between plain-text and HTML composition (e.g., adding some item "Attach from clipboard" to the context menu over the attachment pane). Having said that, always treating a paste action of non-text clipboard items as "attach" should be unambiguous given that there is no other place to paste into.

The broader aspect of relating specific user actions to content vs. attachment including content disposition should be deferred to bug 452092 but can certainly be kept in mind here.

Anyway, let's see what Jens comes up with in terms of a possible implementation.

Comment 26

[Thomas D. (:thomas8)]

Iirc, Jim (:squib) once had plans to do this as part of his attachment pane work.
Jim, can you comment on
- state of that plan, e.g.: Any WIP patches? Any time in near future to contribute here?
- your ideas for UI/behaviour
- pointers to code (for Jens who's willing to try this per comment 19)

Comment 27

[Thomas D. (:thomas8)]

(In reply to rsx11m from comment #25)
> Just to clarify: This shouldn't be restricted to plain-text messages, [snip]
> Having said that, always treating a paste action of non-text
> clipboard items as "attach" should be unambiguous given that there is no
> other place to paste into.
+1.

Whereas for paste action of *text* clipboard content into message body, expected behaviour is ambiguous, even when it's a whole text file, see x-ref bug 253017.

Comment 28

[TB-User]

Over here this has been a sorely missing feature for a long time. 

The most urgent need is for sending screenshots via email. Right now, I do a screenshot (PrtScn or Alt-PrtScn on Windows), start an image editor, paste the clipboard image into the image editor, save a temp file, close the editor, go to my email composition window and browse for the file as attachment, send the message and then delete the temp file. 

I really wish all this could be done with a simple Ctrl-V in the message composition window, maybe followed by a dialog whether I want to attach the data or whether I want to switch to HTML mode.

Sending HTML email messages by default just for this feature is not a good option IMHO.

Comment 29

[.]

+1. This should really be implemented finally for text messages as well (as an attachment then).

Comment 30

[maaaaaaaaaaaaaartin]

Any update on this? That would be so useful? Thanks! Martin

Comment 31

[Michael Ganß]

I've created an extension to provide this feature: https://addons.mozilla.org/thunderbird/addon/attach-from-clipboard/
Source: https://github.com/mganss/AttachFromClipboard

It can handle images, text, HTML, URLs, and files. Temporary files are created in system temp folder and removed when compose window is closed (message sent or saved).

Comment 32

[Thomas D. (:thomas8)]

(In reply to Michael Ganß from comment #31)
> I've created an extension to provide this feature:
> https://addons.mozilla.org/thunderbird/addon/attach-from-clipboard/
> Source: https://github.com/mganss/AttachFromClipboard

Wow! That's great news! Thank you Michael, and thanks for sharing!

Now we need somebody to get some inspiration from your code and land something similar to Thunderbird core... Anyone?

Comment 34

[Thomas D. (:thomas8)]

FYI