327302 - Forward of a message with detached attachment should attach the saved file.

Status: NEW

(MailNews Core :: Composition, defect)

Description

[Jean-Marie COUPRIE]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1

There are no visual clues that the attacment has been detached, you may think that it is still here because you can still open it, and forward the message to someone that does not receive the original attachment but something else. TB has not complained that you have done a wrong thing but the addresse will!

Reproducible: Always

Steps to Reproduce:
THE BUG
I have done the following test using 3 of my addresses ad1, ad2 and ad3.
1) ad1 sends to ad2 a small mail with the attached file AttachTest.txt (that just contains "File to test detached attachment."
2) ad2 receives and detaches the file. On the lower part of the screen, I see that there is an attached file AttachTest.txt, I click and open it and I see the original file content. Now I click on the forward button, add some text e.g. "forwarded", check that it seems to have "AttachTest.txt" attached and send.
If I try to detach a second time to the same folder, I have first a message saying that the file already exist and asking to confirm that I wish to replace, I confirm, a second message saying that it is not possible, check the name and try later! Stupid action from me but also stupid answer from TB! Fortunately no harm has been done.
3) ad3 receives and opens the attachment "AttachTest.txt" which contains now :
'The original MIME headers for this attachment are:
Content-Type: text/plain;
 name="AttachTest.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="AttachTest.txt"'
instead of the original text...


Actual Results:  
This is supposed to simulate the more realistic sequence : user2 has received a mail with attachment from user1, detached the attachment and, a week later having forgotten he has detached, checks the content of the attachment and forwards to user3 or try to detach again.
The same problem occurs with "open to edit (ctrl+E) then send" that send the "false" attachment as "Forward" (I hope that the solution is also the same).


Expected Results:  
WHAT SHOULD DO THUNDERBIRD ?
-write in the "sent" mailbox the message with the attachment detached : the user has manually detached it from the received mail and do not wish to have to do this again in the "sent" mailbox. 
-send exactly the same message as it would have sent if the attachment has not been detached. 
This need that TB has stored sufficient "file informations" to find again the attachment and check that it has not be modified (or erased or moved elsewhere) Else TB should issue a warning error message asking what to do.

What means 'sufficient "file informations" to check that the attachment has not be modified'?
There is a similar problem for the backup programs : is the file candidate to backup identical to the file of the same name already in the older backup? Normally they compare the time and date stamps and the size, some add a "checksum" on the content e.g. CRC32.
I think that path with full name and extension, date/time of last modification and size are sufficient but needed. (I have found same name, same time stamp but different sizes in the 2 versions of the pilot of my scanner for the Windows 9x and XP!). The use of CRC32 or equivalent should be discussed.

Some informations are already hidden in the mailbox in the lines :
'X-Mozilla-External-Attachment-URL: file:///E:/Temp/AttachTest.txt
X-Mozilla-Altered: AttachmentDetached; date="Sat Feb 11 18:26:04 2006" '
The first line is fine but the second is wrong :
-the time stamp disagree with the one in file system shown  by "explorer" : created 18 25 59, last modified 18 26 00. To allow comparison the last modified time of the file system should be recorded.
-the format of the date seems difficult to compare with the one in the file system and is country dependent. I propose to use the "ISO" or "sort" format YYYYMMDD HHmmss with or without separators : it is always and everywhere the same!
-size (and perhaps CRC) should be added.

TO SUMMARIZE : When detaching, TB should replace the attachment by its "file informations" (defined above) in the mailbox. When you try to forward, TB should re-attach the file only (not in the sent mailbox) into the sent message (suppressing in it the "file informations" no more needed and that may cause problems if user3 repeat the detach + forward process of user2) and issue a warning asking what to do if the file is not found or its "file informations" are not the same!
There is no problem with "reply" that does not send attachments except that the "file informations" added at detach time should be suppressed in the sent message.

FURTHER ENHANCEMENTS
Their implementations may be delayed but keep them in mind when correcting the bug : this should allow a more modular final coding.

-a new option : When sending, TB replaces directly the attachment by its "file informations" in the Sent folder. This is functionally equivalent to : normal sending with recording of the attachment in the Sent folder + detaching the attachment (with the replacement of the attached file in its folder) + cleaning the mailbox with "compact".

-There are in fact 2 ways to "detach" : choose "detach" in the menu or "copy" then some time latter "delete". TB should always record "file informations" when it copies. If afterward it "delete", this should be processed and shown as if only a "detach" has been chosen. If there is no "delete" the attachment in the message should take precedence on the one defined by "file informations". If the message is resent to somebody the "file informations" should be deleted in the resent message.

-improve the answer of TB to "detach again" (e.g. saying "Already detached" + the "file informations")and/or give to the user a visual clue that the attachment has already been detached.

Processor AMD64 3000+, OS : Windows XP SP2 with updates, Thunderbird 1.5.0.2 French version (when translating back to English what I have seen on my screen I may have missed the original word and replaced it by a synonym).
Be patient : this is my first entry in Bugzilla. Tell me what is wrong...
I'll be away of the net from February 17th to March 6th.

Comment 1

[Mike Cowperthwaite]

(In reply to comment #0)
> WHAT SHOULD DO THUNDERBIRD ?
> -write in the "sent" mailbox the message with the attachment detached : the
> user has manually detached it from the received mail and do not wish to have
> to do this again in the "sent" mailbox. 

You mean, automatically re-detach the item from the copy in Sent?  That would be a nice feature, but will need to be addressed in a separate bug.


> -send exactly the same message as it would have sent if the attachment has not
> been detached. 

I agree with this; that's the problem this bug is about.

The problem as described applies to inline forwarding.  If the message is forwarded As Attachment, the problem still applies, but may be harder to fix.

Comment 2

[Jean-Marie COUPRIE]

There are 3 parts in the correction of the bug I have reported.
1) add in the message in the mailbox information at the time the attacment is detached to be sure the file attached is the original file received : I have proposed time and date, size equal to the  file saved in Windows. You may add a CRC or equivalent. This part may be done separately but of course before 2 and 3.

2) send a message with the attacment reattached. It may be useful to have in it the information that the attachment has been detached and reattached. If the file is not the original an error message should be issued.

3) write in the "sent" mailbox a copy of the sent message.
- You propose to write it exactely as send and to leave to an other bug to redettacch it. Doing this you create a faulty non manageable situation as long as the other bug is not resolved : to avoid waste of disk space, the user will try either to manualy detach and overwrite the previously detached file and its time and date and the previously received message cannot be forwarded again, or   just delete the attachment in this case the just forwarded message cannot be forwarded again...
- I propose to write in the mailbox a message as is the attachment have been detached with the information of the location, date, time, size copied from the previously received message. You can do this only if 2 and 3 are fixed at the same time. Both the received message and the forwarded message can be forwarded again...

Please reconsider your decision to ask for 2 separated bugs for 2) and 3).

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Still an issue for Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.5pre) Gecko/20070627 Thunderbird/2.0.0.5pre Mnenhy/0.7.5.0 ID:2007062709

Comment 4

[rsx11m]

I think a major problem here is that the MIME type is retained in the attachment headers themselves when detaching. Forwarding inline, the
"X-Mozilla-External-Attachment-URL" header is removed, and only the
"The original MIME headers for this attachment are:..." if forwarded as
the attachment content. For the receiving e-mail client, this appears to
be a valid attachment, but the opening application will obviously fail.
Forwarding as attachment poses a similar issue, but here the file URL is
forwarded as well, which has a meaning at the local machine of the sender
but not at the receiving end.

When attachments are deleted, the original MIME type is replaced with "Content-Type: text/x-moz-deleted" to uniquely identify those. If a detached attachment would be labelled in a similar way (the original MIME type of the attachment is kept in the attachment body itself and thus is still available for opening it), this would avoid the ambiguity. Also, if the attachment
cannot be reattached, or the decision is not to do so, that attachment should be removed entirely. I see that this poses a problem for forwarding as attachment, but the general rule/intent to forward a message "as is" in its original form has been violated at this point already anyway.

Comment 6

[Davor Dundovic]

Still an issue in Eudora OSE v1.0 (Thunderbird v3.0 engine) and in Thunderbird v3.1.6.

Comment 7

[Jean-Marie COUPRIE]

In addition to my comment 2, I have checked the content of the mailbox : now the time and date of detach is included in the mailbox, the format of it is not the same that in my file manager but this may be translated from one to the other.
So now TB can check with a good approximation (except forgery of time/date stamp) that the file is still here and not modified and so can be reattached.

Comment 8

[Jean-Marie COUPRIE]

After 4 and 1/2 years, it is time to do something to correct this bug !
Unfortunately if I have some experience in programming, I know nothing in TB coding and cannot write a patch. All I can do is to contribute a block diagram...

First, I assume that the complete message to be forwarded and its attachment(s) are available as a variable or a temporary file that I call MSG for ease of discussion.

A) What are the part corresponding to an attachment ?
1) Attachment Deleted '
Content-Type: text/x-moz-deleted; name="Deleted: IMG_3599.JPG"
Content-Transfer-Encoding: 8bit
Content-Disposition: inline; filename="Deleted: IMG_3599.JPG"
X-Mozilla-Altered: AttachmentDeleted; date="Wed Oct 20 11:18:05 2010"

You deleted an attachment from this message. The original MIME headers for the attachment were:
Content-Type: image/jpeg; name="IMG_3599.JPG"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="IMG_3599.JPG"
'
2) Attachment Detached '
Content-Type: image/jpeg;
	name="P8270025.JPG"
Content-Disposition: attachment; filename="P8270025.JPG"
X-Mozilla-External-Attachment-URL: file:///X:/Yyyy/Zzzz/P8270025.JPG
X-Mozilla-Altered: AttachmentDetached; date="Tue Oct 19 12:38:25 2010"

You deleted an attachment from this message. The original MIME headers for the attachment were:
Content-Type: image/jpeg;
	name="P8270025.JPG"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="P8270025.JPG"
'
3) Attachment still here '
Content-Type: text/html; charset=ISO-8859-1;
 name="Xxxx.htm"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="Xxxx.htm"
' OR '
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
' OR '
Content-Type: image/jpeg;
	name="Xxxx.JPG"
Content-Transfer-Encoding: base64
Content-Description: Xxxx.JPG
Content-Disposition: attachment;
	filename="Xxxx.JPG"
'

B) What to do ?
1) MSG does not contain a line beginning with "X-Mozilla-Altered: Attachment"
There are no attachment or they are still here : forward as usual...
2) MSG (or its remaining part) contains a line beginning with "X-Mozilla-Altered: AttachmentDeleted;"
At least an attachment has been deleted, TB cannot do anything sensible automatically => ask for user instruction that may be : abort forwarding the message, add some explanation into the text of the message out of the forwarded part, continue as is but the addressee should have a clue that something has been deleted, attach a file that the sender know to be equivalent to the deleted one, etc. .
 Then TB should loop on the remaining part of MSG to see if there are other problems.
3) MSG (or its remaining part) contains a line beginning with "X-Mozilla-Altered: AttachmentDetached;"
At least an attachment has been detached, TB extract the date from this line and the filename with path from the line above ; Then TB ask to the OS file system if the file exist and if its date/time stamp is nearly equal (1 s. ?) ; If yes TB automatically re-attach the file (cleaning its old info in MSG), else TB cannot do anything sensible automatically => ask for user instruction that may be : abort forwarding the message, add some explanation into the text of the message out of the forwarded part, continue as is but the addressee should have a clue that something has been detached, etc. .
 Then TB should loop on the remaining part of MSG to see if there are other problems.
4) If there are no problem in the remaining part of MSG, ask for a last visual check and confirmation that the user still wish to forward.

I know that this seems simple on the paper but may be difficult to code !

Comment 9

[Jean-Marie COUPRIE]

Still nothing done !

Comment 10

[Johannes]

I do like the ideas of solving the bug, but wouldn't it be easier to focus on the main problem to get started?

(In reply to Mike Cowperthwaite from comment #1)
> (In reply to comment #0)

> > -send exactly the same message as it would have sent if the attachment has not
> > been detached. 
> 
> I agree with this; that's the problem this bug is about.


I think checking if the original file might be altered is not necessarily what Thunderbird has to do, I think the user is responsible how he works with detached attachments.