679619 - Deleting an attachment in Thunderbird creates a duplicate message WITH the attachment in Gmail

Status: NEW

(MailNews Core :: Networking: IMAP, defect)

Description

[M8R-qbf2b3]

Attachment: Screenshot of duplicates

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_4) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1

Steps to reproduce:

1. Added my Gmail account to Thunderbird.
2. Selected a message, went to the attachments, and selected "delete all...".
3. Clicked OK when the "cannot be undone" prompt was displayed.


Actual results:

Thunderbird showed that the attachments were deleted. BUT when I checked the messages again via Gmail's web interface, it showed:

1. The original e-mail, with a list of attachments that said "deleted: (filenames of attachment)"
2. A duplicate of the original e-mail, with all the attachments intact (see screenshot)


Expected results:

No duplicate e-mails or attachments, just the one e-mail with no attachments.

This happened with several e-mails, and I only realized it was happening when I saw my "Using x MB of your y MB" increasing in Gmail's web interface WHILE I was deleting attachments in Thunderbird, even though I wasn't receiving any new messages.

Comment 1

[Wayne Mery (:wsmwk)]

I have not researched, so I can't say if this bug is the same issue originally reported in http://getsatisfaction.com/mozilla_messaging/topics/_detaching_attachment_does_not_delete_original_in_gmail

Comment 2

[Roland Tanglao needinfo please :rolandtanglao, :mohnkuchen, :adobo, :sinigang, :roland]

I was able to reproduce using the steps in comment 0

Comment 3

[rsx11m]

This may be related to the unique implementation of Gmail's IMAP to treat Message-IDs as key across multiple folders (mapped from labels). When an attachment is deleted, a copy is created without the attachment and uploaded. Consequently it has the same Message-ID heading as the original message, the latter just being marked as deleted (which is not part of the Gmail model).

Thus, I tend to think this is more a limitation in the Gmail IMAP than by Thunderbird itself, given that duplicate messages and deleted status aren't supported by their implementation.

Comment 4

[M8R-qbf2b3]

Hi, Mr. Mery. Yes, it is the same issue; I'm the Katrina who commented.

(In reply to Wayne Mery (:wsmwk) from comment #1)
> I have not researched, so I can't say if this bug is the same issue
> originally reported in
> http://getsatisfaction.com/mozilla_messaging/topics/
> _detaching_attachment_does_not_delete_original_in_gmail

Comment 5

[WADA:World Anti-bad-Duping Agency]

Following is IMAP log with Tb 6.0 when "delete of attachment" is executed on mail of UID=2 which has flag of "$label4"(To Do) and "nonjunk.
For ease of testing, Gmail IMAP's auto-expunge option is disabled and Tb's delete model of "Just mark it as deleted" is selected.

> imap.gmail.com:S-B:SendData: 17 append "B" (\Seen $label4 nonjunk) "17-Dec-2010 18:31:39 +0900" {2211} 
> (snip)
>   --------------070301070105060708090100 
>   Content-Type: text/x-moz-deleted; name="Deleted: ECMA-357.pdf" 
>   Content-Transfer-Encoding: 8bit 
>   Content-Disposition: inline; filename="Deleted: ECMA-357.pdf" 
>   X-Mozilla-Altered: AttachmentDeleted; date="Tue Sep 13 13:20:39 2011" 
>    
>   You deleted an attachment from this message. The original MIME headers for the attachment were: 
>   Content-Type: application/pdf; 
>    name="ECMA-357.pdf" 
>   Content-Transfer-Encoding: base64 
>   Content-Disposition: attachment; 
>    filename="ECMA-357.pdf" 
>   --------------070301070105060708090100 
>
> imap.gmail.com:S-B:CreateNewLineFromSocket: 17 OK [APPENDUID 602313540 8] (Success) 
>
> imap.gmail.com:S-B:SendData: 21 UID fetch 8:* (FLAGS) 
> imap.gmail.com:S-B:CreateNewLineFromSocket: * 7 FETCH (UID 8 FLAGS ($label4 NonJunk \Seen)) 
> imap.gmail.com:S-B:CreateNewLineFromSocket: 21 OK Success 
>
> imap.gmail.com:S-B:SendData: 22 UID fetch 8 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type received)]) 
>
> imap.gmail.com:S-B:SendData: 23 uid store 2 +Flags (\Seen \Deleted) 
> imap.gmail.com:S-B:CreateNewLineFromSocket: * 1 FETCH (FLAGS (\Seen \Deleted $label4 NonJunk) UID 2) 
> imap.gmail.com:S-B:CreateNewLineFromSocket: 23 OK Success 
>
> imap.gmail.com:S-B:SendData: 24 UID fetch 8 (UID RFC822.SIZE BODY[]) 
> imap.gmail.com:S-B:CreateNewLineFromSocket: * 7 FETCH (UID 8 RFC822.SIZE 2211 BODY[] {2211} 

Tb appeded new version(UID=8), stored required flags to UID=8, then stored \Deleted flag to old version(UID=2), as expected.
After it, next is seen:
  Folder B :
    UID=2 : Shown with strike-thru line, with trash can icon (\Deleted is kept) 
    UID=8 : Attachment part is text/x-moz-deleted, as expected.
  [Gmail]/All Mail :
    UID=MM : Original mail of UID=2 in folder B (single copy of a mail)
             This is mail before attachment delete
    UID=NN : Original mail of UID=8 in folder B (single copy of a mail)
             This is mail after attachment delete

i.e.
Gmail kept two mails of same mail headers(message-id, subject, from, ... are same) except different headers and data in some parts under multipart/mixed.

At folder B, "Mail before attachment delete"(UID=2) is kept with \Deleted flag as expected with auto-expunge=Off. The UID=2 is removed from folder B by "Compact"(expunge is requested). By expunge, Gmail Label of B is removed from mail of "UID=MM in All Mail".

At Gmail Web Interface, UID=MM(has Gmail Label of "B") and UID=NN(has Gmail Label of "B") are shown as single conversation(the conversation contains 2 mails).
And, as "Mail before attachment delete"(UID=MM) was uploaded to or arrived at Gmail earlier, older "Mail before attachment delete"(UID=MM) is shown.
If "Expand All" is requested, "Mail before attachment deleted"(UID=MM) and "Mail after attachment delete"(UID=NN) are shown in a conversation.

Above is same even after "removal of Gmail Label of B from mail of UID=MM by Compact=expunge at folder B via IMAP".
Gmail's mail showing at Web Interface looks "Conversation basis", intead of "mail basis".
Gmail's logic to "show mails labeled XXX" at Web Interface seems;
  Show conversation in which one of member mails has Gmail Label of XXX.
Please note that addition of Gmail Label/removal of Gmail Label at Gmail's Web Interface is possible on "Conversation" only. 

I couldn't observe phenomenon of "Deleting an attachment in Thunderbird creates a duplicate message WITH the attachment in Gmail" written as bug summary.
I could see next phenomenon only;
  Gmail keeps two versions in All Mail after Tb's detach/delete operation;
  - mail before detach/delete by Tb, \Deleted flag is stored  at a mail folder
  - mail after detach/delete by Tb, appended with modified part
Phenomenon is merely combination of (a) "Gmail simply removes Gmail Label from a mail by store flag \Deleted via IMAP", (b) Gmail never removes mail from "All Mail"([Gmail]/All Mail if via Gmail IMAP) until copy/move of a mail to [Gmail]/Trash or [Gmail]/Spam is requested, (c) Gmail's Web Interface is "Converstion basis", is never "mail basis", (d) Gmail IMAP is mail basis.
All of them is "Gmail's design".

Workaround by mailer:
Drafts([Gmail]/Drafts) has same problem. However, Gmail looks to have implemented Gmail side workaround for same problem on old draft mail(deleted old version).
  If \Draft flag is set, automatically move mail flagged \Deleted
  to Trash([Gmail]/Trash via IMAP). See Bug 562748.
There is no such workaround for "original mail of detach/delete by Tb".
Poosible workaround by Tb is;
  Issue "uid copy XXX [Gmail]/Trash" instead of "uid XX store flag \Deleted"
  for old version of detach/delete, if Gmail IMAP.
"Deletion of UID=XXX and corresponding mails at mail folders icluding [Gmail]/All Mail by Gmail" will be notified via IDLE sooner or later, or Tb can know it sooner or later upon next folder access. 
If a mail is copied/moved to Trash([Gmail]/Trash via IMAP), the mail is removed from "Conversation at Gmail" by Gmail.

User's workaround of Gmail's particularity.
(1) Move or copy original mail to [Gmail]/Trash.
(2) Execute detach/delete on a mail at [Gmail]/Trash within 30 days.
(3) Move new version(detached/deleted) to a folder you want within 30 days.
    Mail will appear in [Gmail]/All Mail sooner or later.
(4) If you need to purge old version in [Gmail]/Trash immediately,
    Shift+Delete(\Deleted is stored), then Compact(expunge is requested).

Comment 6

[Roland Tanglao needinfo please :rolandtanglao, :mohnkuchen, :adobo, :sinigang, :roland]

(retested as part of the June 12, 2013 Thunderbird test day:
https://wiki.mozilla.org/Thunderbird:QA_TestDay:2013-06-12
)

this bug can still be reproduced using the steps from comment 0 in thunderbird 24 nightly from June 12, 2013

Comment 7

[WADA:World Anti-bad-Duping Agency]

In addition to Comment #5.

If Gmail considers that "detached/deleted mail" is same mail as "original mail", similar phenomenon to bug 550407 occurs.
Because same Message-Id:, same Subject:, same From:, ..., when "detached/deleted mal", and because difference is a part of mail payload only, Gmail may consider "same".

Decision on "same or not"(assign different X-GM-MSGID, or not) is all up to Gmail.
  1. Full mail (N lines)
  2. Sub set or Super set of message headers + full mail body
     (bug 550407)
  3. full message headers + sub set of mail body (this bug)
  4. Partial mail (error during upload. upload again)
     mail of line 1 to line X, where X = 2 to N
Decision on "same or not" by Gmail may depend also on auto-expunge=On/Off.
Because action by Gmail/Gmail IMAP on "stored \Deleted flag" is different between auto-expunge=On and auto-expunge=Off, externally observed phenomenon may depend on auto-expunge=On/Off.

Comment 8

[arctic.donkey]

I can reproduce this with an non gmail IMAP Account. Even Thunderbird shows the duplicate mails.

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to arctic.donkey from comment #8)
> I can reproduce this with an non gmail IMAP Account. Even Thunderbird shows the duplicate mails.

This bug is for Gmail IMAP specific issue. This bug is not for generic "detach/delete doesn't work as you want".
Flow of Detach/Delete is pretty simple.
If imap,
  0. Call original mail-x-A, UID=A.
  1. Create mail data for mail-x-B with attachment Detached/Deleted
      Upload the data for mail-x-B to imap server (append). Call UID=B
  2. Delete old mail-x-A, UID=A (uid A store +Flags \Deleted)
  That's all.
Because it's merely an "upload of a mail" + "delete of other mail", all is usually up to server.
Get imap log and look log file content and check imap level flow by yourself.
Because of same Message-ID:, same Subject:, same message header, same ..., except Deteched/Deleted part, server may confuse.

After it, search B.M.O well for already reported bugs, and open new bug for your problem if and only if actually new problem in Tb, with attaching sufficient data for problem anlysis, please.

Comment 10

[[e-mail]]

Is there a work around for this problem?

Comment 11

[Hans Martin Galliker]

Unfortunately this problem still persists.

User agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0

Comment 12

[justin.salamon]

Any updates on this?

Still experiencing this issue: using Thunderbird to delete attachments in a gmail account modifies the original email replacing the attachments with a "Deleted: <filename>" placeholder, but also creates a duplicate of the original email with the original attachments still present, the overall outcome being that the attachments are not deleted.

System: Thunderbird 60.6.1 (64-bit) running on macOS Mojave 10.14.3

Steps to reproduce:

• Use Thunderbird to open an email with an attachment from a gmail account (IMAP)
• In the attachment bar at the bottom, click on down facing arrow and then select "Delete" to delete the attachment
• Search for the same email in gmail and open it
• The original message will have attachments replaced with "Deleted: <filename>" placeholders, but a duplicate of the message containing the original full-size attachments will be created on the same message thread in gmail.

Thanks!

Comment 13

[Emre]

For those that land here, and want to make the batch cleanups a bit faster, here is a workaround that at least helps with the manual deletion:

• In Gmail settings make sure "conversation view" is turned off so Gmail doesn't who threads
• In Gmail label the email that you want to detach/delete the attachment of with both "detach" and "detached" labels
• In Thunderbird open the detach folder, perform the action you want.
• In Gmail all duplicates with original attachments will now only be in the detached label, where you can select all and delete it

Hope this helps to someone 🤷‍♂️

Comment 14

[batterystarter]

+1 maybe this can be addressed, for the 12th anniversary of this , still, open bug?

Comment 15

[Arne Brüning]

I have the same problem, but not with Gmail but with Outlook.com. When I delete the attachment from the mail in the folder, everything works fine.
But I have a search folder 'Big-Mails' where I search all mails from a given size. The idea is to find all mails with big attachments and delete the attachments from the search folder. But when I do this from the search folder, it results in 2 Emails. One with the attachment and one with the deleted attachment.
Navigating to the folder where the mail is stored and deleting the attachment from there works fine, but is a bit cumbersome.
But I have no idea, if this is the same problem, leaving this to the experts.