<a class="header-button" href="https://bugzilla.mozilla.org/home" title="Go to home page"> Bugzilla

Reporter

Comment 2

•

7 years ago

Yes, using IMAP. I don't have another IMAP account or mail provider to test it with. Also, it doesn't happen every single time that I delete a bunch of messages in quick succession, just some of the time. (I could set up another account or another mail provider, but the problem is that it only reproduces some of the time, and this would would only be reproducible with an account that regularly gets lots of messages over a period of several hours. So I've seen it happen a couple of times "organically" but it might be hard to reproduce with a test account.)

Flags: needinfo?(bennett)

Updated

•

7 years ago

Status: UNCONFIRMED → RESOLVED

Closed: 7 years ago

Resolution: --- → DUPLICATE

Reporter

Comment 4

•

7 years ago

Thanks. I assume it has the same root cause as the older bug 1293091. However I think my description indicates the problem is more serious than previously believed, since: 1) Older bug 1293091 referred only to permanently deleting messages in Trash. The current bug is about deleting messages in the Inbox (and it probably applies to all labels/folders across the board, although I've only seen it with the Inbox). 2) Browsing Trash and permanently deleting messages is a far less common use case than deleting messages in the Inbox. 3) Accidentally deleting a message in the Inbox is a bigger deal than permanently deleting a message that's already in Trash. (At least if a message is in Trash, that means at one point you decided it wasn't important. If you accidentally delete a message in the Inbox, it might not ever flash on the screen long enough for you to see what it is.) 4) Bug 1293091 is two years old. However I only just recently (in the last few weeks) just started observing the behavior of already-deleted messages re-appearing in the Inbox. Possibly something changed that caused the scope of the bug to widen. If you agree that the current bug describes the seriousness of the problem better than the older bug does, what should I do? Leave this bug resolved as duplicate, and enter the problem description into the comments for the old bug?

Updated

•

7 years ago

Status: RESOLVED → UNCONFIRMED

Resolution: DUPLICATE → ---

Updated

•

7 years ago

Status: UNCONFIRMED → NEW

Ever confirmed: true

Comment 6

•

7 years ago

OK, let's dupe the other one here, but that still doesn't fix it.

Reporter

Comment 7

•

7 years ago

OK let me know if I can do anything else to help narrow it down.

Reporter

Comment 8

•

6 years ago

Is there a way to tell if a bug like this one is in the pipeline to be fixed, or where it's sitting in the pile of priorities? It was filed four months ago as a bug causing data loss (unintentional loss of inbox messages) and I've been trying to re-train my muscle memory to get used to this weird behavior, so that I always wait for Thunderbird to "catch up" with me before deleting more messages so I don't accidentally delete any others, but it doesn't work. I'll need to migrate everything over to a different email program if this isn't going to be fixed. This worked correctly for several years, up to about version 60.

Comment 9

•

6 years ago

Trying to duplicate the bug by deleting some non-Inbox messages. Reporter, can I assume you have tb configured to "move deleted messages to trash"? Also, can you tell me the imap server you are connecting to?

I haven't yet duplicated the problem and I've never seen it, probably because when I clean out unwanted message I just highlight the unwanted ones with ctrl-leftclick and then delete the group with context menu delete or the delete button and I don't use the delete key (there is none on chromebook I'm using now but found that search-backspace acts as delete).

Anyhow, I will try some more to create the problem but I may need to ask you to produce and attach an imap log that shows what's happening when you delete several messages and the problem occurs (google: thunderbird imap log). If you need some junk messages to delete I can forward you some :).

Comment 10

•

6 years ago

Bug 1517918, that I can now duplicate randomly (quickly delete with delete key 10 messages), seems very similar to this bug.

Also, reading again, I see your imap server is gmail.

I'm not sure of the official definition of "dataloss" but, assuming the accidentally deleted messages end up in Trash, they are not actually "lost" but just "misplaced".

Reporter

Comment 11

•

6 years ago

OK, let me know if you need an imap log. I don't need any forwarded junk messages since I get plenty myself :) I will go through the imap log and try to redact any sensitive stuff before posting the log.

Comment 12

•

6 years ago

Reporter Bennett, you didn't answer if you are moving deleted msgs to trash, but I assume you are since that's default. Are the messages you accidentally delete in Trash?

Yes, if you can duplicate the problems while recording an IMAP:5 log that would be great (attach imap.log with the Attach File link above).

I haven't been able to duplicate the problem but I have noticed that tb sees a strange syntax error that causes a server disconnect (tb initiated) on the response to the imap MOVE command when it reports COPYUID info in the OK response. I saw this on a non-gmail server but I haven't checked it on gmail. The response looks fine to me but tb is rejecting it for a yet to be determined reason. The messages are still moved to the trash mailbox but I don't see this causing what you observer.

Also, when you say the messages come back for a few seconds, you mean in the "threads" list for Inbox (not in the reading area), right? Then they go away again from the threads list? I will keep trying to duplicate this but, so far, no luck.

Comment 13

•

6 years ago

(In reply to gene smith from comment #12)

I haven't been able to duplicate the problem but I have noticed that tb sees a strange syntax error that causes a server disconnect (tb initiated) on the response to the imap MOVE command when it reports COPYUID info in the OK response. I saw this on a non-gmail server but I haven't checked it on gmail. The response looks fine to me but tb is rejecting it for a yet to be determined reason. The messages are still moved to the trash mailbox but I don't see this causing what you observe.

This turned out to be a non-gmail imap server producing an incorrect message format. Gmail and others produces a correct response that follows the specs and tb likes.

Still no luck in duplicating the problem. Maybe it is due to a special tb setup you have. Can you tell me any non-default setup you have have, e.g., no offline storage for mailboxes etc. Also, any possible "hidden prefs" you may have set or any add-ons runnung? (If have addons, it's always suggested to try running tb in safe mode.)

If you create the imap.log, I don't know if it is documented but when you restart tb the log is overwritten; I've seen complaints about this.

Comment 14

•

6 years ago

Also, do you have any non-default imap settings for your gmail server account at the gmail site? One that comes to mind is "don't expunge emails that are marked as deleted" which defaults to "expunge emails that are marked deleted" which is how I have it set.

I think with default gmail settings, emails are never lost due to deleting in tb unless you go into "All Mail" folder and delete it from there and it has been removed from all other folders (really labels). But this always confuses me.

How many emails are in Inbox where you do the deletes? With many 1000's emails when tb does a fetch of flags it can take a long time. But I don't see it doing this fetch after each delete or group of deletes if you are fast at selecting and deleting. There is a disabled by default imap feature called CONDSTORE that is supposed to reduce the flag fetches. If you can duplicate the problem you might try switching it on temporarily using the Config Editor (hidden pref editor About:Config for tb) to see if it helps. Then switch it off since it can causes other problems.

Reporter

Comment 15

•

6 years ago

Yes the option is selected for deleted messages to be moved to the Trash.

I could not find the option you referred to, "don't expunge emails that are marked as deleted".

My Inbox usually includes a few dozen messages (not hundreds) when I do the deletes.

Comment 16

•

6 years ago

Attached image gmail-defaults.png — Details

(In reply to Bennett Haselton from comment #15)

I could not find the option you referred to, "don't expunge emails that are marked as deleted".

See the attached picture that shows my setup. I was referring to the gmail imap/pop configuration page at gmail.com, not within tb.

My Inbox usually includes a few dozen messages (not hundreds) when I do the deletes.

I've been testing with 100's of message in mailbox. Will try with much fewer. (That will also be more like Jorg's video he referenced in the other bug.)

Comment 17

•

6 years ago

One more request. I have been assuming that your gmail account works the same as mine when a message is deleted. However, it may be possible that where you are located or the gmail server you connect to doesn't support the MOVE capability. You can check this by recording the imap log and look to see if there are lines like this happening when you delete messages:

[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:SendData: 20 uid move 8671,8699:8700 "[Gmail]/Trash"
[23455:Unnamed thread 0x7aaa1c130820]: D/IMAP ReadNextLine [stream=0x7aaa1c172920 nb=16 needmore=0]
[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:CreateNewLineFromSocket: * 7728 EXPUNGE
[23455:Unnamed thread 0x7aaa1c130820]: D/IMAP ReadNextLine [stream=0x7aaa1c172920 nb=16 needmore=0]
[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:CreateNewLineFromSocket: * 7744 EXPUNGE
[23455:Unnamed thread 0x7aaa1c130820]: D/IMAP ReadNextLine [stream=0x7aaa1c172920 nb=16 needmore=0]
[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:CreateNewLineFromSocket: * 7744 EXPUNGE
[23455:Unnamed thread 0x7aaa1c130820]: D/IMAP ReadNextLine [stream=0x7aaa1c172920 nb=15 needmore=0]
[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:CreateNewLineFromSocket: * 7762 EXISTS
[23455:Unnamed thread 0x7aaa1c130820]: D/IMAP ReadNextLine [stream=0x7aaa1c172920 nb=59 needmore=0]
[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:S-INBOX/boost/boost-dev:CreateNewLineFromSocket: 20 OK [COPYUID 6 8671,8699:8700 4022:4023,4021] (Success)

In this case I just selected 3 messages and clicked the delete button. (Also, I didn't include the timestamp log item.)

Another thing to check in the log is the capability response from gmail. It should include MOVE capability and be about the same as mine:

[23455:Unnamed thread 0x7aaa1c130820]: I/IMAP 0x7aaa1d757800:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584

Note: You may also see some earlier gmail capability responses that are different that occur before login is complete and don't contain MOVE since full capabilities are not advertised until you are authenticated.

Thanks again.

Reporter

Comment 18

•

6 years ago

(In reply to gene smith from comment #16)

Created attachment 9041844 [details]
gmail-defaults.png

(In reply to Bennett Haselton from comment #15)

I could not find the option you referred to, "don't expunge emails that are marked as deleted".

See the attached picture that shows my setup. I was referring to the gmail imap/pop configuration page at gmail.com, not within tb.

OK. I have the same options selected in IMAP settings on Gmail as the ones shown in your image.

Comment 19

•

6 years ago

Still haven't duplicated the reporter's problem but have seen other anomalies such as this:

In gmail, start with trash empty and 19 messages in folder f1.
Using delete key, delete all 19 messages in f1.
Select trash and it contains 20 messages!
Look at Activity Manager and see 3 items (latest on top):

Deleted 3 messages from Trash 4:35pm
gd.smit@gmail.com

Copied 3 msgs from Trash to IBox 4:35pm
from gd.smth@gmail.com to Local Folders

Moved 19 messages from f1 to Trash 4:35pm
gd.smth@gmail.com

In log I see the 19 moves occur OK. Then I see Trash selected and gmail claims 23 messages exist there. Then I see tb download the body of 3 messages and then mark them with \deleted flag. Since this is imap log, I don't see the copy to Local Folder folder IBox, but it does seem to occur.

I look in Local Folders/Ibox and there are the 3 messages copied there. I have no idea why tb decides to copy stuff to Local Folders from Trash. I don't have IBox as a destination folder configured in my gmail account. Also, why does Trash now contain 20 messages when I deleted 19 from f1? I don't see any duplicates in Trash.

(At some time in the past I created LocalFolder/IBox as a test but don't remember why.)

One more thing: While deleting messages in f1, at one point it seemed that one message didn't delete when I hit the delete key combo. It may have come back like the reporter sees, not sure. Didn't see it go away on its own, but maybe I should have waited longer.

Will keep looking...

Comment 20

•

6 years ago

One more observation (don't know if helpful for this bug): In f1 had 20 messages uid from 98-117 and sorted by data order for display. Delete each (bottom to top) with delete key. Act Mgr says I moved 20 msgs to Trash. Looked in Trash and it showed 19 msgs. Looked back in f1 and it appeared empty. Log showed that uid 99 was still present in f1, marked with \Seen and not marked with \Deleted but did not show.
Opened another tb instance on another computer and could see message UID 99 in f1.
Looking at log a bit closer I could see that all the UIDs were moved to Trash except for 99.

Comment 21

•

6 years ago

Another observation: If you "uid copy" a message from gmail/Trash, the message is expunged from Trash -- does not remain in trash. It acts just like a "uid move"! Gimap feature, not tb's fault.

I think the main problem I am seeing is due the copy to Local Folder/Ibox. I think at one point, as a test, I set L.F./Ibox as the trash destination and somehow it is sticking around in the folder configurations. I found that if I "repair" folder f1 I no longer see the weird events for folder f1.

However, now I am using folder f2 and I see weird events for it. I am seeing messages moved to Trash and L.F./Ibox and then Trash is spontaneously expunged (by gmail) and all the messages appear gone (or maybe still in "All Mail" not sure. These are just test messages I found from an old mailing list archive and are nothing important.

Anyhow, this is a different than this big and should probably be a new bug.

BMO Automation

Updated

•

3 years ago

Severity: normal → S3

Wayne Mery (:wsmwk)

Comment 22

•

2 years ago

Does this still reproduce?

Flags: needinfo?(gds)

Flags: needinfo?(bennett)

Comment 23

•

2 years ago

I was never able to duplicate the bug as described. Haven't heard of any other complaints about this on bugzilla. So will wait for the original reporter to respond before looking into this.

Flags: needinfo?(gds)