Closed Bug 1566360 Opened 5 years ago Closed 5 years ago

move filter (for imap) moves, then locally (in the TB instance filtering) deletes all emails in local destination folder and then redownloads all messages

Categories

(MailNews Core :: Networking: IMAP, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: buecher, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

subscribe to all inbox and subfolders

Actual results:

on rightclick-> properties, keep this folder for offline is not checked.

Checking and clicking download now downloads 15000 messages

Expected results:

I had the list of emails (headers), didn't look at bodies for a long time. I thought subscribing also would download - at least it does that for the other folders, and in the account/sync, it is set to have copies locally for all folders of this account.

Looking at file-> subscribe, the folder is already subscribed.

it is a subfolder of inbox, forgot to mention that.

an additional nuisance is that the required email (from yesterday) will not be displayed/downloaded before the 15000 older emails are downloaded. That truely is a nuisance on landline and a disaster on mobile because of data transfer limits and waiting time.

six weeks ago, I did a sync of all emails using file->offline->download all, so the emails really should have been loaded already

(sorry for the information coming in small bits..)

now, after it seemed finished, it has started to (re-?) download 36000 headers, simultaneously, it downloads 5000 bodies for this folder.

The same folder on the business PC seems ok and stable (this happens on my notebook, wlan, same network as imap server)
This is totally confusing.

OS: Unspecified → Windows 10
Hardware: Unspecified → x86

Did this just recently happen?

What version of Thunderbird are you using?

I'm not seeing this behavior on 60.8 but I just created the folder within TB. It may behave differently if the folder already existed.

comment 2: My condolences to your data plan.

I'm going to do a little research to see if I can't find another bug with the same issue. And create a few folders on my IMAP servers to subscribe too.

Component: Untriaged → Networking: IMAP
Flags: needinfo?(buecher)
OS: Windows 10 → Unspecified
Product: Thunderbird → MailNews Core
Hardware: x86 → Unspecified
Summary: subscribed folder (imap) is not synced (bodies) → subscribed [sub]folder (imap) is not synced (bodies) [Only downloading headers]

it is 60.8.0. Happens just now, it it currently downloading 15260 bodies. First it was downloading ca. 15000 bodies. Two hours later, it decided to download 30000 headers, 5000 bodies. Now, 40 min later, it is downloading 15000 bodies. No mention of downloading any headers.

The actual message file in inbox must have vanished. On business PC, it is 2.5GB, 3 hours ago, it was 150 MB on notebook. The partition of profile has 250 GB free space.

I disconnected the desktop from network, just in case TB decides to tell the imap server to delete mails. So I have at least a copy on the desktop. I had similiar things happening a few month ago, where a problem in TB distributed faster to the imap server than I could disconnect network. Since then Win updated to 1903. I think that problem was related to a filter trying to move messages and failing (maybe because being offline?). Maybe it is related. I think there is a bugzilla entry for that. I can search for it if helping.
thanks,

Klaus

Flags: needinfo?(buecher)

this is now critical - TB has deleted all emails (15000) in this subfolder.

6 hours later, when I came back to my PC, TB reported it is downloading 300 bodies from this subfolder. I then clicked download messages. Nothing came into inbox, TB reported it is downloading message xx of 500 for this folder (all within 1 minute).
The folder name was in blue, so I clicked and saw ther are two new messages of ca. 2pm today. Next I went back to inbox (using mouse). When that was displayed, TB reported deleting 15000 messages in that subfolder. it is now empty in TB display. msf and 2.5GB of data still exist in the profile.

Using the mouse, I cannot reeally have deleted them with my actions because a 'mark all' and a delete would be needed, and I actually never did any like this with mouse only. At that time, I didn't touch the keyboard.

I stopped TB, disconnected from network, rebooted, reconnected and am online now writing this message. I will restart TB and see what it does with the folder - I stored the messages and msf on another PC to a folder outside of the profile, so that should be safe

Klaus.

When restarting TB offline, that folder was empty. After reconnecting, TB is now redownloading the 15000 messages in that subfolder. The bad news is that the same is now happening to another folder, where 4000 messages are being reloaded.

Please remember that all was synchronised for my vacation on June 10th.

Both are folders that are filled by filters copying emails from inbox.

I think the filter is deleting the messages.

activity window shows:

3593 messages deleted in folder xxx 22.01
1 email moved from inbox to folder xxx at 22.01 time

printed this order in acticvity window. What is first in time: delete or move?

The filter is ok, I just verified, it does just move, nothing else.

I can confirm from today's operation: per activity window, the very moment the filter copies the messages from inbox to inbox subfolder xxx, all messages in xxx are deleted.

That happens for both filters that I have. For now; I will disable them. Now, again, I have to download 19000 messages in two folders.

Summary: subscribed [sub]folder (imap) is not synced (bodies) [Only downloading headers] → move filter (for imap) moves, then locally (in the TB instance filtering) deletes all emails in local destination folder and then redownloads all messages

I renamed the bug because that is per activity window the actual problem.

Component: does it belong to imap or to filters?

Wow, I have no idea what could cause this that wouldn't affect others as well.

I'm not sure what the component is after the change in behavior let me ping a few people to see if maybe they have seen anything like it.

Flags: needinfo?(vseerror)
Flags: needinfo?(mkmelin+mozilla)
Flags: needinfo?(jorgk)

you wouldn't notice if the folder is small or if you are on LAN because redownload will be fast. I had 15000 messages in the folder where I noticed. From activity window, I know that it happens in other folders as well but never noticed before.

the local delete does not happen if the filter is applied manually. (then, the messages are already fully downloaded to inbox).

The local delete happens if filter is set to automatic when receiving messages (before junk). At least until 60.7.?, all worked fine.

It happens in several accounts in the same TB.

Just saw that a filter protocol is available, just switched that on to see tomorrow what it reports.

Depending on that, I will put the filter instead to my desktop TB to see whether the same hapens there.

The only change in addition to TB update is Win 10 update to 1903 (home), but I think that was still at TB 60.6?, so before this came up.

Perhaps I'm a poor reader but skimming a couple comments I don't get what this is about. But clearly "subscribe to all inbox and subfolders" isn't the only step in this process. Further, comment 0 doesn't mention filters but now the summary does. So I would ask to have a single comment that has clear STR, and add a bugzilla STR label to that comment.

Flags: needinfo?(vseerror)

I have trouble understanding what this is about. We can't fix: I did this, and then this, and then this happened, so I did this, and then this, and now there's a total mess.

Generally speaking, TB might be over-zealously redownload messages, so you should tell us when exactly how that happens.

The mix of folders synchronised for offline use and folders not synchronised for offline use complicates things tremendously. Some actions, like repairing a folder, also causes redownload. Our IMAP expert Gene is constantly facing bugs with download issues, here's a selection of the last ones received: bug 1566750 and bug 1566717.

Flags: needinfo?(mkmelin+mozilla)
Flags: needinfo?(jorgk)

observed: running an automated filter (on receiving emails) to copy a mail from inbox to subfolder xxx deletes all messages in folder xxx (only locally, not on imap server), then the messages are redownloaded from imap to that folder. TB gives this info in activity window.

So each time a single message is moved from inbox to xxx: TB deletes 15000 messages and redownloads them.

Expected behaviour: the message is moved, maybe redownloaded if move done on server, older messages are left as is.

maybe this bug should be closed as 'misinterpretation of real events' and a new opened?

Status: UNCONFIRMED → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID

after:
deactivating filter
activating filter

the filter no longer deletes all emails in the receiving folder.

See Also: → 1461194
You need to log in before you can comment on or make changes to this bug.