Open Bug 328717 Opened 18 years ago Updated 2 years ago

IMAP folder being cached locally although not told to do so, seems never to be cleaned up

Categories

(MailNews Core :: Backend, defect)

x86
All
defect

Tracking

(Not tracked)

REOPENED

People

(Reporter: andi, Unassigned)

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1

I have an IMAP Mailaccount set up as A. After reading mails, i move them to the local folder for later backup.

I was wondering about the big mail-profile on my hard disk. So i checked this and found a file INBOX (500 MB) in my imap-mail-profile-folder. ALL mails that ever got sent to my imap mail account had been cached in this file. Also those mails, that have been moved to the local folders.

So I checked preferences and found "Make messages available for offline use" and UNCHECKED it (it was checked but no folder was checked in the management dialog). Restart of thunderbird... nothing changed. compact folders... nothing changed. so i checked to delete messages older than 60 days (oldest of them are several months old)... restart... no changes.
Then i made sure that working offline will be set to YES for the IMAP Inbox (check folder, check offline use) and set thunderbird to go offline. thunderbird then asked (after going offline) whether to compact all folders or not. i said yes... and magic: the local cache was cleaned up.


So to sum all this up:
Preferences set:
Make available offline ... YES
manage folder for offline download: NONE set to yes.

Actual results:
Thunderbird cached data from an IMAP folder locally without being told to do so for this folder in special, but being told to cache locally in general.

Expected results:
No data being cached as no folders was checked for being download.

Reproducible: Always

Steps to Reproduce:




As data is being held locally, although being told not to do so, this could be a (high user) security risk! (minor urgency, major security --> normal severity)
Same problem with 1.5.0.4, Win2000
As i was asked to check again... 

The problem still occurs in Thnuderbird 2.0.0.16 (20080708)!

2 years unconfirmed? Could anyone else please reproduce verify this?

It is possible to reproduce by simply moving mails around in an imap account.
Move from any folder to Inbox and after that out of Inbox again. Check file size for caching file aftter every step. 

1000 Mails (can be found in every junk folder) are enough to move into Inbox and out again. After moving, try compact, restart, ... nothing happens. Just when checking "make avilable offline" for Inbox the file size decreases to comprehensible size. (Uh... i'm not really sure now whether this happened after checking or after unchecking the option for Inbox again).
As my last post seemed to be a little bit confusing, i try to describe it more detailled again.

Steps to reproduce the problem:

(Please note, that the mentioned file sizes just are examples to point out the problem. While reproducing, your observations may vary.)

01. Take any mail account

02. Setup IMAP connection in thnuderbird for this mail account

03. setup "account settings" / "offline & disk space" and set "make the messages in Inbox avaiable when i am working offline" to YES.

04. open "folder properties" for inbox (right click on inbox folder) and set "select this folder for offline use" ("offline" tab) to NO.

05. Look into Inbox folder... let's assume: 0 mails in there

06. Look into Junk folder... let's assume: 2000 mails in there

07. check file size on local disk for inbox folder: let's assume 10 kb.

08. move 1000 mails from junk to inbox folder

09. check file size on local disk for inbox folder: let's assume 10 mb.

10. delete all mails in inbox folder (then it's empty)

11. check file size on local disk for inbox folder: still 10 mb.

12. move rest of mails from junk to inbox folder

13. check file size on local disk for inbox folder: let's assume the before mentioned 10 mb + additional 10 mb = 20 mb.

14. delete all mails in inbox folder (then it's empty again)

15. check file size on local disk for inbox folder: still 20 mb.

16. select folder inbox and select command "compact folders" in thunderbird

17. check file size on local disk for inbox folder: still 20 mb.

18. restart thunderbird

19. check file size on local disk for inbox folder: still 20 mb.

20. open "folder properties" for inbox folder and set "select this folder for offline use" to YES and close properties

21. open "folder properties" for inbox folder and set "select this folder for offline use" to NO and close properties

22. check file size on local disk for inbox folder: only 10 kb.

Maybe step 21 is disbensable for reproducing the effect.

The problem seems, that thunderbird is caching mails for offline use although not told to do so. 
Maybe caching is enabled for performance reasons. Then, locally cached mails should be deleted from cache file at the latest when not existing online any more.
Assignee: mscott → nobody
Indeed, I have seen something similar with SeaMonkey on Linux and OS/2.
Status: UNCONFIRMED → NEW
Component: General → Backend
Ever confirmed: true
OS: Windows XP → All
Product: Thunderbird → MailNews Core
QA Contact: general → backend
Andreas, Peter can you still reproduce this using current release, or current trunk or alpha builds?
Summary: IMAP folder being cached locally seem never to be cleaned up → IMAP folder being cached locally although not told to do so, seems never to be cleaned up
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.16) Gecko/20101123 SeaMonkey/2.0.11 [I have newer SeaMonkeys installed on other machines, but don't use them for Mail.]

I still sort of see the problem, in the sense that the cache files like ImapMail/<account_dir>/INBOX are never removed or cut to size 0 once the the "download for offline use" checkboxes are removed. I had to remove them using system measures, and only then they got recreated with size 0.
Sorry for the delay...

For me, it's nearly the same as for Peter.

...

04a. open "folder properties" for inbox (right click on inbox folder) and set
"select this folder for offline use" ("offline" tab) to YES.

--> TB caches mails locally as expected


04b. now uncheck "offline use" from 04a

--> TB does not download NEW mails to the local INBOX file. => bug seems fixed.
--> But TB also does not empty the INBOX file when unchecking.


05a. Select "File > Compact Folders"

--> TB cuts the INBOX file to 0 bytes - nothing left.


So for me, the bug seems fixed - but there's another small bug: When unchecking "offline use", TB should empty the according local cache file (i.e. INBOX in the example above). Should we file a new issue for this and close this one?



PS. Just to mention: setting of step 03 in description above is now called: "Synchronisation & Storage", Section "Disc Space" and must be set to "Synchronize all messages locally regardless of age".
Oh boy... please don't tell me that's true?!
After checking the issue, I just realized that I had a 400 MB INBOX file for my common POP account! And only 4 messages in my Inbox. It seems that caching for IMAP is not a problem any more. But compacting folders seems to be a problem, now. Compact folders and magix: Inbox for POP account is only a few bytes now.

As far as I can remember, there was a setting to tell TB when it should try to compact folders. But I cannot find this setting any more - even in config editor. Please give me a hint so that I can check this. Thanks.
I do not get the same results as Andreas on step 5a.  It looks to me like thunderbird totally ignores the mail cache after unchecking "select this folder for offline use"; compact no longer affects it.  If I turn the checkbox back on, then compact correctly reduces the size of the cache by discarding mail older than 2 weeks ( I have the account setting set to only synchronize the last two weeks ).

When you disable synchronization, the cache should be cleared rather than ignored.
old issue, seems irrelevant today
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → INVALID
No, issue is very much still alive and well and relevant today in thunderbird 31.5, please reopen.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
(In reply to Phillip Susi from comment #9)
> It looks to me like thunderbird totally ignores the mail cache after unchecking "select this folder for offline use";
> compact no longer affects it.
> If I turn the checkbox back on, then compact correctly reduces the size of the cache by
> discarding mail older than 2 weeks ( I have the account setting set to only synchronize the last two weeks ).
> When you disable synchronization, the cache should be cleared rather than ignored.

"select this folder for offline use" !=== auto-sync of imap offline-Use=On folder.
"Offline use" is "Offline Use".
"select this folder for offline use" is used by ;
    (a) "Offline Use" : download mails into Offline-Store file upon "go Work Offline" in order to read mails while offline mode.
          Offline-Use=On folder : Automatically ask "download or not" when go Work Offline.
          Offline-Use=Off folder : Don't ask whether download or not when go Work Offline.
          In both cases, manual download to Offline-Store file is always possible via Folder Properties/Synchronization.
    (b) auto-sync of imap Offline-Use=On folder
          If auto-sync of this imap account is enabled, auto-sync is executed for Offline-Use=On folder only.
          Even when auto-sync of this imap account is enabled, if Offline-Use=Off, auto-sync is not executed, and (a) is applied.
   
(i) When auto-sync of this imap account is disabled, (a) is applied to any folder of this imap account.
     Offline-Store file is used only while Offline mode.
(ii) If auto-sync of this imap account is enabled, "select this folder for offline use" is used by auto-sync too.
     And, if Offline-use=On, mail is downloaded to Offline-Store file. Offline-Store file is used while Online mode and while Offline mode.
(iii) If auto-sync of this imap account is enabled, but "select this folder for offline use" is unchecked(Offline-Use=Off),
      auto-sync is not executed and (a) is applied, so Offline-Store file is used while Offline mode.

Default setting is for (ii), so user usually starts from (ii), and, when transfered to (iii), user opens bug like this...
Please note;
    "select this folder for offline use" !== auto-sync of imap offline-Use=On folder.
    "Offline use" is "Offline Use".

About "Compact Folder" in (i)/(ii)/(iii).
If IMAP, Compact consists of "send Expunge to server" + "compaction of Offline-Store file".
Sorry but I don't know about "compaction of Offline-Store file" when (i) and (iii).
   When (iii), from perspective of auto-sync, Offline-Store file should be deleted,
   However, from perspective of Offline-Use, Offline-Store file should not be deleted. It should be "compaction of Offline-Store file".
   When (i), "compaction of Offline-Store file" should be "remove deleted(flagged \Deleted) or purged(expunged) mail at server".
I am having a very hard time following what you wrote, but let me see if I got it.  You are saying that if I check "Keep messages for this account on this computer" in the account Synchronization and Storage settings, then under the advanced button, select a specific folder for offline use, then go offline and say yes to download, that folder's messages are downloaded to the local mbox.  At that point, if I understand you correctly, if I uncheck the folder, the local mbox should be deleted right?

If so, that is the problem: the local mbox isn't deleted.  It isn't used any more, but it still hangs around, and there doesn't seem to be any way to get rid of it short of deleting the file from the filesystem.
(In reply to Phillip Susi from comment #13)
> If so, that is the problem: the local mbox isn't deleted.

I can't follow you. "Offline-store file" is "Offline-store file".
What do you call by your "the local mbox"?
The downloaded email is stored in a unix mbox format file inside the thunderbird profile directory.  I assume that is what you are referring to by "Offline-store file".  i.e. C:\Documents and Settings\psusi.IRADIMED.000\AppData\Roaming\Thunderbird\Profiles\gv5pbjw1.default\ImapMail\iridev-svr1\some-folder.

It is this file that is filled up whenever you synchronize or download for offline use, and then there doesn't seem to be a way to make it go away when you no longer wish to keep the data offline.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.