539494 - IMAP Offline Store Should be Stored in "Local Settings" Application Data Folder (IMAP Cache: IMAP offline-store file by offline-use=on, never Disk Cache)

Status: RESOLVED DUPLICATE of bug 517425

(Thunderbird :: OS Integration, defect)

Description

[Daniel Morante]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 FirePHP/0.3
Build Identifier: 

The IMAP mail and folder synchronization cache that gets downloaded when using Thunderbird 3.0 should be stored (by default) in.

"%profile%/Local Settings/Application Data/Thunderbird"

The current setup stores the cache in: "%profile%/Application Data/Thunderbird", which (when using roaming user profiles in Active Directory) gets copied back and forth between the server and desktops.  Not only does this create an unwanted waste of disk space, but also slows down the log in and log off process significantly when the cache is very large (5GB for example).

Reproducible: Always

Steps to Reproduce:
1. Upgrade to or Install Mozilla Thunderbird 3.0
2. Accept Defaults
3. Sign on to IMAP mail server
4. (mozilla thunderbird begins to download all mail from server for offline use)
5. User logs off
Actual Results:  
Cache is copied to server because it is stored in the roaming Application Data folder, if cache is large the operation takes several minutes, maybe hours.

Expected Results:  
Cache should not be copied to server.

This issue focuses on IMAP accounts, I'm not sure if POP3 accounts are affected.  The current work around is to disable synchronization.

Comment 1

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> The IMAP mail and folder synchronization cache (snip)

What do you mean by "folder synchronization cache"?
Please distinguish "offline store"(Folder Properties, Synchronization) and Disk Cache(Tools/Options/Advanced/Network&Disk Space, Disk space).

> Thunderbird 3.0 should be stored (by default) in.
> "%profile%/Local Settings/Application Data/Thunderbird"

Set next in your prefs.js(via config editpr etc.), restart Tb, click a mail folder.
> mailnews.start_page.url;about:cache?device=disk
Tb3 on Win-XP displayes next for Disk Cache.
> Cache Directory: C:\Documents and Settings\wada\Local Settings\Application Data\Thunderbird\Profiles\wxkq5msh.Gmail-IMAP\Cache
What is displayed in your environment?

> Actual Results:  
> Cache is copied to server because it is stored in the roaming Application Data
> folder, if cache is large the operation takes several minutes, maybe hours.
> Expected Results:  
> Cache should not be copied to server.

What do you mean by "copied to server"? Is "roaming Application Data folder" located at network server?
"offline-store for auto-sync" is held in directory specified by you at Account Settings/Server Settings, Local Directory:.
If mail data should always be held locally, you need to specify local directory for it.

Comment 2

[Daniel Morante]

(In reply to comment #1)

This is regarding the "offline store".  Consider the following example on a new installation of Windows XP SP3, Thunderbird 3.0, and a new IMAP account setup.

The "Cache Directory" is:

C:\Documents and Settings\Administrator\Local Settings\Application Data\Thunderbird\Profiles\dvn7cd4b.default\Cache (OK)

The "Local Directory" is: 

C:\Documents and Settings\Administrator\Application Data\Thunderbird\Profiles\dvn7cd4b.default\ImapMail\mail.domain.net (OK)

The "offline-store for auto-sync" is:

C:\Documents and Settings\Administrator\Application Data\Thunderbird\Profiles\dvn7cd4b.default\ImapMail\mail.domain.net (BAD)

Why Bad?...

Windows Active Directory "Roaming User Profiles" is the feature allowing users to sign on to any computer and have his/her personal files and settings applied.  When the user signs on, *special* folders are recursively downloaded to the computer from the server.  When signing off, the same folders are recursively uploaded from the computer to the server.

The "C:\Documents and Settings\Administrator\Application Data\" folder is a *special* folder, and as a result "C:\Documents and Settings\Administrator\Application Data\Thunderbird\Profiles\dvn7cd4b.default\ImapMail\mail.domain.net" gets copied back and forth upon login and logoff.

To prevent this, the "offline-store for auto-sync" should be:

C:\Documents and Settings\Administrator\Local Settings\Application Data\Thunderbird\Profiles\dvn7cd4b.default\OfflineCache\mail.domain.net

Because "C:\Documents and Settings\Administrator\Local Settings\" is NOT a *special* folder that will get copied to and from the server.

Comment 3

[WADA:World Anti-bad-Duping Agency]

"C:\...\Thunderbird\Profiles\dvn7cd4b.default\ImapMail\mail.domain.net" is directory pointed by Server Setting/Local Directory: of each account settings.
You can change location by one of next:
1. Move Tb's profile directory to appropriate place.
>  Move current profile directory to C:\<any_path_in_local_drive>\<any_directory_name>
>  Create new profile with "Choose Folder" 
2. Move mail directory of Tb to appropriate place.
>  Server Settings/Local Directory:(after change, restart is needed due to bug)

Another choice: Don't use auto-sync for any IMAP account.

Comment 4

[mstraszak]

I stumbled across this bug too some time ago. I agree with Daniel Morante - these indexes and local copies of IMAP mails should be stored in Local Settings _by default_ (I know I can change the location, but I will not do it for hundreds of users). Even if I turn the synchronization off, indexes themselves can take enough space to make some trouble.

Comment 5

[Charles]

Yes, it can be changed manually, but the point the OP is making is the *default* location for *all* IMAP accounts should always be the Local Settings folder, and I agree.

Comment 6

[Charles]

Now... there is one problem with this... when you log in with a different PC, since the .msf files will no longer be included in the roaming profile, you will always lose any changes you make to the message pane columns/view, since these are now stored in the .msf file.

Guess its time to go open an enhancement request to split these off from the indexes, which are also contained in the .msf files, so the *config* settings can be stored in the main %AppData% profile folder.

Hmmm... even better - I wonder if  there is a way to store these on the IMAP server itself? t should be possible.

Comment 7

[Ken Alker]

I have been using the Mulberry email cient for years, and store all my config settings, windows sizes, columns/view, etc. on the a server via the IMSP protocol.  I'm told IMSP was replaced with ACAP (which may also be a dead protocol now), but IMSP still works great for me.  I used to run the IMSP daemon on the same server as our IMAP daemon, but now run it on a totally different server after moving from Cyrus to Zimbra.

Mulberry also stores its address book via IMSP.  In this way, every instance of Mulberry on each of my many computers grabs my settings and address from the IMSP server, and all updates are stored on the server.  There does not seem to be support for locking on the file, however, which might be why IMSP isn't recommended any longer, but it hasn't really become a problem for me as I often only have one client open at a time.  I highly recommend something like this for Thunderbird.

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> Yes, it can be changed manually, but the point the OP is making is the
> *default* location for *all* IMAP accounts should always be the Local Settings
> folder, and I agree.

If "*default* location for *all* IMAP accounts when account is created", it's easily be customized before first annount creation after profile creation. 

> (for mail.server.serverX.type = imap)
> user_pref("mail.root.imap", "C:\\wada\\MAIL-NEWS\\ImapMail");
> user_pref("mail.root.imap-rel", "[ProfD]../../../MAIL-NEWS/ImapMail");
> (for other types, next entries)
> mail.root.nntp / mail.root.nntp-rel
> mail.root.none / mail.root.none-rel
> mail.root.pop3 / mail.root.pop3-rel
Above is for next environment.
> profile directory       : C:\wada\MOZ-PROF\prof-1\ss93y1tt.slt\
> mail directory for IMAP : C:\wada\MAIL-NEWS\ImapMail\


(In reply to comment #6)
> Now... there is one problem with this... when you log in with a different PC,
> since the .msf files will no longer be included in the roaming profile, you
> will always lose any changes you make to the message pane columns/view, since
> these are now stored in the .msf file.

Charles Marcus, if you want ".msf" is sync'ed and you don't need mail data in offline-store(absolutely different concept from cache) upon your location change(use different PC), why you don't disable auto-sync of IMAP folders? 
  - offline use = off of all IMAP folders (Folder properties/Synchronization)
  - or, mail.server.default.autosync_offline_stores=false
In this case "Disk Cache" of each PC is used for caching of mail data, if cache directory location is not explicitly specified in prefs.js.
"Disk Cache" of Tb 3 is currently next on Win-XP.
> C:\Documents and Settings\<user>\Local Settings\Application Data\Thunderbird\Profiles\<same_as_profile_dir>\Cache\
> Disk Cache size can be specified at Tools/Options/Advanced/Network & Disk Space.
i.e. Your next bug summary is already implemented and works well.
> IMAP Cache Should be Stored in "Local Settings" Application Data Folder
Tb simply doesn't use Disk Cache for an IMAP mail folder if auto-sync is enabled for the IMAP folder.  

I think "use Disk Cache instead of offline-store" is sufficient for next use case (a), although it is not sufficient for use case (b).
(a) Roming profile is used.
    AM: PC-1 at office-1, PM: PC-2 at office-2, night: PC-3 at home
(b) Roming profile is used.
    a month    : PC-1 at office-1 in country-1,
    next month : PC-2 at office-2 in country-2, next of next month : ...
    In this case, "download all mail data to offline-store during first
    several days and use offline-store during one month" is hard,
    because some setting change is required upon each location change.

Comment 9

[Charles]

(In reply to comment #8)
> (In reply to comment #5)
>> Yes, it can be changed manually, but the point the OP is making is the
>> *default* location for *all* IMAP accounts should always be the Local
>> Settings folder, and I agree.

> If "*default* location for *all* IMAP accounts when account is created", it's
> easily be customized before first annount creation after profile creation. 

?? Please read the comment you are replying to again.

We *know* it is 'easily customizable' - that is beside the point. We are talking about the *default*. The OP is asking for the *default* to be changed to the Local Settings so we don't *have* to change it manually.

> (In reply to comment #6)
>> Now... there is one problem with this... when you log in with a different PC,
>> since the .msf files will no longer be included in the roaming profile, you
>> will always lose any changes you make to the message pane columns/view, since
>> these are now stored in the .msf file.

> Charles Marcus, if you want ".msf" is sync'ed and you don't need mail data in
> offline-store(absolutely different concept from cache)

I know that...

The OP wants the default location for the offline store to be in the Local Settings, and I agree with him.

I pointed out that the .msf files need to be separated out from the offline store - or, at least, the code in them that remembers customized column/column order/sort view settings - so that this *can* be stored in the roaming profile, separate from the offline store (bug 543963).

> upon your location change(use different PC), why you don't disable auto-sync
> of IMAP folders?

I do - but I don't want to have to do that manually each time - but that's a different bug (don't recall the number off the top of my head)...

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #9)
> I pointed out that the .msf files need to be separated out from the offline store

I see.
Different location of .msf and mbox file is also a needed enhancement for solution of "easy access to backup of non-zipped mbox file on CD" etc. 
If different location of .msf and offline-store of IMAP will be implemented, (1) rebuild-index from mbox file on CD, (2) treat mbox file on CD like offline-store of IMAP, will probaly be a simplest solution. Because offline-store file larger than 4GB is already supported by Tb trunk(may be by Tb 3.1), backup of large mbox file by other system or backup of large offline-store can be accessed as mail folder.

I think bug summary is better to be changed like next.
> IMAP offline-store Should be able to be Stored in "Local Settings" Application Data Folder

Comment 11

[Charles]

(In reply to comment #10)
> I think bug summary is better to be changed like next.
>> IMAP offline-store Should be able to be Stored in "Local Settings" Application
>> Data Folder

True - the summary says 'cache' but from everything I read in the comments, he meant 'offline store'...

Daniel? U still around?

Comment 12

[brian]

Hi,

Came across this bug cause its killing me! I install thunderbird for lots of users, and would like to see the imapmail\account(s) folders moved to local settings too.

I got one user with ~8GB of imap email - its backed up on the mailserver, we dont need it being backed up on the windows box as well.

I just had to upgrade a servers disks to cope with this.

I have written a vb script that I'm planning on putting in the login script to move the folders when the users log in.

Comment 13

[Daniel Morante]

(In reply to comment #11)
> (In reply to comment #10)
> > I think bug summary is better to be changed like next.
> >> IMAP offline-store Should be able to be Stored in "Local Settings" Application
> >> Data Folder
> 
> True - the summary says 'cache' but from everything I read in the comments, he
> meant 'offline store'...
> 
> Daniel? U still around?

(In reply to comment #11)
> (In reply to comment #10)
> > I think bug summary is better to be changed like next.
> >> IMAP offline-store Should be able to be Stored in "Local Settings" Application
> >> Data Folder
> 
> True - the summary says 'cache' but from everything I read in the comments, he
> meant 'offline store'...
> 
> Daniel? U still around?

Yes, you are correct, "Offline Store" is the proper term here.