535137 - File location of index (global-messages-db.sqlite) should be customizable

Status: NEW

(MailNews Core :: Backend, enhancement)

Description

[Jack]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: Thunderbird/3.0

Thunderbird should let me choose where to store the global index file (global-messages-db.sqlite).

it is now located in the profile directory, but I would prefer to save it elsewhere, because my global-messages-db.sqlite is about 800Mb... 

Reproducible: Always

Comment 1

[Ludovic Hirlimann [:Usul]]

hum can't find a duplicate while I'm sure there is one.

Comment 2

[rmvbfk]

Maybe bug 518812 ?

Comment 4

[Russell East]

The gloda index file, and any other derivative files, should be placed in a non-profile folder - preferbaly not a sub-folder of the profile.  This is important for backups of the profile folder, particularly when the derivatives become humongous like the the gloda file.

Comment 5

[WADA:World Anti-bad-Duping Agency]

There are two kinds of issue of big file in profile directory when Roaming Profile is used or profile directory is located on network resource.
(a) Big offline-store file by auto-sync in Roaming Profile
    Workaround : (a-1) Disable auto-sync
                 (a-2) Locate mail directory at local drive
(b) Big global-messages-db.sqlite by Gloda in Roaming Profile
    Workaround : (b-1) Disable Gloda
                 (b-2) Locate global-messages-db.sqlite at local drive
Problem-1.
(b-2) is currently impossible. (this bug)

Problem-2.
If user chooses workaround of (a-2), and if user uses a Roaming Profile on different PC, problem will probably arise around using Gloda, because content of offline-store/.msf depends on PC.
It may be resolved by this bug, because offline-store/.msf and global-messages-db.sqlite becomes always consistent on each PC by fix of this bug.

Confirming, as I think reasonable request, although I'm not developer.

Comment 6

[Russell East]

How about place global-messages-db.sqlite (and other such files) in (new) {profile-directory}/temp folder by default.   Then allow end-user to change this location, to another drive if desired, as Jack has requested.   By using "temp" (or choose other favourite name) backup software most likely can avoid backing up that sub-folder, plus, by default it is located with roaming profile.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #6)
> By using "temp" (or choose other favourite name) backup software most likely can avoid backing up that sub-folder,
> plus, by default it is located with roaming profile.

(1) I believe Tb's directory/file name and/or attribute of Tb's directory/file (exclusive lock or not, when/how file content is updated, ...) shouldn't be determined by behaviour of specific backup software which has very limited capability to control behaviour of the software.
(2) What should be default location of something is different issue from enhancement request of this bug.

Comment 8

[Russell East]

I take your point - I was hoping for a generic solution that handles all TB "temporary" files in a similar fashion, instead of solving the narrow scope of the current enhancement request.

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> I was hoping for a generic solution that handles all TB "temporary" files
> in a similar fashion, instead of solving the narrow scope of the current enhancement request.

This bug is request of imporovement only on global-messages-db.sqlite, which produces problem of (b) in comment #5. If you believe general improvements on Tb's files such as one you think "temporary" file, please open separate bug for your request.

Comment 11

[Ian.Leveson]

My Global-messages-db.sqlite is "only" ca. 400 Mb but still runs foul of the quota set on our roaming profile size.
If I keep deleting it to avoid error-messages concerning the quota being exceeded, it requires half a work-day to re-index, which makes Thunderbird unusable.

It can't be too difficult to permit a configuration change to move it out of the profile.
Why has this still not been tackled, since it restricts the use of Thunderbird in an enterprise context?

Comment 12

[seigneurao]

I definitely subscribe to this request.

I feel that any "volatile" (i.e. temporary and/or rebuildable) data should not be part of the profile.
Hence, this includes (for those I have identified as of yet) :
- Cache directory
- global-messages-db.sqlite file

The use I'd have of this, is that my TB profile is stored in a Google Drive directory, which consequently syncs each time I close TB.
This is the expected behaviour but currently every single sync includes -by default- the Cache directory (which I have now moved thanks to a custom parameter), as well as global-messages-db.sqlite and this baby is ~50MB on my account (I realize I must stand below average in that).

Any improvement regarding this particular issue would be much welcome.

Comment 14

[Gianni]

It seems that there is an overall agreement on the benefit of storing either all temporary files or only the global-messages-db.sqlite (that is the largest by far) on a customable directory (which could be written in the profile.ini file).
It seems not complicated to me, but I'm not TB developer. 
Does anyone could make a prediction on whether this issue is being solved for coming versions or will be shortly? I see the discussion dates back several years.

Comment 15

[jeff.rollason]

In passing I need this change. My global-messages-db.sqlite file is 1.5gig and killing my system. I seem to be bleeding disk space all over (I had 14 gig last week but this dropped to 43meg after no new installs). Unfortunately my system was configured with a too small a Windows 7 partition of only 48gig although my entire system has 3 terabytes. It was originally delivered as XP main and Windows 7 extra but now is fully Windows 7, but the boot partition is at the end of a drive so cannot be expanded.

Any other tips about where I can configure Thunderbird to minimise space (the mail directories are located elsewhere) would be appreciated.

Comment 16

[Russell East]

This could hardly be termed "NEW" any more...    :-\

I now work from home and must perform my own backups.   Using Windows 10 FileHistory, I can only specify folders to include or folders to exclude, not individual files.   Consequently,  the gloda index gets backed up on the hour, every hour.  It's surprising how quickly backup disk storage is filled up as a result.

Comment 17

[luison]

Not sure what the exact future situation of TB is but here is another vote for this feature. In my case (+8 account over IMAP) my index file can reach 3GB of which I do backups without any use. 

I take great care of moving to a quick non-backup partitions things like windows search index, tmp, spotify cache, etc so they do not get backed up as they are not required for a system restore. All my IMAP email files are moved there as well on each account definition as I can re-download all that in case of failure and would have to reindex in any case. Storing the gloda index proves useless.

Luckily enough for me we don not use domains on windows, as in that case that 3GB index file would have to traffic constantly through the network.

The solution to me would be as simple as allowing a system wide variable on TB setup for a path for cache/tmp files which included this index.

Comment 18

[sondra.kinsey]

+1

Comment 19

[Zbigniew Gralewski]

+1
and change the name from global-messages-db.sqlite to global-messages-db.sqlite.tmp and any other temporary files (not needed for roaming profiles or synchronization) to .tmp
I personally would like to synchronize full profile directories between computers, because more and more people use clouds public/private like dropbox/google/synology-drive to sync files and folders. Syncing thunderbird profile would be nice, but now it becomes trouble. Cache filess pollute the directory, and index data, too.
To have good profile backup I also delete all cache subfolders (cache, cache2, startupcache) and global-messages-sd.sqlite, then zip whole dir then put zip to cloud. Not having to do these things would be nice.

Good solution would be to move all cache/index files from %appdata% to %localappdata% then allow syncing or quickly zipping whole profile dir to have a backup to sync it or when moving it.

Comment 20

[vmtitov42]

+1

Comment 22

[Qubist Pixel]

+1