511118 - Enable Space policy for autosync

Status: NEW

(Thunderbird :: General, defect)

Description

[David :Bienvenu]

+++ This bug was initially created as a clone of Bug #482476 +++

The default autosync policy downloads aggressively, all messages. Bug 482476 implemented date constraints, so messages older than XX days wouldn't be downloaded, or if they were downloaded, they would be purged from the offline store at compaction time.

Some users will want disk size constraints on the offline stores. This bug is for that. This could partly be implemented by an extension which added a pref to set the space constraint, and code that figured out which messages to expunge, and calling nsIMsgImapMailFolder::MarkPendingRemoval with the msg hdr.

The part that would be harder to do in an extension is making autosync respect the space constraints.

Comment 2

[Bryan Clark (DevTools PM) [:clarkbw]]

Getting this bug fixed is going to be an important part of our Netbook integration story, https://wiki.mozilla.org/Thunderbird/Netbooks

I'm going to mark this blocking so we can get this on the radar.  I'll post additional comments as we have some progress on design.

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Given my current understanding of UX priorities, it's my belief that this still wants to block 3.1, so marking as such.  Bryan, feel free to let me know if that's incorrect.  Targeting at beta2, so that we have a chance to get feedback and change in RC1 if necessary.

Comment 4

[Charles]

I know this in only tangential to this particular bug, but I would appreciate if while working on this bug, consideration would be given to bug 543956 with regard to space constraints and what TB downloads when offline/sync settings are totally disabled for a given IMAP account.

In short, I did a quick test to see just how much space is consumed by the Partial Headers TB currently downloads when offline/sync settings are disabled.

The test account has 7.6GB of mail on the server, with 3.6GB in her Sent folder, and the rest spread out over about a hundred different folders.

After clicking on each folder and letting TB download the (partial) headers it downloads, the total disk space consumed was a whopping 14MB.

Now, I don't know how much more space would be consumed if TB instead downloaded *All* headers as opposed to just the partial ones, but I can't believe that it would be more than 3 times as much. So, that would mean that an account with 10GB of mail would consume less than 50MB of disk space for all headers.

In this day and age (of cheap large hard drives), I really cannot see this being a problem - certainly not enough of a problem to base default settings on.

Comment 5

[Jens Müller (:tessarakt)]

50 MB should even be ok for SSDs in most cases.

Comment 6

[David :Bienvenu]

(In reply to comment #3)
> Given my current understanding of UX priorities, it's my belief that this still
> wants to block 3.1, so marking as such.  Bryan, feel free to let me know if
> that's incorrect.  Targeting at beta2, so that we have a chance to get feedback
> and change in RC1 if necessary.

If Netbook is a blocker for 3.1, then I suggest the following variation on the strategy I described in c#0. The autosync code can implement a hidden pref autosync.max_disk_space and not autosync when offline storage disk space usage exceeds that limit, if set (or we expose hooks in autosync for extensions to control whether autosync should happen). Then, the netbook extension can handle figuring out which messages to mark for pending removal, and handle calling compact if needed, and perhaps also set the auto compact pref when more than XX Kb will be saved by default. If the netbook extension wants to have per-folder limits for offline stores, it could do that as well.

Comment 7

[Charles]

I would argue that for netbooks (and any IMAP client that deals with accounts with huge mail stores), the most intelligent use of disk space would be to download All headers, set offline settings to be enabled (for 'sync on demand' behavior), auto-sync disabled (so only messages that are intentionally clicked are downloaded), and then apply the disk storage constraints to the offline store, eliminating oldest offline messages first.

But again, All headers should always be available, so that more intelligent management capabilities are available - ie, apply filters to messages in IMAP folders based on headers that are not currently available since TB does not currently download All headers when auto-sync is disabled, choose how different mime-type message parts are handled, etc.

Comment 8

[Charles]

(In reply to comment #4)
> Now, I don't know how much more space would be consumed if TB instead
> downloaded *All* headers as opposed to just the partial ones, but I can't
> believe that it would be more than 3 times as much. So, that would mean that
> an account with 10GB of mail would consume less than 50MB of disk space for
> all headers.

(In reply to comment #5)
> 50 MB should even be ok for SSDs in most cases.

Even if the amount of space consumed by the full headers was *10* times, it would still only be @ 160MB for 10GB of mail - still plenty ok even for SSDs in most cases...

Comment 9

[David :Bienvenu]

This bug is really about the message bodies, not the headers.

Comment 10

[Charles]

(In reply to comment #9)
> This bug is really about the message bodies, not the headers.

I know, hence the 'I know this in only tangential to this particular bug' qualifier in my first post on this aspect. But, it *is* about what is downloaded and storage requirements.

I just wanted to make sure whoever was working on this had at least seen that enhancement request and would take it into consideration if/when working on the code.

Anyway, sorry, didn't intend to hijack the bug...

Comment 11

[Dan Mosedale (:dmosedale, :dmose)]

We're resetting the blocking flag for 3.1 on this bug and instead setting the wanted-thunderbird+ flag. We have too many blocking-3.1 bugs, to the point where it doesn't mean much, and managing the list is making it hard to actually work on closing bugs, which helps no one.

Thunderbird 3.1's primary purpose is to allow us to offer a prompted major update to Thunderbird 2 users, to ensure their continued ability to safely use Thunderbird.  Thunderbird 2 is built on an outdated version of Gecko, and our long-term ability to maintain the users' safety for Thunderbird 2 users is limited.

If you think this bug meets the requirements below, please renominate with a detailed explanation of how it meets the following two criteria, and we will reconsider.  To qualify, this bug must either:

a) make the upgrade experience from TB2 very painful for a large number of users

or

b) be a new, reproducible, severe quality issue (eg dataloss, frequent crashes)

Just because this bug doesn't block TB3.1 doesn't mean it can't or won't make the release.  Once they're done with their blockers (if any), we encourage developers to keep working on non-blocking bugs, and to try to land them as early in the cycle as possible, as non-blocking bugs will become increasingly difficult to land in the later stages of the cycle.