1764857 - Clarify meanings of nsIMsgDBHdr.offlineMessageSize and nsIMsgDBHdr.messageSize

Status: RESOLVED FIXED

(MailNews Core :: Database, task)

Description

[Ben Campbell]

I'm a little hazy on what nsIMsgHdr.offlineMessageSize is, and how it relates to .messageSize and the size of the original message on IMAP.

Ideally I'd like to ditch it completely, if possible, but first I need to pick through the code and work out exactly how it's used.

(I have a nasty suspicion that for the pluggable messageStore, .messageSize is used to hold the size for local folders, whereas .offlineMessageSize is used for IMAP folders. Thus necessitating a whole separate set of functions (nsMsgDBFolder::GetOfflineFileStream() et al) to be used by IMAP folders to access the messageStore... this would suck, and if it turns out to be the case it should be rationalised to use the messageStore in a consistent fashion).

Comment 1

[Ben Campbell]

Findings

OK, so I've picked my way through all the uses of .offlineMessageSize and done some testing and I think I've got it sorted out:

For local folders:
.messageSize holds the size of the message, as written into the msgStore. The message may include a leading "From " line and synthesized X-Mozilla-(Status|Status2|Keys) header lines, all of which will be included in .messageSize.
.offlineMessageSize - is unset or zero.

For folders with "offline" support, ie IMAP and NNTP:
.messageSize is the original size of the message, on the server.
.offlineMessageSize - if the message is stored offline, this holds the size of the message, as written into the msgStore. The entire size, including any added "From " separator and X-Mozilla-* headers.
.offlineMessageSize will usually be larger than .messageSize by a few bytes (anywhere up to a few hundred bytes difference).
I haven't checked, but I assume that if the message is not stored offline (in the msgStore), .offlineMesssageSize is unset or zero.

Implications:

Currently, the mbox msgStore exposes a raw nsIInputStream which sees the whole mbox, leaving it up to the caller to stay within the message bounds.
This is problematic, because the message size is stored in the different attributes in local and IMAP/NNTP folders. This means the different folder types have to handle message access differently. Even though the nsIMsgPluggableStore is meant to provide uniform access.
Ugh.

Proposed improvements

A lot of this mess will be sorted out in Bug 1733849, which makes sure nsInputStreams used to read messages contain just a single message, so the size is irrelevant - callers can just read as much as they want until they hit an EOF.

But, to help get to that point, here are my plans:

1. .messageOfflineSize should be the exact size of the data you read from msgStore, for all folder types:
   • for local folders: .messageSize and .messageOfflineSize will be the same, and always set.
   • for IMAP/NNTP: .messageSize is always set (to the original size), and if the "Offline" flag is set, .messageOfflineSize holds the size of the message in the msgStore. If the "Offline" flag is clear, .messageOfflineSize should be unset or zero.
     There's a migration issue here, as local folders don't currently set .messageOfflineSize currently (but the value is available on .messageSize, so a fallback could be used).
2. Unify msgStore inputstream access through a single function which handles any fallbacks required for 1). nsMsgDBFolder::GetSlicedOfflineFileStream() is a good example, which could be altered and renamed to handle local folders too. This is basically Bug 1733849.

In fact, that's basically a plan to resolve Bug 1733849, so I'll close this and further updates will happen over there!

Comment 2

[Ben Campbell]

Attachment: Bug 1764857 - Comment tweaks to clarify msghdr .messageSize and .offlineMessageSize meanings. r=#thunderbird-reviewers

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/542ef278cf7f
Comment tweaks to clarify msghdr .messageSize and .offlineMessageSize meanings. r=aleca

Comment 4

[Ben Campbell]

Just an breadcrumb:

The IMAP gmail hack further complicates things. It allows multiple msgHdrs to refer to the a single locally-stored offline message.
However it looks like the Offline flag is set only for the msgHdr representing the "real" message. So you can't safely use the Offline flag to determine if the length of the locally-stored message is in .messageSize or .offlineMessageSize. Sigh.