Bug 1781134 Comment 2 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

I've only had a cursory look at this, but I think this is down to test messages being generated without `X-Mozilla-Status`, `X-Mozilla-Status2` or `X-Mozilla-Keys` headers.
For example, a lot of the tests use [MessageInjection.jsm](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageInjection.jsm) to generate dummy messages. That ends up using [`MessageGenerator.makeMessage()`](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageGenerator.jsm#1278) to generate individual messages, with no obvious sign of `X-Mozilla-*`  headers.

It's reasonable enough that the test messages don't have those headers - the `X-Mozilla-*` headers are only used for local folders. But even with local folders it's a little inconsistent. I'm pretty sure there are a few pathways where messages slip into local folders the extra headers.

In any case, missing those headers isn't fatal. They are backups of message flags and keywords which are stored in the message database.

I _think_ the correct thing would be to just ditch the `NS_WARNING`. Shall I do that?



Anyway... for completeness and some background, here are my notes on the `X-Mozilla-*` headers:

--------------

# `X-Mozilla-*` headers

Local folders add a few custom headers to messages when they are stored to disk.
These headers stash message flags and keywords in the message, mirroring the values in the message database.

I think the original idea was that the mbox was the 'definitive' source of truth, and that the message database was just a summary - a throwaway index to make accessing messages easy, one which could always be rebuilt from the mbox.
This is still mostly the case for local folders - a "Repair Folder" will rebuild the database by scanning the mbox (or maildir!).

To avoid having to repack messages on disk when the metadata changes, the headers are written with some blank space which can later be rewritten in place.
The functions which perform this in-place update are `nsMsgLocalStoreUtils::RewriteMsgFlags()` and `nsMsgLocalStoreUtils::ChangeKeywordsHelper()`.

If a header is missing or doesn't have enough space for a in-place edit, it is generally just left up to the next folder compaction to rewrite the header with the full value.

## `X-Mozilla-Status` & `X-Mozilla-Status2`

These hold the message flags (`Read`, `Replied`, `Marked` etc. See 
`comm/mailnews/base/public/nsMsgMessageFlags.idl` for the full list).

`X-Mozilla-Status` holds the lower 16bits worth of flags, as a 4 digit hex number: `X-Mozilla-Status: xxxx`.

`X-Mozilla-Status2` holds the upper 16bits flags only, with the lower 16 bits zeroed. It's a 8-digit number with lower 4 digits always zero: `X-Mozilla-Status2: xxxx0000`.

Yes, this seems a bit bonkers. It should just be a single header. Historical reasons, I guess.

The flags are defined by `nsMsgMessageFlags`, in `comm/mailnews/base/public/nsMsgMessageFlags.idl`.


## `X-Mozilla-Keys`

This holds a list of any keywords the message is tagged with, eg:
`X-Mozilla-Keys: $label1 $label2`
By default, the value is padded with spaces to reserved about 80 characters for further in-place edits.
I've only had a cursory look at this, but I think this is down to test messages being generated without `X-Mozilla-Status`, `X-Mozilla-Status2` or `X-Mozilla-Keys` headers.
For example, a lot of the tests use [MessageInjection.jsm](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageInjection.jsm) to generate dummy messages. That ends up using [`MessageGenerator.makeMessage()`](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageGenerator.jsm#1278) to generate individual messages, with no obvious sign of `X-Mozilla-*`  headers.

It's reasonable enough that the test messages don't have those headers - the `X-Mozilla-*` headers are only used for local folders. But even with local folders it's a little inconsistent. I'm pretty sure there are a few pathways where messages slip into local folders without the extra headers.

In any case, missing those headers isn't fatal. They are backups of message flags and keywords which are stored in the message database.

I _think_ the correct thing would be to just ditch the `NS_WARNING`. Shall I do that?



Anyway... for completeness and some background, here are my notes on the `X-Mozilla-*` headers:

--------------

# `X-Mozilla-*` headers

Local folders add a few custom headers to messages when they are stored to disk.
These headers stash message flags and keywords in the message, mirroring the values in the message database.

I think the original idea was that the mbox was the 'definitive' source of truth, and that the message database was just a summary - a throwaway index to make accessing messages easy, one which could always be rebuilt from the mbox.
This is still mostly the case for local folders - a "Repair Folder" will rebuild the database by scanning the mbox (or maildir!).

To avoid having to repack messages on disk when the metadata changes, the headers are written with some blank space which can later be rewritten in place.
The functions which perform this in-place update are `nsMsgLocalStoreUtils::RewriteMsgFlags()` and `nsMsgLocalStoreUtils::ChangeKeywordsHelper()`.

If a header is missing or doesn't have enough space for a in-place edit, it is generally just left up to the next folder compaction to rewrite the header with the full value.

## `X-Mozilla-Status` & `X-Mozilla-Status2`

These hold the message flags (`Read`, `Replied`, `Marked` etc. See 
`comm/mailnews/base/public/nsMsgMessageFlags.idl` for the full list).

`X-Mozilla-Status` holds the lower 16bits worth of flags, as a 4 digit hex number: `X-Mozilla-Status: xxxx`.

`X-Mozilla-Status2` holds the upper 16bits flags only, with the lower 16 bits zeroed. It's a 8-digit number with lower 4 digits always zero: `X-Mozilla-Status2: xxxx0000`.

Yes, this seems a bit bonkers. It should just be a single header. Historical reasons, I guess.

The flags are defined by `nsMsgMessageFlags`, in `comm/mailnews/base/public/nsMsgMessageFlags.idl`.


## `X-Mozilla-Keys`

This holds a list of any keywords the message is tagged with, eg:
`X-Mozilla-Keys: $label1 $label2`
By default, the value is padded with spaces to reserved about 80 characters for further in-place edits.
I've only had a cursory look at this, but I think this is down to test messages being generated without `X-Mozilla-Status`, `X-Mozilla-Status2` or `X-Mozilla-Keys` headers.
For example, a lot of the tests use [MessageInjection.jsm](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageInjection.jsm) to generate dummy messages. That ends up using [`MessageGenerator.makeMessage()`](https://searchfox.org/comm-central/source/mailnews/test/resources/MessageGenerator.jsm#1278) to generate individual messages, with no obvious sign of `X-Mozilla-*`  headers.

It's reasonable enough that the test messages don't have those headers - the `X-Mozilla-*` headers are only used for local folders. But even with local folders it's a little inconsistent. I'm pretty sure there are a few pathways where messages slip into local folders without the extra headers.

In any case, missing those headers isn't fatal. They are backups of message flags and keywords from the message database.

I _think_ the correct thing would be to just ditch the `NS_WARNING`. Shall I do that?



Anyway... for completeness and some background, here are my notes on the `X-Mozilla-*` headers:

--------------

# `X-Mozilla-*` headers

Local folders add a few custom headers to messages when they are stored to disk.
These headers stash message flags and keywords in the message, mirroring the values in the message database.

I think the original idea was that the mbox was the 'definitive' source of truth, and that the message database was just a summary - a throwaway index to make accessing messages easy, one which could always be rebuilt from the mbox.
This is still mostly the case for local folders - a "Repair Folder" will rebuild the database by scanning the mbox (or maildir!).

To avoid having to repack messages on disk when the metadata changes, the headers are written with some blank space which can later be rewritten in place.
The functions which perform this in-place update are `nsMsgLocalStoreUtils::RewriteMsgFlags()` and `nsMsgLocalStoreUtils::ChangeKeywordsHelper()`.

If a header is missing or doesn't have enough space for a in-place edit, it is generally just left up to the next folder compaction to rewrite the header with the full value.

## `X-Mozilla-Status` & `X-Mozilla-Status2`

These hold the message flags (`Read`, `Replied`, `Marked` etc. See 
`comm/mailnews/base/public/nsMsgMessageFlags.idl` for the full list).

`X-Mozilla-Status` holds the lower 16bits worth of flags, as a 4 digit hex number: `X-Mozilla-Status: xxxx`.

`X-Mozilla-Status2` holds the upper 16bits flags only, with the lower 16 bits zeroed. It's a 8-digit number with lower 4 digits always zero: `X-Mozilla-Status2: xxxx0000`.

Yes, this seems a bit bonkers. It should just be a single header. Historical reasons, I guess.

The flags are defined by `nsMsgMessageFlags`, in `comm/mailnews/base/public/nsMsgMessageFlags.idl`.


## `X-Mozilla-Keys`

This holds a list of any keywords the message is tagged with, eg:
`X-Mozilla-Keys: $label1 $label2`
By default, the value is padded with spaces to reserved about 80 characters for further in-place edits.

Back to Bug 1781134 Comment 2