495572 - gloda should explicitly handle compaction

Status: RESOLVED FIXED

(MailNews Core :: Database, defect)

Description

[Andrew Sutherland [:asuth] (he/him)]

<bienvenu> asuth, what makes gloda work after compacting a local folder?
<bienvenu> I get messages with gloda id's, but gloda won't fetch snippets, for example
<bienvenu> and I assume it's because the msg key changed
<asuth> yeah, gloda will need to re-index
<asuth> I don't think I ever specifically dealt with that case
<bienvenu> so it doesn't know it has to re-index?
<asuth> I was hoping the events just worked out for us
<asuth> that sounds likely
<asuth> I realize there is the foldercompacted event
<asuth> gloda should probably be listening to that
<bienvenu> compact is unfortunately maintaining the gloda-id when compacting...
<bienvenu> but it would be so nice if gloda could just realize the keys had changed
<asuth> yeah, we should listen to compaction, mark the folder as compacted-dirty, then do a specialized pass through where we just fix up the message keys
<asuth> I think that is probably as efficient as we can be?
<bienvenu> asuth, yes, that's as efficient as you could be, if you hang onto msg keys
<asuth> gloda used to have fallback logic that might have hidden things...
<asuth> if it went to grab a message but couldn't find it by the message key, it would do a last stab using the message id
<bienvenu> one thing to note is that compact might result in the first 100 keys not changing, but next 50 definitely changing
<bienvenu> asuth, yes, that would fix it as well

Comment 1

[David Ascher (:davida)]

asuth to at the very least attach his WIP patch

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Very close to having this working and tested.  I have added a nsIMsgFolderListener.itemEvent notification for the start and end of compaction.  Combined with the ability to cleanly kill indexing jobs, I believe this covers the semantic edge cases related to compaction.

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

Compaction unit test test_index_compaction.js passes.  Another victory for unit tests!

Comment 4

[Andrew Sutherland [:asuth] (he/him)]

Fixed in gloda correctness patch tracked on bug 465618.
trunk commit: http://hg.mozilla.org/comm-central/rev/413b2018349c