834482 - [email] Store non-$meta per-folder folderInfo individually instead of per-account (speculative scaling/perf issue)

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P3)

Description

[Andrew Sutherland [:asuth] (he/him)]

Currently, all per-folder info ($meta and non) is aggregated into a single per-account record.  All of these records are loaded as part of our startup process.

Each folder-info entry looks a bit like the following IMAP case:
    var folderInfo = this._folderInfos[folderId] = {
      $meta: {
        id: folderId,
        name: name,
        path: path,
        type: type,
        delim: delim,
        depth: depth,
        lastSyncedAt: 0
      },
      $impl: {
        nextId: 0,
        nextHeaderBlock: 0,
        nextBodyBlock: 0,
      },
      accuracy: [],
      headerBlocks: [],
      bodyBlocks: [],
      serverIdHeaderBlockMapping: null,
    };

The $meta stuff is good/compact stuff that is reasonable and appropriate for us to store and load on a per-account basis.  Specifically, I find it reasonable to consume resources proportional to the number of folders a user has.  From Thunderbird, we know there are people with 10k folders; they will just have more resources consumed.

"accuracy" tracks accuracy range info, which for ActiveSync will be a single-entry array.  For IMAP it will usually be a short-list; 10's of entries would be the.

headerBlocks/bodyBlocks are O(log(n)) where n is the number of messages.  serverIdHeaderBlockMapping is not used in IMAP and is currently a simple O(n) mapping from server id's to block id's, although we have ideas about how to make that more clever by using bloom filters/etc.  $impl stores related book-keeping info.

All of the non-$meta information is required in order to read or write to a folder.  These can be stored in a single per-folder row that is asynchronously fetched on demand when sliceOpenMostRecent or a SearchSlice wants to be opened on the folder.  The FolderStorage could then drop its non-dirty contents when so requested.


The current implementation was an initial simplifying decision.  Unless we are dealing with accounts with many folders or many, many synchronized messages, I would expect that most perf wins would come from simply deferring initialization of FolderStorage instances, and that is something we can do before making the database changes.

Comment 1

[Mike Lee [:mlee]]

Andrew, is this work the email team still wants to do?

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Yes, this is still an enhancement we would like to implement.

Comment 3

[BMO Automation]

Firefox OS is not being worked on