1631964 - Improve RemoteSettings Reads

Status: NEW

(Firefox :: Remote Settings Client, enhancement)

Description

[Emma Malysz]

Attachment: ff_startup.zip

For this particular startup profile, we only read 1.639MB of the remote settings file from a range of 13.353 MB. Since this is only a fraction of the range, we should see how we can improve this read performance. This file is read at various points throughout startup (first read ~9s all the way up until ~17.5s)

With the above attachment in https://procmon-analyze.github.io/, search "Rsegmnoittet-es.sqlite" to find the RemoteSettings reads.

Comment 1

[Emma Malysz]

Gijs, do you have any suggestions on how we could improve this? Doug had suggested experimenting with caching or figuring out a way to improve our indices.

Comment 2

[:Gijs (he/him)]

(In reply to Emma Malysz from comment #1)

Gijs, do you have any suggestions on how we could improve this? Doug had suggested experimenting with caching or figuring out a way to improve our indices.

This will be because it's 1 db with data for multiple consumers. This is, per my discussion with Andrew earlier today, the "right" way of using indexeddb. RemoteSettings segments the data that comes out of the stores based on who the consumer is (based on idb keyranges), and indexeddb is able to optimize this, AIUI. I think there are already indices in place for listing all data for a given consumer, which is likely what we're doing here. (Mathieu, can you confirm?)

I don't really have context here. Is this the first run on this profile, or a subsequent one? Do we know which remote settings consumers are causing the IO?

I'm not sure how we'd fix this. RS is supposed to provide an intuitive API surface over a DB that is synced to a remote service. It's used by multiple consumers, and some of them run on startup and others do not. Given the ratio of the file size and read size in comment #0, caching the entire thing is clearly silly. But there's a bunch of abstractions (remote settings, indexeddb, sqlite) inbetween the consumers and the actual IO, so I'm not sure how we could reasonably cache this info. My best guess would be duplicating the data to a cache file if we notice we load it at startup, but that would be copy #3 of the data (we also ship initial dumps of a bunch of these datasets in omni.ja - but those will miss updates distributed via remote settings since then), plus we'd need to invalidate it every time the data changes, which can happen every 6 hours. Plus, 2mb is quite a significant amount of data to just duplicate in order to make the read slightly faster...

If we knew what the consumers were, we could look at whether they really need this data so early, or if they could cache the salient details some other way.

Comment 3

[:Gijs (he/him)]

(In reply to :Gijs (he/him) from comment #2)

This will be because it's 1 db with data for multiple consumers. This is, per my discussion with Andrew earlier today, the "right" way of using indexeddb. RemoteSettings segments the data that comes out of the stores based on who the consumer is (based on idb keyranges), and indexeddb is able to optimize this, AIUI. I think there are already indices in place for listing all data for a given consumer, which is likely what we're doing here. (Mathieu, can you confirm?)

Oops, +ni

Comment 4

[Emma Malysz]

This was taken from a cold startup after restarting the system, and the build had been run a few times before the restart. We're hoping to get stacks and incorporate that into our analyzer tool soon, so hopefully that will give more context once it's ready.

Comment 5

[Mathieu Leplatre [:leplatrem]]

I think there are already indices in place for listing all data for a given consumer, which is likely what we're doing here. (Mathieu, can you confirm?)

Yes, by collection, for the timestamp and the records:
https://searchfox.org/mozilla-central/rev/41c3ea3ee8eab9ce7b82932257cb80b703cbba67/services/settings/Database.jsm#327-330