1883503 - [meta] Add size measurements for things we'd like to back up

Status: RESOLVED FIXED

(Firefox :: Profile Backup, task)

Description

[Mike Conley (:mconley) (:⚙️)]

One of the things we'd like to do early on is get a sense of how large (in megabytes) a generated backup might be. Our approach is to collect technical telemetry that stats the various data stores that we're hoping to back up, and sending those up via Glean not long after startup.

The things we think we'd like to measure right off the bat:

• places.sqlite [1]
• favicons.sqlite [1]
• cookies.sqlite
• formhistory.sqlite
• cert9.db
• key4.db
• logins.json
• prefs.js
• extensions/
• extensions.json
• addonStartup.json.lz4
• xulstore.json
• sessionstore.jsonlz4
• sessionstore-backups
• permissions.sqlite
• content-prefs.sqlite
• autofill-profiles.json
• containers.json
• credentialstate.sqlite
• enumerate_devices.txt
• ExperimentStoreData.json
• extension-settings.json
• extension-store-permissions/data.safe.bin
• handlers.json
• protections.sqlite
• search.json.mozlz4
• SiteSecurityServiceState.bin
• user.js
• chrome/userChrome.css
• chrome/userContent.css

This metabug will have bugs filed against it to land individual Glean probes for these where appropriate.

[1]: These are already measured via Histograms: https://searchfox.org/mozilla-central/rev/202c48686136360a23b73a49b611a19e64f3e1b8/toolkit/components/telemetry/Histograms.json#6778,6792

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Hey mak, I see that we already measure the places.sqlite and favicons.sqlite databases in MB via those two histograms.

Of the things we're hoping to measure above, some might definitely be several megabytes in size. Others might be less than 1 megabyte. To make analysis easier, I'm debating whether or not to collect the sizes of these things in kilobytes.

I've asked our data scientist to see if it'll be a pain to combine that with the existing probes for favicons.sqlite and places.sqlite. If it is as much of a pain as I suspect it might be, which would you prefer?:

1. We add a second set of new Glean probes measuring places.sqlite and favicons.sqlite in kilobytes for the backup case?
2. We replace the existing Histograms with two new Glean probes measuring in kilobytes?
3. We try to make the existing Histograms work, despite using legacy Telemetry and in units of megabytes?

Comment 2

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #1)

Hey mak, I see that we already measure the places.sqlite and favicons.sqlite databases in MB via those two histograms.

Of the things we're hoping to measure above, some might definitely be several megabytes in size. Others might be less than 1 megabyte. To make analysis easier, I'm debating whether or not to collect the sizes of these things in kilobytes.

I think that's fine. We used MiBs for Places because both dbs were large, I'd have no concern with moving to KiBs. And if you're planning to implement a centralized way to measure profile files, I'm also happy to remove the places-only measurement and use the common one at a certain point. I don't think DS is using database sizes much, Engineering uses it to ponder changes and keep an eye on health. So I don't expect anyone to complain about moving measurement to another probe with another unit.

1. We add a second set of new Glean probes measuring places.sqlite and favicons.sqlite in kilobytes for the backup case?

That's fine

2. We replace the existing Histograms with two new Glean probes measuring in kilobytes?

This can also happen at a later time, I can update my Redash Places Dashboard to use the new Glean measure, and then we can remove the old histograms.

3. We try to make the existing Histograms work, despite using legacy Telemetry and in units of megabytes?

I don't think it's necessary.

Comment 3

[Mike Conley (:mconley) (:⚙️)]

All dependencies closed. We're done here.