Open Bug 965448 Opened 6 years ago Updated 1 year ago

Investigate HomeProvider DB maintenance

Categories

(Firefox for Android :: Data Providers, defect)

All
Android
defect
Not set

Tracking

()

People

(Reporter: mcomella, Unassigned)

References

(Blocks 1 open bug)

Details

Summary: Investigate required home panel DB maintenance → Investigate required HomeProvider DB maintenance
Priority: -- → P1
Priority: P1 → P2
mcomella, do you know what needs to be done here? I don't think this is a high priority, but correct me if I'm wrong.
Flags: needinfo?(michael.l.comella)
Priority: P2 → --
(In reply to :Margaret Leibovic from comment #1)
> mcomella, do you know what needs to be done here?

One form of database maintenance is vacuuming. I forget the specific pros, cons, pitfalls, etc. of vacuuming but a short summary can be found here: http://lzone.de/How%20to%20Vacuum%20SQLite The short summary is vacuuming removes fragmentation, but at the cost of a full DB copy.

I implemented vacuuming for the FHR database in bug 870171, which has a little bit of discussion (interspersed with a lot of other things). Desktop implemented this for places.db in bug 512854 (but it has since gotten much more complicated and some decisions were determined to have been wrong).

> I don't think this is a high priority,
> but correct me if I'm wrong.

I don't actually know what the performance implications might be if one does not vacuum, and thus how necessary it is. Note that we don't actually vacuum browser.db (bug 974995), so it may not be that urgent.

Additionally, a relevant comment from rnewman via bug 974995 comment 1:

> Vacuuming isn't always worthwhile, as you know from FHR!
>
> It's expensive, and it can make subsequent writes more costly.
>
> But yes, research! Perhaps a free page percentage in FHR or telemetry would be interesting…

Richard, do you have anything to add?
Flags: needinfo?(michael.l.comella) → needinfo?(rnewman)
Summary: Investigate required HomeProvider DB maintenance → Investigate HomeProvider DB maintenance
Vacuuming only makes sense in limited situations: e.g., you've deleted a lot of data, don't plan to insert a lot more, and wish to reclaim the space; or you've got a very heterogeneous database that sees a heavy delete/insert write load and you want to periodically reduce fragmentation to improve read efficiency and disk space utilization.

Both of these apply, to an extent, to FHR: we go through frequent purges after which we'd like to compact the DB, and those purges also provide opportunity for new environments and events to be interspersed in free pages, which increases fragmentation.

They also apply to places.db, which regularly prunes your history list and then accrues more visits and bookmarks. But literally years have been spent tuning that.

A database that only grows, a database that grows and shrinks predictably within known parameters, or a database that never gets large, isn't a good candidate for vacuuming.

In short: you need to know (a) some domain-level usage parameters, and (b) record telemetry about fragmentation, before you can really make a good call on whether (and when!) to vacuum.

You might consider other kinds of maintenance first -- e.g., just straight-up deleting content, dropping tables or indices, and so forth.

If you do decide that bloat and fragmentation are a problem, you might then consider other kinds of remediation: vacuuming, yes, or perhaps using a different storage layer altogether.

Go measure!
Flags: needinfo?(rnewman)
I feel compelled to share this relevant link.

http://i.imgur.com/bDA2AnF.gif
Component: Awesomescreen → Data Providers
You need to log in before you can comment on or make changes to this bug.