Firefox remembers less browsing history than before
Categories
(Firefox :: Bookmarks & History, defect, P3)
Tracking
()
Tracking | Status | |
---|---|---|
firefox100 | --- | fixed |
People
(Reporter: botond, Assigned: gliu10000, Mentored)
References
Details
(Keywords: dataloss)
Attachments
(3 files, 1 obsolete file)
I don't remember exactly when this changed, but I've noticed that Firefox remembers less browsing history than before.
My recollection is it used to remember something like the past 6 months, and these days it's only the past ~2 months or so.
I run into this a lot -- typing things into the awesomebar that I know I have visited within the past 6 months or so but are no longer in my history -- and it would be nice to be able to increase the retention period back to 6 months (or another period of the user's choosing).
Is there a preference for this, or can one be added?
Comment 1•3 years ago
|
||
No, there is no official preference, the hard limit is based on the database file size and, unless there has been a dataloss due to corruption two months ago, the only way to reduce history length is either a huge change in browsing behavior (you suddenty start browsing 10 hours a day instead of 1) or we start saving more data.
We have telemetry over that, and afaict it doesn't show a major reduction in the stored history size, there is a small reduction but indeed we added a few more things and we're comfortable with that (We're storing on average a bit more than a year of history). Nothing as big as what you are seeing though, so there must be more to this case.
Do you have a places.sqlite.corrupt in your profile folder?
If you order history in the Library by most recent date, what is the oldest date you see?
Reporter | ||
Comment 2•3 years ago
|
||
(In reply to Marco Bonardo [:mak] from comment #1)
Do you have a places.sqlite.corrupt in your profile folder?
Nope
If you order history in the Library by most recent date, what is the oldest date you see?
December 20 2021, so not even a full two months of history.
Comment 3•3 years ago
•
|
||
Could you please open chrome://browser/content/places/interactionsViewer.html go to Places Stats section, and get a screenshot of those stats?
And yes, we should make it easier to get stats from about:support, I'll file a bug for that.
Comment 4•3 years ago
|
||
could you please also tell me the value of any pref under places.history.expiration.* branch?
Reporter | ||
Comment 5•3 years ago
|
||
(In reply to Marco Bonardo [:mak] from comment #3)
Could you please open chrome://browser/content/places/interactionsViewer.html go to Places Stats section, and get a screenshot of those stats?
This page does not load for me. I should mention that the profile in question is running in a 91.5esr installation.
(In reply to Marco Bonardo [:mak] from comment #4)
could you please also tell me the value of any pref under places.history.expiration.* branch?
There aren't any.
Comment 6•3 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #5)
(In reply to Marco Bonardo [:mak] from comment #3)
Could you please open chrome://browser/content/places/interactionsViewer.html go to Places Stats section, and get a screenshot of those stats?
This page does not load for me. I should mention that the profile in question is running in a 91.5esr installation.
Ah yes, we added that page after 91.
Could you please run Database Integrity from about:support Places Database section and attach here results from running it?
Unfortunately, being an older version it will have a lot less details, but hopefully something useful.
Comment hidden (typo) |
Reporter | ||
Comment 8•3 years ago
|
||
Attached is the output of the integrity verification.
Reporter | ||
Updated•3 years ago
|
Comment 9•3 years ago
•
|
||
Ok, something's very wrong here:
Places.sqlite size is 20480KiB
History can store a maximum of 24265 unique pages
That number is very low, I'd expect it to be > 100k in most cases. Because that number is low, Places will try to reduce history into that limit by expiring the oldest stuff.
How much memory does your system have?
Could you please run this code in the browser console and report results: Services.sysinfo.getProperty("memsize")
How much free space on disk does your profile folder have?
Could you please run this code in the browser console and report results: PlacesUtils.history.DBConnection.databaseFile.QueryInterface(Ci.nsIFile).diskSpaceAvailable
Reporter | ||
Comment 10•3 years ago
|
||
(In reply to Marco Bonardo [:mak] from comment #9)
How much memory does your system have?
Could you please run this code in the browser console and report results:Services.sysinfo.getProperty("memsize")
33551912960
That's 32GB, which is what I would expect.
How much free space on disk does your profile folder have?
Could you please run this code in the browser console and report results:PlacesUtils.history.DBConnection.databaseFile.QueryInterface(Ci.nsIFile).diskSpaceAvailable
882970624
That's 842MB, which seems low (I have more free disk space than that). I'm guessing this one is the determining factor for the history size limit?
Reporter | ||
Comment 11•3 years ago
|
||
How much free space on disk does your profile folder have?
Could you please run this code in the browser console and report results:PlacesUtils.history.DBConnection.databaseFile.QueryInterface(Ci.nsIFile).diskSpaceAvailable
882970624
That's 842MB, which seems low (I have more free disk space than that). I'm guessing this one is the determining factor for the history size limit?
It appears that I'm using profile-sync-daemon to run my profile from a RAM disk, and the RAM disk indeed has 842MB of free space.
Comment 12•3 years ago
|
||
The severity field is not set for this bug.
:mak, could you have a look please?
For more information, please visit auto_nag documentation.
Comment 13•3 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #10)
How much free space on disk does your profile folder have?
Could you please run this code in the browser console and report results:PlacesUtils.history.DBConnection.databaseFile.QueryInterface(Ci.nsIFile).diskSpaceAvailable
882970624
That's 842MB, which seems low (I have more free disk space than that). I'm guessing this one is the determining factor for the history size limit?
Yes, this is the limiting factor, we keep a smaller database if the available space is limited. We use 2% as the threshold, so on your ~800MB it would be 17MB. A 4GB space would reach our standard db size, but 4GB may be a lot for a ram disk.
Maybe it's not the right approach and we should just half the target db size on small disks, it would still cause a restriction but not so drastic.
A similar issue happens on profile folders of users with a limited quota.
I'd suggest we simplify this and just half DATABASE_MAX_SIZE value if the calculated memory or disk threshold is < DATABASE_MAX_SIZE.
The code is at
https://searchfox.org/mozilla-central/rev/211649f071259c4c733b4cafa94c44481c5caacc/toolkit/components/places/PlacesExpiration.jsm#691-695
We should probably also print out the memory and disk size on about:support, I'm not sure why we don't.
Assignee | ||
Comment 14•3 years ago
|
||
Hi there! I'd be open to taking this issue. From what I understand, the cause of the "data loss" is that the calculated optimal database size is too small. This size is currently calculated with:
let optimalDatabaseSize = Math.min(
(memSizeBytes * DATABASE_TO_MEMORY_PERC) / 100,
(diskAvailableBytes * DATABASE_TO_DISK_PERC) / 100,
DATABASE_MAX_SIZE
);
I'd suggest we simplify this and just half DATABASE_MAX_SIZE value if the calculated memory or disk threshold is < DATABASE_MAX_SIZE.
If we detect that the memory or disk is small, would it be possible thathalf DATABASE_MAX_SIZE
exceeds the memory or the disk space available? Or can we safely assume thatDATABASE_MAX_SIZE
being 75MiB is never going to realistically exceed that? If we do have to account for this scenario, I think we end up back to where we were; we have to take into account the disk space available and the memory size, so I'm not sure how much code simplification we can do in this case.
I also noticed that MEMSIZE_FALLBACK_BYTES
and DISKSIZE_FALLBACK_BYTES
are both 256 MiB. This means that any time diskAvailableBytes
or memSizeBytes
is set to the fallback value, we also end up in a situation with an unreasonably low database size (~10 MiB or ~5 MiB).
Assignee | ||
Comment 15•3 years ago
|
||
Depends on D140912
Updated•3 years ago
|
Assignee | ||
Comment 16•3 years ago
|
||
(In reply to gliu20 from comment #15)
Created attachment 9267529 [details]
Bug 1753729 - Add system memory and disk size and placeDB page limit to about:support. r?#fluent-reviewers,GijsDepends on D140912
I'm still new to contributing, but it seems like I made a child revision. I just removed it.
Comment 17•3 years ago
|
||
(In reply to Marco Bonardo [:mak] from comment #6)
(In reply to Botond Ballo [:botond] from comment #5)
(In reply to Marco Bonardo [:mak] from comment #3)
Could you please open chrome://browser/content/places/interactionsViewer.html go to Places Stats section, and get a screenshot of those stats?
This page does not load for me. I should mention that the profile in question is running in a 91.5esr installation.
Ah yes, we added that page after 91.
Could you please run Database Integrity from about:support Places Database section and attach here results from running it?
Unfortunately, being an older version it will have a lot less details, but hopefully something useful.
I have FF97 (dev edition) running and that page doesn't load for me either.
(In reply to Marco Bonardo [:mak] from comment #9)
Ok, something's very wrong here:
Places.sqlite size is 20480KiB
History can store a maximum of 24265 unique pagesThat number is very low, I'd expect it to be > 100k in most cases. Because that number is low, Places will try to reduce history into that limit by expiring the oldest stuff.
https://bugzilla.mozilla.org/attachment.cgi?id=9267952
For me it is lower...: "History can store a maximum of 20936 unique pages"
Actually lower than the amount of bookmark records: "+ Table moz_bookmarks has 22889 records"
Comment hidden (advocacy) |
Assignee | ||
Comment 19•3 years ago
|
||
Depends on D140918
Comment hidden (advocacy, off-topic) |
Comment 21•3 years ago
|
||
Comment 22•3 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/b6773b17a61c
https://hg.mozilla.org/mozilla-central/rev/9f4330cf340a
Description
•