1817248 - Order of bookmarks is messed up since 110.0

Status: RESOLVED FIXED

(Firefox :: Address Bar, defect, P2)

Description

[seko.idiootti]

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:109.0) Gecko/20100101 Firefox/110.0

Steps to reproduce:

Since updating to 110.0, the order of bookmarks when typing in address bar is always wrong and clearing history does not fix it. In the past, the order of bookmarks sometimes got messed up if Firefox had been open for some time, but clearing history always fixed it.

Actual results:

Bookmarks are ordered in a weird messed up and seemingly random (but consistent) way, with almost all bookmarks that show up being some really old ones.

Expected results:

Bookmarks should be ordered most recent first, and then in descending order.

Comment 1

[seko.idiootti]

For clarification, if I type just "." or just "-" or just "/", all the bookmarks that show up are in correct order (descending, most recent first). Same with some letters like "a" and "t", they'll be in correct order, but if I type "x", only the first one is recent and the rest are from literally over a decade ago, and with eg. "!" they're ALL really old ones. The more I type, the more messed up the order gets, eg. if I type "bas", the first half are from over a decade ago but the second half are in corder order.

This is really frustrating, since at least I often use bookmarks to temporarily bookmark something to check out later and might not remember what the URL was. Now that's impossible, so bookmarks have lost half of their function; now if I want to find something I've recently bookmarked but don't remember the exact URL, I'll have to type a LOT more and it might still not show up, so I'd have to open up all bookmarks and scroll through them, which isn't exactly practical...

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::Bookmarks & History' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[seko.idiootti]

I've realised that all the incorrectly ordered bookmarks that pop up in the address bar have a few things in common:

1. they're really old, bookmarked years ago
2. they don't have tags
3. they're http instead of https

Changing them to https re-bookmarks them, so I guess it's impossible to know if the problem is that they're http (but I doubt it, since I do have a few recent non-https bookmarks as well). Adding tags doesn't fix it, but I don't know if that 100% certainly means it could be ruled out as the cause since they didn't have tags when this problem started? Still, I guess it's probably not lack of tags it since with a lot of stuff typed into the address bar, the bookmarks that show up are in correct descending order even if they don't have tags.

I'm basically tech illiterate, but unless I'm missing something obvious, that just leaves the bookmarks being old as the cause? Could it really be that? Like, the most recent incorrectly ordered bookmarks that I've seen pop up are probably from a decade ago. Well, obviously it could be that I'm just missing something obvious, so dunno.

Also, something changed on its own since I made the bug report. The problem is still the same, but the details are a little different. Now, typing "!" is just like with other punctuation marks, they're in the correct order. Same with all other individual letters and numbers; with more than one letter or number, the orders in some cases are messed up. So, it's like the problem got a little bit less bad on its own... but with a lot of words it's still completely messed up, so it's still really bad. For example, if I type "Latin" in the address bar, all the bookmarks are really old except the one I changed to https, which is now first since it got re-bookmarked.

Comment 4

[seko.idiootti]

Actually, I shouldn't have assumed that all punctuation marks in the address bar would show bookmarks in the right order just because ".", ",", "_", "-", "/" and now "!" do, so I tried """ (as in the double quotation mark itself), ";", "~" and "", and the bookmarks are in messed up order with those... so, yeah, not all bookmarks that have punctuation show up in the correct order. All the ones that do show up with those punctuation marks (that are in the wrong order) are also old, http and don't have tags, though, so... dunno if that means anything?

Comment 5

[Marco Bonardo [:mak]]

Thank you for reporting this problem, I have some theories about why this may be happening, and I'll investigate in the coming days.
I have a few questions that may help my investigation:

1. Is your history enabled or disabled?
2. Do you keep the browser open for many hours, or do you just open it when needed, closing it once done?
3. Do you clear history often?
4. Is your ideal order from the most recently created bookmartk to the oldest one?

Comment 6

[seko.idiootti]

(In reply to Marco Bonardo [:mak] from comment #5)

1. Disabled
2. Usually open whenever I have my laptop on (so like 95% of the time I'm awake), sometimes even up to a week (I tend to turn my laptop off when I sleep, but not always, and then I'll usually also leave Firefox open)
3. Sometimes I'll clear it pretty much every hour, occasionally not for a couple of days... but clearing it often is more common, so yeah
4. Yes

I thought I'd try if running CCleaner would fix the problem, and kinda surprisingly at least for now it did! So, I guess it is/was probaby some kind of cache thing that Firefox's own clearing doesn't "reach" for some reason? I had deleted the "cache2" folder before reporting the bug, which didn't fix it. If it's not a cache thing, maybe it could be some kind of issue with bookmark backups, since those are another thing CCleaner cleared? It was the first time I ran CCleaner in months.

Comment 7

[seko.idiootti]

Well, it got messed up again (to the state before running CCleaner, so not as bad as it was at first; it's like Firefox "forgot" the "fix" that CCleaner did, but not whatever happened when it first got less bad on its own) so I ran CCleaner again, and it's fixed again for now. I've heard running CCleaner too often is bad and it seems like I'd have to run it at least daily, so I hope there's another solution. Before running CCleaner again, I tried setting Firefox to clear "site settings" and "offline site data" too (I had those unchecked), but that didn't help.

Since CCleaner just cleared "cache" and "bookmark backups" from Firefox, logically it has to be one of those two causing it, right? To maybe see if it's because of bookmark backups, I've set the maximum to just one. I didn't try setting it to zero (I don't know if that's even possible), but if it still happens, I'll try that next. If that won't help, then I guess it has to be that Firefox's own cache-clearing isn't clearing everything the way it used to until 110.0 or something like that? Or it's something else that's just considered part of "cache" by CCleaner...

Comment 8

[seko.idiootti]

They still got messed up (after about nine hours) with just one bookmark backup, so I've set it to zero and will run CCleaner again and then restart Firefox... but it seems more like it has to be some kind of "cache" thing, or at least something that CCleaner counts as "cache" that Firefox itself doesn't clear when clearing cache, history, cookies, etc. Still, I'll try with no bookmark backups just in case.

Comment 9

[seko.idiootti]

Sorry for the barrage of comments, but if it can help narrow down what the problem is, it should be fine...?

As a test I first tried clearing JUST the Firefox-related things with CCleaner, and it doesn't fix this. Clearing Firefox's "cache", "bookmark backups" and even both with CCleaner didn't fix it... but clearing EVERYTHING does fix it. So, it's something that CCleaner doesn't recognise as Firefox-related even though literally the only change is with Firefox. I'm really confused. Anyway, I'll keep bookmark backups disabled for now just to see if it somehow relates to that anyway (even if it's kinda obvious that it doesn't).

The next time it happens, I'll try clearing something else that CCleaner detects as clearable. No idea which of them it could be since it's obviously a Firefox bug and only started with the most recent update, but I guess it's a matter of trying the other things one by one...

Comment 10

[Marco Bonardo [:mak]]

CCleaner is not a got choice to cleanup Firefox data, it doesn't know about internals, thus when we make changes it is more likely to break things. My suggestion is to not use it.

Also bookmark backups have no influence here, unless ccleaner is effectively forcing Firefox to rebuild the database from scratch importing from the backup when you choose to clear everything. In that case all the bookmarks would get the same score, that may explain this issue. I'll have to test what it does exactly.

My theory is that because history is disabled the oldest bookmarks were pending a ranking recalculation from a long time. We changed code to recalculate in a more consistent way, to not leave entries behind, and because our recalculation gives a full score at the time it happens, and then that score slowly decays with days, it may have set score of ancient bookmarks to the same score of recent ones.

We'll have to provide a fix on Firefox side, that I assume will be close to what we were already planning to do in the future, give unvisited bookmarks a score that depends on when they were created.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1618602

Comment 12

[seko.idiootti]

I know CCleaner is bad, but running it is the only way to get bookmarks to work how they always did, so I don't really have a choice. The last time it happened, I tried individually clearing all the other things CCleaner can clear (so just "Firefox cache" was left) and it didn't help, but even then clearing the remaining "Firefox cache" didn't fix it... BUT weirdly then clearing everything does, as in it still clears "Firefox cache" after it's already been cleared; like, there's always some "Firefox cache" left unless EVERYTHING is cleared. So, the "Firefox cache" that CCleaner clears if it's selected to be cleared isn't all the "Firefox cache" that it actually clears. No idea what that's about, or how to set it to just clear all of Firefox's cache without clearing everything else... but that's not a Firefox problem, it's a CCleaner problem.

Now that for some reason this bug has been tagged "wontfix" (even though Marco Bonardo said something like a fix would be made even without this bug report?), I guess I really have no choice but to run CCleaner every day. Inconvenient and possibly bad for my laptop's lifespan, but eh. The rules say no complaining, so I'm not going to complain. It just seems weird that one person says it'll be fixed and then another decides it won't be with no explanation.

Comment 13

[Marco Bonardo [:mak]]

wontfix only refers to Firefox 110, we can't fix this bug in an already released version. We'll look for a fix in the next versions.

Comment 14

[seko.idiootti]

Ah, I see! Thanks for the explanation. Sorry, I didn't realise it was only about 110...

Anyway, for now whenever it happens, I'll still try different settings in CCleaner about what things to clear from Firefox if that could maybe help narrow down the exact cause, as unlikely as it is since it seems like it's "cache" even though clearing that (when selected) doesn't do anything about it. At least it's 100% (99.999999%?) certainly not about bookmark backups since it still happens even with them disabled.

Comment 15

[Marco Bonardo [:mak]]

Attachment: Bug 1817248 - Use an appropriate frecency bucket for unvisited bookmarks. r=daisuke

Rather than giving all bookmarks the same ranking score (differentiating only
by typed), retrieve the most recent bookmark date, and use an appropriate
frecency age bucket.
This should ensure more recently created bookmarks are scored higher.

Comment 16

[Marco Bonardo [:mak]]

Comment on attachment 9319509 [details]
Bug 1817248 - Use an appropriate frecency bucket for unvisited bookmarks. r=daisuke

Beta/Release Uplift Approval Request

• User impact if declined: All the unvisited bookmarks get the same ranking. Now that we're more consistent with recalculating the ranking, the order of bookmarks in profiles with a disabled history is surprising users. This patch makes so that older bookmarks get a lower ranking.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce: This is not trivial to replicate manually, since it requires to create bookmarks in the past (feasible only through code snippets).
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This only affects unvisited bookmarks.
• String changes made/needed:
• Is Android affected?: No

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/77b788c7cf2e
Use an appropriate frecency bucket for unvisited bookmarks. r=daisuke

Comment 18

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/77b788c7cf2e

Comment 19

[Donal Meehan [:dmeehan]]

Comment on attachment 9319509 [details]
Bug 1817248 - Use an appropriate frecency bucket for unvisited bookmarks. r=daisuke

Approved for 111.0b7

Comment 20

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/efc3c466e619