1253698 - Old things remembered and listed before recent things

Status: RESOLVED DUPLICATE of bug 1239708

(Firefox :: Address Bar, defect)

Description

[Peter Bengtsson [:peterbe]]

Attachment: Screen Shot 2016-03-04 at 2.44.58 PM.png

See attached screenshot. I start typing "10." and it suggests (ignoring beyond the first two):

* 10.0.0.118/
* 10.0.0.118:8080/

The former isn't working. I hit it once or twice before. The right one is the latter (10.0.0.118:8080/). That's one I've visited several times in recent days/weeks. The first one (10.0.0.118/ I haven't loaded in many weeks)

What I (have to) do next is that instead of typing the whole URL I hit the down key to highlight the second suggestion. Then I hit Enter. All is well.

The problem is that I don't want to have to press the down arrow. I don't want a URL matched to show higher than one I actively use.

Comment 1

[Peter Bengtsson [:peterbe]]

Attachment: Screen Shot 2016-03-04 at 2.49.34 PM.png

It's nothing unique to the URL mentioned in comment #0. Here's another one. 

I start typing "what" and it suggests a domain I haven't used in months. That's really annoying. 

Result number 2 and result number 4 I visit a several times every week and have been done so for some time.

Comment 2

[Panos Astithas (he/him) [:past] (please ni?)]

I've seen this too with my router admin pages and in the past I assumed bug 737836 was at play. Marco, is there a reason for ranking the top-level URL higher than the one with the highest frecency score?

Comment 3

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

(In reply to Peter Bengtsson [:peterbe] from comment #0)
> The former isn't working. I hit it once or twice before. The right one is
> the latter (10.0.0.118:8080/). That's one I've visited several times in
> recent days/weeks. The first one (10.0.0.118/ I haven't loaded in many weeks)

The fact you typed it in even just once is enough for us to consider it a typed host and thus autoFill it.
You should be able to just remove that wrong entry from history.

> What I (have to) do next is that instead of typing the whole URL I hit the
> down key to highlight the second suggestion. Then I hit Enter. All is well.

The main problem here is bug 764062, this should probably be duped to that. Do you agree?

(In reply to Panos Astithas [:past] from comment #2)
> I've seen this too with my router admin pages and in the past I assumed bug
> 737836 was at play. Marco, is there a reason for ranking the top-level URL
> higher than the one with the highest frecency score?

There is not just one frecency value globally, there is an implicit ordering in autocomplete providers, then internally each provide has its own frecency.
By design we "autofill to the next slash", so hosts always have priority over urls. We also keep hosts sorted by frecency clearly, but due to bug 764062, hosts are lacking ports, so we end up suggesting the host without port.

Comment 4

[Peter Bengtsson [:peterbe]]

Panos, Marco,

No the issue has nothing to do with ports or slashes. Don't be fooled by the particular values in the first screenshot. The point is, it suggests things I haven't used in ages when there are good suggestions available that I have visited recently. 

The sorting function is too naive. It doesn't take "history age" in mind. After all, the reason they show up is because of history but I fear the sorting within those found is entirely based on how it's string matched or something. 

I.e. if I type "trump" in Google, if two documents have very similar PageRank and keyword locations, I'll want to be shown the more recent one.

Comment 5

[Daniel Veditz [:dveditz]]

(In reply to Peter Bengtsson [:peterbe] from comment #4)
> The sorting function is too naive. It doesn't take "history age" in mind.
> After all, the reason they show up is because of history but I fear the
> sorting within those found is entirely based on how it's string matched or
> something. 

The sorting function, as alluded to in comment 2, is called "frecency" because it _does_ take into account "history age" ("recency"), as well as how many times you've visited ("frequency"). I think it takes even more into account now but they kept the name.

This algorithm is letting you down, but they're going to need more detailed data to know why because it is not internally as simplistic as you imply. If you go into your "Library" you can turn on columns like "last visited" and "visit count". Or maybe you could send your places database to someone so they could debug why you get the ordering you see for a particular testcase and whether there's any reasonable tweaks to the heuristic that would get closer to the order you expect. And then comes the fun of trying to decide whether those tweaks actually improve the results for most people or if you're an outlier and this makes it overall worse.

Comment 6

[Peter Bengtsson [:peterbe]]

(In reply to Daniel Veditz [:dveditz] from comment #5)
> (In reply to Peter Bengtsson [:peterbe] from comment #4)
> > The sorting function is too naive. It doesn't take "history age" in mind.
> > After all, the reason they show up is because of history but I fear the
> > sorting within those found is entirely based on how it's string matched or
> > something. 
> 
> The sorting function, as alluded to in comment 2, is called "frecency"
> because it _does_ take into account "history age" ("recency"), as well as
> how many times you've visited ("frequency"). I think it takes even more into
> account now but they kept the name.
> 
> This algorithm is letting you down, but they're going to need more detailed
> data to know why because it is not internally as simplistic as you imply. If
> you go into your "Library" you can turn on columns like "last visited" and
> "visit count". Or maybe you could send your places database to someone so
> they could debug why you get the ordering you see for a particular testcase
> and whether there's any reasonable tweaks to the heuristic that would get
> closer to the order you expect. And then comes the fun of trying to decide
> whether those tweaks actually improve the results for most people or if
> you're an outlier and this makes it overall worse.

Thanks! 
This isn't easy stuff to get right since it depends on guesswork of what users "just want". My original point comes straight from the gut and it says I'm often disappointed. 

My guess is that the "frequency" trumps the "history age" too much.

Where can I find what the formula for this is? (assuming I understand the code) On Air Mozilla, the list of trending events uses a similar formula as Hacker News's home page which is "number of hits" / "age in days" ** 1.8 [0]. It basically gives a score whereby the "freshness" significantly trumps the popularity.

I'd love to create some reports based on my places database for people to debug if it's done in a PII respecting and secure way. Please please ping me if you want to hack on it. 

[0] https://github.com/mozilla/airmozilla/blob/1637bc162a104caf36dc00ee42951db6acd9fda6/airmozilla/main/context_processors.py#L328-L329