720258 - Inline autocomplete should only autocomplete URLs you've typed

Status: RESOLVED FIXED

(Firefox :: Address Bar, defect)

Description

[Justin Lebar (not reading bugmail)]

There's already anger over inline-autocomplete, and I don't think that's going to go away as the change gets rolled out to more users.  See e.g. bug 720240.

One of the perceived problems is that inline-autocomplete does not always autocomplete to the first hit in the awesomebar.  It usually does, which makes it particularly flummoxing when it doesn't.

We may be able to mitigate this issue by making inline-autocomplete autocomplete only domains you've actually typed into the awesomebar.  If I've never typed a link to a given domain, it shouldn't be autocompleted.

Note to commentors new to bugzilla: If you want this change made, probably the worst thing you can do is post comments about how much bug 566489 in its current form ruins your Firefox workflow.  Bugzilla is not a discussion forum, and contributors often shy away from fixing bugs with angry participants.

Comment 1

[Mardeg]

So will this be the equivalent of setting both browser.urlbar.match.url and browser.urlbar.restrict.typed to empty strings to get identical results in the dropdown?

Comment 2

[DavidC]

(In reply to Justin Lebar [:jlebar] from comment #0)
> There's already anger over inline-autocomplete...

Yes, I detected some defensiveness which was quite puzzling.

> One of the perceived problems is that inline-autocomplete does not always
> autocomplete to the first hit in the awesomebar.  

It is not "perceived". It really is a problem that reduces usability.

> We may be able to mitigate this issue by making inline-autocomplete
> autocomplete only domains you've actually typed into the awesomebar.  

So URLs visited from e.g. a link on a page will not autocomplete? This sounds like it will confuse users even more. It will become a mystery as to why some domains appear and some do not.

And none of this fixes the key issue that the Awesomebar just got a lot dumber by potentially putting a URL in the address bar that has been visited once, when the user really wanted to see the URL they have visited 1000 times.

> ...contributors often shy away from fixing bugs with angry participants.

And defensive, angry contributors often make participants wonder why some contributors can't just discuss the bug without trying to introduce emotive language and personalise the debate.

Comment 3

[Justin Lebar (not reading bugmail)]

David,

I'm really sorry if you feel like I'm being defensive and angry.  I think it's fair to say that bug 720240 reflects "anger" on the part of users (you and others), but I'm sorry if you feel like I've misrepresented your tone.  By "perceived problem", I didn't mean to suggest that your problem is non-existent or invalid -- in fact, I think it's a completely valid complaint -- rather, I just meant that it's a point of confusion for some people.

However, I'd really like to keep the discussion in this bug on topic and, crucially, technical, because I'd like this bug to get the attention it deserves.  That's what I was trying to get at in the last paragraph of comment 0, although I agree I could have phrased it better.  I hope you'll try to give me the benefit of the doubt going forward so we can concentrate on what's important here.

> And none of this fixes the key issue that the Awesomebar just got a lot dumber by potentially 
> putting a URL in the address bar that has been visited once, when the user really wanted to see 
> the URL they have visited 1000 times.

This is covered by bug 720240, right?

> So URLs visited from e.g. a link on a page will not autocomplete? This sounds like it will confuse 
> users even more. It will become a mystery as to why some domains appear and some do not.

I can imagine this being true in some cases.  For example, maybe I visit nytimes.com many times a day, but always via a link on Google News.  Then I might be confused when "nytimes.com" doesn't inline-autocomplete.

On the other hand, it's precisely those sites I *type* all the time that should be inline-autocompleted.  If I've never typed a domain before, it doesn't seem particularly strange to me that it shouldn't be inline-autocompleted.  Remember that something else probably *will* be inline-autocompleted, and if that's something I have typed, it may seem quite natural for it to come up.

It's not a perfect heuristic by any means; the question is whether it's an improvement over what we have now.

Comment 4

[Marco Bonardo [:mak]]

I don't think making it return only typed entry is going to make it any better.

First it would not ensure that those entry match the first entry in the popup any way better than the current system, so if it's intended as a way to mitigate that it will just fail. Surely typed entry may be a bit higher in the popup, but that's not going to work with adaptive entries.

Second it would dumb down the usefullness of inline autocompleting, that also allows to avoid typos, to go faster to the main page of a website when you just have visited sub-articles, to give you more probably what you were tring to get.

I have an alternative suggestion that may be to add adaptiveness to the results. e.g. when I typo "en.", inline resolves to en.wikipedia.org, but I wanted en.othersite.org and I actually select it from the popup, we can store a bonus so that next time it will suggest the domain I selected. This still doesn't solve the "not the first entry" issue, but would adapt to the user's needs.

Comment 5

[Justin Lebar (not reading bugmail)]

> First it would not ensure that those entry match the first entry in the popup any way better than 
> the current system, so if it's intended as a way to mitigate that it will just fail.

Agreed, and that means comment 0 is wrong.  However, I now think the purpose of this bug is to reduce  "WTF is this website" moments and make inline-autocomplete more predictable.  This bug would reduce the number of candidate completions, which can only improve predictability.  See bug 566489 comment 13 for the importance of predictability.

> Second it would dumb down the usefullness of inline autocompleting, that also allows to avoid 
> typos, to go faster to the main page of a website when you just have visited sub-articles, to give 
> you more probably what you were tring to get.

I think it's wrong that this change would "reduce the probability of getting what I was trying to get."  In fact, I think it can only increase that probability.  Here's why:

Typed domains should follow a power law [1].  That is, we expect roughly 20% of the URLs you visit make up 80% of the URLs you type, because you go to some sites all the time.  So as soon as you type a domain even once, the probability that you're going to type it again increases dramatically; conversely, if you never type a domain but have only visited it before, it's relatively unlikely that you'll ever type it.

The power law suggests that if we get inline-autocomplete working well for the relatively few sites you type all the time, we've gotten it to work for the vast majority of cases.

Let me present the reverse question to you: Why is it important that we auto-complete to URLs which have never been typed?  Why should we optimize for the first time you type a URL at the expense of a much more common activity, typing a URL you *have* typed before?

[1] http://en.wikipedia.org/wiki/Power_law

Comment 6

[Justin Lebar (not reading bugmail)]

Marco, neither you nor I is a UI person, so perhaps neither of us is qualified to make this decision.  Can we please get some advice from the UI team on this thread?  I think this is critical to get right.

Comment 7

[Marco Bonardo [:mak]]

(In reply to Justin Lebar [:jlebar] from comment #5)
> Typed domains should follow a power law [1].  That is, we expect roughly 20%
> of the URLs you visit make up 80% of the URLs you type, because you go to
> some sites all the time.

Sure, though we should not only concentrate on technical users who type domains the locationbar, most users don't even know what's a domain and type them in the google search field because when they type in the locationbar they get a lot of entries in the popup that confuse them (none of them is the website they are looking for, most will be subpages, articles they reached through a search egine.

(In reply to Justin Lebar [:jlebar] from comment #6)
> Marco, neither you nor I is a UI person, so perhaps neither of us is
> qualified to make this decision.  Can we please get some advice from the UI
> team on this thread?

Absolutely, note that the current behavior was actually tested by UX team, though clearly having it in product changes cards on the table, so we can improve further.

Comment 8

[Michael Ventnor]

When limi designed the UX of this feature, he explicitly mentioned wanting to be able to complete URLs you haven't necessarily typed. Without this requirement then implementing the autocomplete would've been easier :P

Comment 9

[Alex Limi (:limi) — Firefox UX Team]

We should complete URLs that haven't been typed (since that's part of what makes the feature useful), but the scoring could be improved, and keep track of what the most commonly typed domains for the set of given letters. 

This "typing" would also include the acceptance of a suggested autocomplete — e.g. if I've never typed cnn.com and get that as a result when typing C, then accept it by hitting enter, I've effectively typed it.

I'm not saying the power law has no merit, but I think we can do even better — the internet has a pretty long tail. :)

Comment 10

[Justin Lebar (not reading bugmail)]

> I'm not saying the power law has no merit, but I think we can do even better — the 
> internet has a pretty long tail. :)

The Internet has a much longer tail than the set of URLs I type into my awesomebar.  In fact, I'd bet that over the course of a month, I type fewer than 50 different URLs, while I visit thousands.  Optimizing inline-autocomplete for anything other than this tiny set seems incorrect to me, because prima facia, the set of URLs I type (or select out of the awesomebar) is the set of URLs I want to go to.

I think the principal should be "if you don't have anything smart to say, don't say anything at all."  It's not a matter of tuning the heuristic for ranking autocompleted URLs, but rather tuning the heuristic to *exclude* URLs from the list of autocomplete candidates, so as to avoid making ourselves a punching bag over users' WTF moments.

After all, as you pointed out, the set of domains I've visited has a long tail, whereas the set of sites I want to go to is tiny.

I feel compelled to point out that AIUI Chrome uses precisely the heuristic I'm suggesting here for their inline-autocomplete.  Not to imply that they're the font of everything good in the world, but it's clear that they have a lot of experience tweaking this feature.

But it sounds like this bug should be WONTFIX'ed and limi should file a new bug detailing how he wants to change the heuristic.

Comment 11

[Justin Lebar (not reading bugmail)]

> Optimizing inline-autocomplete for anything other than this tiny set seems incorrect to 
> me, because prima facia, the set of URLs I type (or select out of the awesomebar) is the 
> set of URLs I want to go to.

I guess this is ipso facto, not prima facie.

Comment 12

[Marco Bonardo [:mak]]

We are likely going to do this for now, to reduce false positives hit ratio.

Comment 13

[Marco Bonardo [:mak]]

Attachment: wip

Comment 14

[Marco Bonardo [:mak]]

Attachment: patch v1.0

Comment 15

[Justin Lebar (not reading bugmail)]

Not to suggest I don't approve of this change, but I thought limi said we weren't doing this?

Comment 16

[Marco Bonardo [:mak]]

We discussed a bit possible approaches, and decided to go for this to limit the number of false positives. Personally I still prefer the background autocomplete approach (requiring RIGHT or TAB to complete) but I'm not against testing this behavior, can be toggled easily with the pref.

Comment 17

[Marco Bonardo [:mak]]

Attachment: patch v1.1

I messes up the parenthesis in the test, this fixes them.

Comment 18

[Dietrich Ayala (:dietrich)]

Comment on attachment 597890 [details] [diff] [review]
patch v1.1

Review of attachment 597890 [details] [diff] [review]:
-----------------------------------------------------------------

looks good, r=me

Comment 19

[Marco Bonardo [:mak]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/095d7986dfd0

Comment 20

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/095d7986dfd0

Comment 21

[u428464]

I hope Firefox still autocomplete bookmarks, because Chrome doesn't do it and it's very annoying.

Comment 22

[Marco Bonardo [:mak]]

you can flip the browser.urlbar.autoFill.typed pref to change the behavior.

Comment 23

[u428464]

I know, but it should be the default behaviour that Firefox both autocompletes URL you've typed and bookmarked ones.

Comment 24

[Marco Bonardo [:mak]]

Please file a separate bug for evaluation, that said don't think may happen shortly since it's expensive without caching bookmarked status.

Comment 25

[u428464]

I've filed (I hope correctly) bug 731560.

Comment 26

[Daniel C]

Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120313 Firefox/13.0a1

This feels much nicer having used it for a few hours today, thank you. I'm seeing inline autocompletions for the URLs I commonly type, and no inline autocompletion for other URLs (as opposed to before which inlined a suggestion for somewhere I've never typed before but possibly visited once upon a time).

The one problem I have is still bug 720081. For domains which don't have a non-www domain it leads to an error page, and for domains which don't redirect their non-www domain to the www one then you end up bypassing the cache and messing up visited links etc. (because Firefox sees it as a new page).

Comment 27

[vp2177+mozbugz]

I can't believe why isn't the (correct, tried & tested) behaviour of every other browser is cloned in Firefox.
This clearly is inferior. Being primarily a Firefox user, I instantly found Safari's URL autocomplete much more intuitive.

Comment 28

[Marco Bonardo [:mak]]

Peter, you are free to use other browsers, we never enforced anything on you, differently from others. Please keep constructive in comments (your comment doesn't contain any useful suggestion) and avoid commenting in resolved bugs, file your own bug.