1464272 - Awesomebar suggestions for "fa" are entirely made up of DuckDuckGo search results pages from other devices

Status: RESOLVED FIXED

(Firefox :: Address Bar, enhancement, P1)

Description

[Cameron McCormack (:heycam)]

Attachment: screen shot

If I type "fa" into the address bar, the first suggestion is to visit fastmail.com, and the remaining suggestions are all tabs from another device that are DuckDuckGo results pages.  I realized that this is because the URLs for those pages all include "&t=ffab" in them (indicating the search came from the Firefox address bar).

Is there some way we can avoid these pages from being included in the Awesomebar suggestions?

Comment 1

[Marco Bonardo [:mak]]

This is an interesting report, some time ago we moved remote tabs to the top, and they don't have a limit, so this is a case where you typed a string that returns enough remote tabs to push away any other local result.
Remote tabs don't have a sorting, afaict.

I think the bug here is that we should figure out some score for remote tabs (for example matching in the query string could be considered less important than matching in the title or in the host) and we should maybe limit the number of remote matches we pick, unless the user types the restriction character for tabs ("%", since there's no specific char for Remote Tabs).

Mark, any insight?

Comment 2

[Mark Hammond [:markh] [:mhammond]]

(In reply to Marco Bonardo [::mak] from comment #1)
> This is an interesting report, some time ago we moved remote tabs to the
> top, and they don't have a limit, so this is a case where you typed a string
> that returns enough remote tabs to push away any other local result.

I can't find the bug where the limit was removed - the tl;dr of the rest of this comment is that IMO, those changes are the bug here. I'm glad this bug was opened, because I've personally noticed the urlbar has become less useful recently but never filed anything.

> Remote tabs don't have a sorting, afaict.

There is a lastUsed timestamp for a tab which isn't currently used, but could be if desired.

> I think the bug here is that we should figure out some score for remote tabs
> (for example matching in the query string could be considered less important
> than matching in the title or in the host) and we should maybe limit the
> number of remote matches we pick, unless the user types the restriction
> character for tabs ("%", since there's no specific char for Remote Tabs).

I'm inclined to agree that stuff in the query string probably shouldn't match (at least not match as "strongly" as the rest of the URL), but IIUC, this happens for *all* URLs. For example, if I type "id" into the address bar, it gets filled with bugzilla results from my history (the ?id=XXXXXXX part of the URL). If I have 10 bugzilla tabs open, typing "id" fills the bar with "switch to tab" results. If a different device has 10 bugzilla tabs open, "id" matches those, as reported here. IMO, *none* of those use-cases should dominate the results and SyncedTabs itself isn't special in that regard.

IOW, it seems that if Cam was on that other device, there's a good change that typing "fa" would end up being dominated in exactly the same way (either as "switch to tab" entries or from history)

(More generally, I think there's a great opportunity for us to look at a more holistic review of the awesome bar - eg, the ability to say "7 more [remote tabs/local tabs/history entries] match" with a UI affordance for expanding them - but I understand that's a much larger job.)

Cam's screen-shot is more pronounced as IIUC, he appears to have "Show search suggestions in address bar results" disabled, meaning he gets all 10 entries matching, where users who do see search results see 5 matches, then 5 search results. As an experiment, I bumped browser.urlbar.maxRichResults to 20 and the results were disappointing - you now see all 20 results being dominated by a single match type (with search results disabled), or see 10 matches and 10 search results (with search results enabled).

So my tl;dr is that the problem is caused by a combination of:
* Matching the query params for *any* URLs when other "stronger" matches exist.
* Having no limit on some of these results, which is even more obvious when there are more slots available either due to disabling "Show search suggestions in address bar results" or by modifying browser.urlbar.maxRichResults.
* No UI in about:preferences for disabling this feature entirely (which is maybe all that Cam is actually after?)

Comment 3

[Cameron McCormack (:heycam)]

(In reply to Mark Hammond [:markh] from comment #2)
> * No UI in about:preferences for disabling this feature entirely (which is
> maybe all that Cam is actually after?)

If by "this feature" you mean showing tabs from remote devices, I don't mind that, and it's sometimes useful to see tabs from other devices suggested.  But generally it's much less likely that I'm wanting to find a remote tab rather than visit a site that I often visit, or switch to an existing tab on the same device.

Comment 4

[Marco Bonardo [:mak]]

(In reply to Mark Hammond [:markh] from comment #2)
> I'm inclined to agree that stuff in the query string probably shouldn't
> match (at least not match as "strongly" as the rest of the URL), but IIUC,
> this happens for *all* URLs.

True, but other urls have the adaptive or frecency advantage. So they appear high only if they are visited recently and often.
It's not the case for remote results.

> * No UI in about:preferences for disabling this feature entirely (which is
> maybe all that Cam is actually after?)

Actually, open tabs controls both switch to tab and remote tabs, I think.
It's still a workaround though.

Comment 5

[Mike de Boer [:mikedeboer]]

(In reply to Mark Hammond [:markh] from comment #2)
> So my tl;dr is that the problem is caused by a combination of:
> * Matching the query params for *any* URLs when other "stronger" matches
> exist.

This made me think: do we have qualitative weighted matching in place to complement our frecency scoring algorithm? I mean, a hit when a substring matches is one thing, but to have a higher score when the domain part is matched vs. the query part or the page title might be good fallback when there's no frecency data available. Is there a spec for that and if not, would it be worth making one?

> * No UI in about:preferences for disabling this feature entirely (which is
> maybe all that Cam is actually after?)

Last resort option to think about and implement, IMO. Tabs from other devices are simply yet another source of URLs to autocomplete and there's only going to be more of those in the future, not less.

Comment 6

[Marco Bonardo [:mak]]

(In reply to Mike de Boer [:mikedeboer] from comment #5)
> This made me think: do we have qualitative weighted matching in place to
> complement our frecency scoring algorithm? I mean, a hit when a substring
> matches is one thing, but to have a higher score when the domain part is
> matched vs. the query part or the page title might be good fallback when
> there's no frecency data available. Is there a spec for that and if not,
> would it be worth making one?

We have nothing like that, and it should probably be part of the rearchitecture. In a world where we have multiple providers give their results with a score and a muxer putting them together, it may be useful to have a "hint" of what was matched.

Comment 7

[Marco Bonardo [:mak]]

I'll see if we can easily reorder and limit remote tabs. An approach similar to the one I recently used in bug 1464454 could do (that is, insert a few entries, cache the others, if no other matches are found reuse the cached ones).

Comment 8

[Mark Hammond [:markh] [:mhammond]]

My previous reply probably could have been more helpful :) While it wouldn't solve every use-case, it might still make pragmatic sense to *also* only match remote tabs on the host and path portions of the URL as you suggested above - I can't believe people actually expect matches on the query portion, but also limiting/caching the matches still makes sense so, eg, "bug" still doesn't only match remote bugzilla tabs.

Comment 9

[Marco Bonardo [:mak]]

There's another thing, imo we should limit top remote tabs suggestions by time, I just got a suggestion from a tab I opened weeks ago, and probably I didn't close it. Since it's so old, it doesn't really help me continuing my workflow. Showing first the tabs opened recently may be a good idea in general.
So my current proposal is that we add tabs opened recently (we must define what the timeframe is, 48 hours, 72 hours, 1 week?), and add the others only if there's available space (no better matches found elsewhere).

I'd probably leave the query_string problem to the future, it sounds like it would be useful for any kind of match, not just remote tabs.

Comment 10

[Marco Bonardo [:mak]]

Attachment: Bug 1464272 - Remote Tab matches may dominate the address bar results.

Review commit: https://reviewboard.mozilla.org/r/249974/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/249974/

Comment 11

[Marco Bonardo [:mak]]

note, this doesn't address the query_string VS other problem, for that I'll file a separate bug to evaluate a global frecency bonus change.

Comment 12

[Lina Butler [:lina]]

Comment on attachment 8984175 [details]
Bug 1464272 - Remote Tab matches may dominate the address bar results.

https://reviewboard.mozilla.org/r/249974/#review256290

Awesome, thanks!

::: toolkit/components/places/PlacesRemoteTabsAutocompleteProvider.jsm:122
(Diff revision 1)
> -    let { tabs, clients } = ensureItems();
> -    for (let [url, { clientId, tab }] of tabs) {
> +    let tabsData = await ensureItems();
> +    for (let {tab, client} of tabsData) {
> +      let url = tab.url;
>        let title = tab.title;
>        if (url.match(re) || (title && title.match(re))) {
>          // lookup the client record.

This comment can probably go, too.

Comment 13

[Marco Bonardo [:mak]]

Comment on attachment 8984175 [details]
Bug 1464272 - Remote Tab matches may dominate the address bar results.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/249974/diff/1-2/

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e207930f9e8a
Remote Tab matches may dominate the address bar results. r=kitcambridge

Comment 15

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/e207930f9e8a