1610718 - "www" should be excluded from URL suggestions and autofill when user types "w"

Will

Reporter

Description

•

4 years ago

Attached image w urlbar.png — Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0

Steps to reproduce:

Simply type "w" into the empty URL bar and I'm sure Firefox autofills a website starting with "www" (in my case, it's www.reddit.com). This is not right and Firefox should be more intelligent than this. The URL Firefox is autofilling (www.reddit.com) doesn't start with a "w", it starts with an "r", it should not be suggesting www.reddit.com when I've only typed "w". When a user types "w" into the URL bar, they should either be getting URL's that start with "w" (such as walmart.com or wikipedia.org etc.) or site titles that start with "w" like "Wikipedia, the free encyclopedia".

Expected results:

screenshot attached I've what I'm seeing

Will

Reporter

Comment 1

•

4 years ago

I forgot to add: Firefox already does something like this for "h". When I type "h" into the empty URL bar, I do NOT get autofill for a URL that starts with "https://" or "http://" , even though basically every URL starts with that. This behaviour should be extended to "www."

Gingerbread Man

Updated

•

4 years ago

Component: Untriaged → Address Bar

Updated

•

4 years ago

Status: UNCONFIRMED → NEW

Ever confirmed: true

Priority: -- → P3

Drew Willcoxon :adw

Comment 2

•

4 years ago

That's debatable. It's easy to imagine people typing "w" in order to to visit www.facebook.com.

Will

Reporter

Comment 3

•

4 years ago

(In reply to Drew Willcoxon :adw from comment #2)

That's debatable. It's easy to imagine people typing "w" in order to to visit www.facebook.com.

Yes and it's been debated a lot over the years but there does have to be a line drawn at some point: what is correct and what is incorrect. If someone got used to typing "w" to go to facebook, that's wrong because it makes no sense. It was an unintended quirk that they user got used to. "Facebook" doesn't even have a "w" in it! If they want to go to facebook, they should type "f", this is just as fast/efficient as typing "w" AND it actually makes sense, that is how Firefox should work. If we let scenarios like the one you mentioned dictate Firefox, Firefox would be a complete mess, it would be madness. I'm well aware that people really despise changes but that shouldn't discourage us from improving Firefox in ways like this.

Also, we have to remember that there are websites that start with "w" (walmart.com) that people are wanting and expecting to autofill when they type "w", but instead they're getting autofills such as www.google.com, www.facebook.com etc. And I'm sure those websites aren't happy about that either!

Will

Reporter

Comment 4

•

4 years ago

Also I should change the title of this bug because, if someone types 2 double-ues ("ww") into the URL bar, of course "www.facebook" etc. should autofill, unless there are URL's in their history/bookmarks such as wwe.com or wwf.ca. Then those should autofill before www.facebook.com, www.google.com etc.

Marco Bonardo [:mak]

Updated

•

4 years ago

Blocks: qb-results-papercuts

Marco Bonardo [:mak]

Updated

•

4 years ago

Points: --- → 2

Will

Reporter

Comment 5

•

4 years ago

This should also apply to Fenix (same with typing "h" resulting in "https://example.com" being autofilled. Firefox desktop already fixed the "h" scenario a while ago though)

Harry Twyford

Updated

•

4 years ago

Assignee: nobody → htwyford

Status: NEW → ASSIGNED

Iteration: --- → 76.1 - Mar 9 - Mar 22

Harry Twyford

Comment 6

•

4 years ago

I discussed this bug with Drew yesterday and its scope is larger than expected. The moz_origins table we use for autofill splits URLs up into prefix and host, where prefix is the scheme, a colon, and possibly two slashes, and host is the rest, possibly including www.. The autofill SQL query searches the host field. This is how we don't search for http when the user types "h".

Drew initially suggested modifying the SQL query to add something like this:

WHERE (host BETWEEN :searchString AND :searchString || X'FFFF') OR
      (
        substr(host, 0, 4) = 'www.' AND
        substr(host, 4) BETWEEN :searchString AND :searchString || X'FFFF'
      )

But he pointed out this would simply search for www.site.com and site.com for every autofill query. This means www.site.com might still be autofilled. To not search www., we'd need to strip www. from the host field in moz_origins. That means typing www.mozilla.org would never autofill www.mozilla.org. Instead, you’d have to type mozilla.org to autofill mozilla.org instead. This seems undesirable.

There's probably a middle path with a more specific implementation. For example, if one types "w”, we could prefer matching domains that start with w after stripping www. from every domain, but fall back to matching www.. This needs some discussion and an engineering definition. I'm un-assigning myself, pending discussion.

Marco, do you have any thoughts on how to handle this?

Assignee: htwyford → nobody

Status: ASSIGNED → NEW

Iteration: 76.1 - Mar 9 - Mar 22 → ---

Flags: needinfo?(mak)

Will

Reporter

Comment 7

•

4 years ago

Should we retitle this just to be clearer? Maybe "(URL bar autofill) domains/origins should always be prioritized over "www" when user types "w".

Like I said in comment 4, if someone types 2 double-ues (and they've never visited a site that starts with 2 or 3 w's, such as wwe.com), they should continue to get autofills like www.google.com or www.facebook.com etc.

Marco Bonardo [:mak]

Comment 8

•

4 years ago

•

Edited

(In reply to Harry Twyford [:harry] from comment #6)

I discussed this bug with Drew yesterday and its scope is larger than expected. The moz_origins table we use for autofill splits URLs up into prefix and host, where prefix is the scheme, a colon, and possibly two slashes, and host is the rest, possibly including www..

We have a similar problem in bug 843357, we should start investigating the removal of www from both, but that needs an analysis to understand possible drawbacks. That discussion may also depend on the outcome of the stripWww pref. Mostly we should ensure in cases where those 2 origins differ, the user has still ways to make a choice in the urlbar.
This requires checking how rev_host and moz_origins are used today across the codebase, and marking down every behavior that'd change. If those changes are acceptable (for example grouping by host in history UI without www would be acceptable, forcing the user to go the non-www version when it doesn't exist wouldn't be acceptable) then we could proceed with a schema migration.
It requires some design effort but it should be feasible, and then we'd have all the data we need to decide.

There's probably a middle path with a more specific implementation. For example, if one types "w”, we could prefer matching domains that start with w after stripping www. from every domain, but fall back to matching www.. This needs some discussion and an engineering definition. I'm un-assigning myself, pending discussion.

I'm not sure how you'd do that, in particular the stripping for every domain would be far too expensive, we rely on the data store capabilities here, we can only match at the beginning of the url/origin string, not in the middle of it.
I wonder if bug 1466233 would help here, but we should change the input history table contents, in particular split "scheme://www." into a separate column. That is sort of planned anyway if we want to use that data for autofill.

Bumping up the points value here, keeping it as a P3 but requires some deeper refactoring, so not immediately actionable.
Thanks for the initial investigation.

Points: 2 → 8

Depends on: 1466233, 843357

•

3 years ago

(In reply to Will from comment #9)

that way, when the user types "w" into the URL bar, https://en.wikipedia.org/ would autofill. It really does feel like this is how it should behave, I hope it's possible.

Technically it would not be impossible,there's a few risks to consider because urls can be built in very fancy ways. It could probably be done for a subset of cases (lang-code.domain), but it's even more complex than www. One step at a time.

(In reply to Will from comment #11)

Oh wow, I'm really worry for posting yet again because I know this one isn't a high priority but I had a weird config issue and I noticed a bug that proves this issue really does need to get fixed (tab-to-search wasn't working for me at all, it's working now). Now, I type "w", "www.google.com/" autofills. Then I hit "Tab" and Wikipedia tab to search comes up! That's a really confusing issue, see attached screenshots

Hm, that looks indeed strange, but unrelated to this bug.
I suspect it's a regression from recent results composition work, please file a separate bug so we can fix it.

Flags: needinfo?(billdillensrevenge)

Will

Reporter

Comment 13

•

3 years ago

But I think it IS related to this bug. If both of these issues are fixable (comment #0 and comment #9 too), it would autofill "https://en.wikipedia.org/" when you enter "w". That would solve the issue I brought up in comment #11

Flags: needinfo?(billdillensrevenge)

Marco Bonardo [:mak]

Comment 14

•

3 years ago

This bug is a lot more complex to fix and we don't have the time to do it now. In comment 11 the tab to search result for wikipedia should not be there, because wikipedia was not autofilled, we should fix that bug and still have this open for the future.

Will

Reporter

Comment 15

•

3 years ago

About the comment #11 issue, maybe ask the people that worked on tab-to-search how this is supposed to work. I assumed they purposely designed it so that, when the user types "w", wikipedia comes up (because it's one of the default search engines included in Firefox).

Harry Twyford

Comment 16

•

3 years ago

Attached image github-google-engine.png — Details

(In reply to Will from comment #15)

About the comment #11 issue, maybe ask the people that worked on tab-to-search how this is supposed to work. I assumed they purposely designed it so that, when the user types "w", wikipedia comes up (because it's one of the default search engines included in Firefox).

It's not supposed to work that way. We were specifically concerned about inundating the user with too many tab-to-search results. We're only supposed to show a tab-to-search result if the engine domain is autofilled. It would be annoying if we showed a Google tab-to-search result every time "g" was typed, but that appears to be almost what we're doing. I can reproduce in a different scenario: when an autofilled site starts with "g", we show a tab-to-search result for Google Drive. Weirdly, if I disable my Google Drive engine, neither Google nor Google Maps (which I have installed as an OpenSearch engine!) shows a tab-to-search result. It's not clear what's broken here.

I ran mozregression and the Wikipedia issue was sort of reproducible on the 2020-10-01 build when tab-to-search was introduced. A key difference being that we didn't yet show a Wikipedia tab-to-search result when "wikipedia.org" was autofilled, because the engine was actually for <locale>.wikipedia.org. I could reproduce the issue after installing an OpenSearch engine that pointed directly to wikipedia.org.

Something weird is happening with the muxer here. I'll file and fix.

Harry Twyford

Comment 17

•

3 years ago

•

Edited

(In reply to Harry Twyford [:harry] from comment #16)

I'll file and fix.

Bug 1713662.

Pablo

Comment 18

•

3 years ago

This affects the Wikipedia tab to search. Testes on nightly 92.0a1
when i type "w" it does not give me the wikipedia tab to search option.

not sure if another bug is needed, please let me know and i will open one.

Drew Willcoxon :adw

Comment 19

•

3 years ago

It's a little confusing, but you have to get en.wikipedia.org (or whatever subdomain matches your Wikipedia search engine) to start autofilling, and then when you start typing "wikipedia", you should get a TTS result. If instead you get wikipedia.org (without the subdomain) to start autofilling, that won't trigger it because there's no wikipedia.org search engine.

i.e., it should work with these steps on a new profile on Nightly:

Type en.wikipedia.org and hit enter (you'll also see the TTS result for en.wikipedia.org during this, just ignore it)
Now start typing wikipedia

You should see the Wikipedia TTS at that point.

Marco Bonardo [:mak]

Updated

•

2 years ago

Keywords: papercut

•

2 years ago

Sorry for the back to back comments but I was just looking around and noticed Chrome people are working on "rich autocompletion" , does Firefox have an equivalent tracking bug for this? https://bugs.chromium.org/p/chromium/issues/detail?id=1062446

Marco Bonardo [:mak]

Comment 26

•

2 years ago

that's a bit off topic, but no, so far we don't plan to provide "in-the-middle" autofill. We had something similar in the past, and it was very confusing for the users, and hard to get it right in code. There is a reason if the Chrome fix is still pending after 2 years, it's not trivial to get that working in a non confusing way for the user. If their new solution works better than what we had we'll surely evaluate it as an idea.

BMO Automation

Updated

•

2 years ago

Severity: normal → S3

Dale Harvey (:daleharvey)

Updated

•

1 year ago

Whiteboard: [snt-scrubbed][snt-write-proposal]

Jira Integration Bot

Updated

•

1 year ago

See Also: → https://mozilla-hub.atlassian.net/browse/SNT-477

Will

Reporter

Comment 27

•

1 year ago

The reason I brought up rich autocompletion in comment 25 is because if Firefox had that, it could make implementing this bug (1610718) possible or at least easier. Of course I don't know exactly how this bug (1610718) would work if implemented 'as is' but there does seem to be a few valid/legit concerns (see comments 23 and 24). Does anyone know if there's a bug or something for Firefox to investigate rich autocompletion? Using rich autocompletion in Chrome Canary, I think it's definitely worth exploring and experimenting but some of the UI/UX decisions the Chrome team have went with are poor, but that doesn't mean the entire idea of rich autocompletion should be thrown in the garbage.

Marco Bonardo [:mak]

Comment 28

•

1 year ago

(In reply to Will from comment #27)

The reason I brought up rich autocompletion in comment 25 is because if Firefox had that, it could make implementing this bug (1610718) possible or at least easier. Of course I don't know exactly how this bug (1610718) would work if implemented 'as is' but there does seem to be a few valid/legit concerns (see comments 23 and 24). Does anyone know if there's a bug or something for Firefox to investigate rich autocompletion?

We have investigated it in the past, but the UX for it seems a bit confusing, we're not convinced it's a net benefit for users in the current status.
We don't need that to fix this bug anyway, it can be fixed on its own.

Will

Reporter

Comment 29

•

8 months ago

Attached image firefoxMautocomplete.png — Details

I'm so sorry to come back to this yet again but this one is biting me yet again: a website I work on every day starts with an "m" (https://www.mEXAMPLE.com) and Firefox is autofiling Gmail when I hit "m"....

Marco Bonardo [:mak]

Comment 30

•

8 months ago

Typing a single letter is unfortunately not a strong enough signal for any system to make a good choice out of hundreds thousands URLs. typing "me" would help discerning. Maybe bug 1823449 will help, you could try enabling browser.urlbar.autoFill.adaptiveHistory.enabled in about:config, though note it's an experimental feature with bugs.

That said this bug is at maximum handling cases like www. or m., not mail. or other subdomains, so it's probably not the best place to report your feedback.

Will

Reporter

Comment 31

•

8 months ago

Marco of course a single letter is strong enough signal, Firefox does this beautifully the vast majority of the time. It's one of the nicest things about Firefox!

I wasn't reporting feedback, it was a ping because this issue pops up a lot. Also in this example, "Gmail" starts with a "g", so Google should have gone with "gmail.com" instead of "mail.google.com". Typing in "m" resulting in Gmail's URL autofilling doesn't make sense. Like I said in comment #3, people getting used to something that's not working the way it should is no reason to keep it that way. Though I admit some URL's do make this issue tricky, such as tumblrs like the one I mentioned above https://siriuslymeg.tumblr.com . Ideally it would be https://www.tumblr.com/siriuslymeg

Marco Bonardo [:mak]

Comment 32

•

8 months ago

(In reply to Will from comment #31)

I wasn't reporting feedback, it was a ping because this issue pops up a lot.

What I'm saying is that his bug won't fix your issue, because this is just about widely used subdomains like www. and m. that don't change the expected destination. On the other side mail.google.com is very different from google.com. So you should file your request as a separate bug.

Also in this example, "Gmail" starts with a "g", so Google should have gone with "gmail.com" instead of "mail.google.com".

There's a lot to unwrap here, included the fact gmail.com is a permanent redirect to mail.google.com, that for our ranking is a negative signal (we prefer redirect destinations usually). So it would need ranking to change, potentially breaking other ton of things.
Additionally such a drastic change could make many other users ask "why is g now suddenly going to gmail.com instead of google.com?"
We need a more generalized approach that is directly affected by user choices, bug 1823449 is one possibility. But feel free to file a bug to evaluate a better approach for ranking these redirects.

Will

Reporter

Comment 33

•

7 months ago

Thank you! I filed it, here it is for anyone interested https://bugzilla.mozilla.org/show_bug.cgi?id=1854284

Marco Bonardo [:mak]

Updated

•

7 months ago

w urlbar.png 4 years ago Will 19.61 KB, image/png		Details
w autofill domain tab to search issue.png 3 years ago Will 121.01 KB, image/png		Details
github-google-engine.png 3 years ago Harry Twyford 42.27 KB, image/png		Details
m autofill.png 2 years ago Will 47.42 KB, image/png		Details
firefoxMautocomplete.png 8 months ago Will 46.77 KB, image/png		Details