Closed Bug 1242094 Opened 8 years ago Closed 8 years ago

Opening top site tile truncates URL to domain

Categories

(Firefox for iOS :: Home screen, defect)

All
iOS
defect
Not set
normal

Tracking

()

RESOLVED INVALID

People

(Reporter: stef, Unassigned)

Details

In bug 1231515 domain opening behavior for default sites was changed and it would be great if it could happen for user generated tiles as well.

Current behavior is problematic and from 1.4 even more unexpected for users. I don't know full top sites domain grouping story for sure but from what I could find this looks more and more like a wrong solution to a problem (like for "no one needs four facebook top sites?" from bug 1184582).

I believe we can easily come up with something better that will serve top level domains opening users equally well as the whole rest. (it depends on the expectations of course)
It's not the wrong solution to the problem. It's the best compromise we could achieve at the time.

Top Sites can't be all things to all users, and for some users it won't be useful. So it goes.


At some point in the future we want to explore some tweaks to the way it works now:

        // Eventually we can do something more sophisticated — e.g., if the site only consists of one
        // history item, show it, and otherwise use the longest common sub-URL (and take its title
        // if you visited that exact URL), etc. etc. — but not yet.


Using a single URL for a top site tile, though, doesn't make sense in the general case.

We could go back to not clustering by domain at all. That turns your Top Sites into a mess of similar URLs. Nope; we added grouping for a reason. We're not going back to the days of having twelve Bugzilla URLs filling the screen.

We could use one of the candidate URLs for the domain tile, which is how this worked prior to this change. That means tapping on "reddit.com" takes you, quite surprisingly, to one particular thread on one particular subreddit. Tapping on mail.google.com takes you to one of your 'favorite' emails. Nope, that's just stupid.

So… what else?

Better customization, certainly, including turning off Top Sites altogether -- it works for Karen, and it doesn't work for me, because Karen reads the same few news sites all the time, and I read thousands of different things across my devices

Better handling of single URLs, certainly. That's the crux of the tiles fix in Bug 1231515, and at some point we'll file a bug to do that.

This bug as you've phrased it is WORKSFORME. What do you propose as an alternative?
Flags: needinfo?(splewako)
Hardware: Other → All
(In reply to Richard Newman [:rnewman] from comment #1)
> It's not the wrong solution to the problem. It's the best compromise we
> could achieve at the time.
>
> Top Sites can't be all things to all users, and for some users it won't be
> useful. So it goes.
> 
> 
> At some point in the future we want to explore some tweaks to the way it
> works now:
> 
>         // Eventually we can do something more sophisticated — e.g., if the
> site only consists of one
>         // history item, show it, and otherwise use the longest common
> sub-URL (and take its title
>         // if you visited that exact URL), etc. etc. — but not yet.
> 
> 
> Using a single URL for a top site tile, though, doesn't make sense in the
> general case.
> 
> We could go back to not clustering by domain at all. That turns your Top
> Sites into a mess of similar URLs. Nope; we added grouping for a reason.
> We're not going back to the days of having twelve Bugzilla URLs filling the
> screen.

I hardly believe that setting tld domain as top site tile address and grouping visits around it is the best and only solution to query not returning unique enough results.

> We could use one of the candidate URLs for the domain tile, which is how
> this worked prior to this change. That means tapping on "reddit.com" takes
> you, quite surprisingly, to one particular thread on one particular
> subreddit. Tapping on mail.google.com takes you to one of your 'favorite'
> emails. Nope, that's just stupid.
> 
> So… what else?

Only if it is domain tile, if it is expected by the user top site tile it makes much more sense.

Frecency should guarantee that user will get appropriate landing page, be it "https://www.reddit.com/" or "https://www.reddit.com/controversial/" or "https://accounts.google.com/ServiceLogin" or "https://mail.google.com/mail/u/0/#inbox" or even said message if user really opens it much more often then his inbox.

Lowering scores for every next address in the domain may be the solution, another could be only one result per domain…

> Better customization, certainly, including turning off Top Sites altogether
> -- it works for Karen, and it doesn't work for me, because Karen reads the
> same few news sites all the time, and I read thousands of different things
> across my devices
> 
> Better handling of single URLs, certainly. That's the crux of the tiles fix
> in Bug 1231515, and at some point we'll file a bug to do that.
> 
> This bug as you've phrased it is WORKSFORME. What do you propose as an
> alternative?

I'm suggesting creating a list of expectations and investigating if the feature could be improved.
Flags: needinfo?(splewako)
(In reply to Stefan Plewako [:stef] from comment #2)

> Frecency should guarantee that user will get appropriate landing page, be it
> "https://www.reddit.com/" or "https://www.reddit.com/controversial/" or
> "https://accounts.google.com/ServiceLogin" or
> "https://mail.google.com/mail/u/0/#inbox" or even said message if user
> really opens it much more often then his inbox.

Frecency does not — cannot — guarantee any such thing.

The 'right' landing page is not necessarily the most frecent page, because 'right' is very much a human concept. The most frecent might be a login page, a recently re-visited attachment, or something else. The right page might be your Bugzilla Dashboard, the Enter Bug page, or the front page.

Furthermore, the most frecent URL for a particular domain isn't stable. One day you might tap the Reddit tile and get "reddit.com/r/eyebleach", and another day get "reddit.com/r/bifl". If you're in an active conversation in a thread, that thread might take over.

Your Google Maps tile will — apparently at random — select a business, search term, or place that you looked up recently, rather than taking you to the main Maps UI.

Some people might like that. But if you're a user who's reliant on Top Sites to reach a reasonable, consistent entry point to a site, rather than a random history URL, this behavior is maddening. Surprise is not a good feature of a user interface.

The only thing we can stably compute is the root URL for a collection of URLs with a common subdomain, which convention and conciseness has decided is a pretty good entry point to a site. (If you want your Bugzilla Dashboard or to file a bug, those are both linked from Bugzilla's front page.)

So our choice is between computing some stable root, or not doing clustering at all.

Not clustering really sucks. So here we are.


> I'm suggesting creating a list of expectations and investigating if the
> feature could be improved.

Oh, I thought you said in Comment 0 that we can easily come up with something better?

Can you at least offer a list of expectations?
(In reply to Richard Newman [:rnewman] from comment #3)
> Frecency does not — cannot — guarantee any such thing.

Right, I should have used the word "algorithm" instead.

> The 'right' landing page is not necessarily the most frecent page, because
> 'right' is very much a human concept. The most frecent might be a login
> page, a recently re-visited attachment, or something else. The right page
> might be your Bugzilla Dashboard, the Enter Bug page, or the front page.

I think user data is better source of rightfulness than developer decisions in general. 

> Furthermore, the most frecent URL for a particular domain isn't stable. One
> day you might tap the Reddit tile and get "reddit.com/r/eyebleach", and
> another day get "reddit.com/r/bifl". If you're in an active conversation in
> a thread, that thread might take over.

That would be expected and not that different from current situation, just on different level (domain vs tile).

> Your Google Maps tile will — apparently at random — select a business,
> search term, or place that you looked up recently, rather than taking you to
> the main Maps UI.

That would be surprising, haven heard about anything like that in desktop Firefox.

> Some people might like that. But if you're a user who's reliant on Top Sites
> to reach a reasonable, consistent entry point to a site, rather than a
> random history URL, this behavior is maddening. Surprise is not a good
> feature of a user interface.

Should I fill a bug every time when tile surprisingly gets removed from top sites? What are bookmarks for then?

> The only thing we can stably compute is the root URL for a collection of
> URLs with a common subdomain, which convention and conciseness has decided
> is a pretty good entry point to a site. (If you want your Bugzilla Dashboard
> or to file a bug, those are both linked from Bugzilla's front page.)
> 
> So our choice is between computing some stable root, or not doing clustering
> at all.
> 
> Not clustering really sucks. So here we are.

As a desktop Firefox user I'm really surprised and it makes me wonder what group of users Firefox for iOS is targeted at.

> > I'm suggesting creating a list of expectations and investigating if the
> > feature could be improved.
> 
> Oh, I thought you said in Comment 0 that we can easily come up with
> something better?
> 
> Can you at least offer a list of expectations?

I cannot as I don't know the whole story. If expectations will be known I think it should be easy to come up with something better (also because of what it is right now is so bad).

Or maybe I'm wrong and it is OK? Maybe there is a data available about addresses without path part popularity suggesting anything?
(In reply to Stefan Plewako [:stef] from comment #4)

> That would be surprising, haven heard about anything like that in desktop
> Firefox.

Desktop, just like iOS, truncates to a root domain in most cases. Take a look.

For instance, my top sites on desktop right now are facebook.com, reddit.com, twitter.com, maps.google.com, todoist.com, smile.amazon.com, github.com/mozilla/firefox-ios, and irccloud.

Note that the hundreds of reddit URLs are collapsed into the root domain, my thousands of Amazon URLs collapse into smile.amazon.com, and the thousands of GitHub URLs are collapsed into the "greatest common sub-URL" of the majority of my visits.

On desktop users can set a particular URL as their new tab page, and they can opt out of Top Sites, so user disappointment with Top Sites is lessened. I don't use it, and I'm not alone.


> Should I fill a bug every time when tile surprisingly gets removed from top
> sites?

For most users, with most behaviors, top sites should reach a point of stability and not churn much. They're not necessarily useful, but they tend to be fairly stable. But there are a limited number of slots, and so they'll change over time as you visit new sites.

If you repeatedly and recently visited a site and it surprisingly gets removed, then yes, please file a bug.


> > Not clustering really sucks. So here we are.
> 
> As a desktop Firefox user I'm really surprised and it makes me wonder what
> group of users Firefox for iOS is targeted at.

Desktop Firefox clusters by domain on the new tab page, almost identically to iOS. (We actually go a step further, up-weighting pages you visited on your mobile device and down-weighting pages you visited elsewhere, so that your desktop history doesn't dominate.)

In my profile, only two of my nine top sites on desktop are full URLs, which are "greatest common sub-URLs" with massive visit counts, but the overall approach is much the same.


> Or maybe I'm wrong and it is OK? Maybe there is a data available about
> addresses without path part popularity suggesting anything?

Our current collective opinion is essentially:

For some set of users (perhaps half, perhaps more; this is conjecture based on feedback and user research), particularly those who don't use Sync, Top Sites with domain clustering does pretty much what they want it to do. It'll pick out their favorite news site, their social networks, and so on, and will drop them at the front door of the site when they tap. Lots of people go to the same sites over and over again, in their little corner of the web.

For another set of users — often high-volume, varied browsers with Sync configured; often 'typers' versus 'tappers' — we stand pretty much zero chance of guessing where they're going to go next just based on their browsing history. Their behaviors are so varied and Top Sites just doesn't have enough space, or enough awareness of context.

For those users the best we can do is be consistent and surprise them as little as we can. I'm one of those users. I turn of Top Sites on desktop. Top Sites on iOS actually gets it right for me often enough that I'm a little surprised.


In summary: we'd eventually like to do what desktop does, which is to present full URLs for one-URL-only tiles, and present greatest-common-subURLs for tiles with an obvious pattern. But we haven't seen a dramatically different approach that does better overall than clustering by domain, and desktop seems to agree. If you've got one, I'd love to hear it. Otherwise, we can close this bug.
(In reply to Richard Newman [:rnewman] from comment #5)
> (In reply to Stefan Plewako [:stef] from comment #4)
> 
> > That would be surprising, haven heard about anything like that in desktop
> > Firefox.
> 
> Desktop, just like iOS, truncates to a root domain in most cases. Take a
> look.

I did and do not see nothing like current *#$! iOS truncations (and keep in mind that redirects are much more expensive on mobile), even if I see some main pages (not always "/"). For the record, I check normal users profiles and exclude my personal trying to avoid bias.

> For instance, my top sites on desktop right now are facebook.com,
> reddit.com, twitter.com, maps.google.com, todoist.com, smile.amazon.com,
> github.com/mozilla/firefox-ios, and irccloud.
> 
> Note that the hundreds of reddit URLs are collapsed into the root domain, my
> thousands of Amazon URLs collapse into smile.amazon.com, and the thousands
> of GitHub URLs are collapsed into the "greatest common sub-URL" of the
> majority of my visits.

Grouping (probably good thing but not only solution) doesn't force collapsing into the root domain - it could be done better as desktop shows.

> On desktop users can set a particular URL as their new tab page, and they
> can opt out of Top Sites, so user disappointment with Top Sites is lessened.
> I don't use it, and I'm not alone.

Either I don't understand how it is relevant or it is not.

> > > Not clustering really sucks. So here we are.
> > 
> > As a desktop Firefox user I'm really surprised and it makes me wonder what
> > group of users Firefox for iOS is targeted at.
> 
> Desktop Firefox clusters by domain on the new tab page, almost identically
> to iOS. (We actually go a step further, up-weighting pages you visited on
> your mobile device and down-weighting pages you visited elsewhere, so that
> your desktop history doesn't dominate.)

What I see on desktop Firefox is very different and if it would almost identical I wouldn't fill this bug.

> For some set of users (perhaps half, perhaps more; this is conjecture based
> on feedback and user research), particularly those who don't use Sync, Top
> Sites with domain clustering does pretty much what they want it to do. It'll
> pick out their favorite news site, their social networks, and so on, and
> will drop them at the front door of the site when they tap. Lots of people
> go to the same sites over and over again, in their little corner of the web.

That is good thing and I do not suggest changing it.

> For another set of users — often high-volume, varied browsers with Sync
> configured; often 'typers' versus 'tappers' — we stand pretty much zero
> chance of guessing where they're going to go next just based on their
> browsing history. Their behaviors are so varied and Top Sites just doesn't
> have enough space, or enough awareness of context.

"pretty much zero chance of guessing where they're going to go next just based on their browsing history" - I cannot disagree more and desktop Firefox suggest the same.

> For those users the best we can do is be consistent and surprise them as
> little as we can. I'm one of those users. I turn of Top Sites on desktop.
> Top Sites on iOS actually gets it right for me often enough that I'm a
> little surprised.

I also tend to disable desktop tiles on new tab page but in many aspects address bar searches are similar. It takes only single not exactly working truncation to transition "often enough" to not enough or misimplemented.

> In summary: we'd eventually like to do what desktop does, which is to
> present full URLs for one-URL-only tiles, and present
> greatest-common-subURLs for tiles with an obvious pattern. But we haven't
> seen a dramatically different approach that does better overall than
> clustering by domain, and desktop seems to agree. If you've got one, I'd
> love to hear it. Otherwise, we can close this bug.

What desktop does is much better, clustering by domain isn't necessarily bad but truncating URLs to tld in all cases creates poor experience even when user finally lands on mostly good page.
(In reply to Stefan Plewako [:stef] from comment #6)

> > Desktop, just like iOS, truncates to a root domain in most cases. Take a
> > look.
> 
> I did and do not see nothing like current *#$! iOS truncations

Can you describe what you do see?

In my profile, iOS and desktop have identical behavior, with the exceptions I note in Comment 5. That is, both iOS and desktop collapse

  https://reddit.com/*                 => https://reddit.com/
  https://{m.,www.,}facebook.com/*     => https://facebook.com/
  https://{m.,www.,}twitter.com/*      => https://twitter.com/
  https://maps.google.com/*            => https://maps.google.com/


> Either I don't understand how it is relevant or it is not.

In short: if you don't like top sites, you can turn it off, which means top sites doesn't need to be all things to all people. That's not (yet) the case on iOS.


> What desktop does is much better, clustering by domain isn't necessarily bad
> but truncating URLs to tld in all cases creates poor experience even when
> user finally lands on mostly good page.

What do you see desktop doing that's better? The iOS approach is based on desktop's.

Unless you tell me what you see different, I can't understand what's making you upset.
(In reply to Richard Newman [:rnewman] from comment #7)
> (In reply to Stefan Plewako [:stef] from comment #6)
> 
> > > Desktop, just like iOS, truncates to a root domain in most cases. Take a
> > > look.
> > 
> > I did and do not see nothing like current *#$! iOS truncations
> 
> Can you describe what you do see?
> 
> In my profile, iOS and desktop have identical behavior, with the exceptions
> I note in Comment 5. That is, both iOS and desktop collapse
> 
>   https://reddit.com/*                 => https://reddit.com/
>   https://{m.,www.,}facebook.com/*     => https://facebook.com/
>   https://{m.,www.,}twitter.com/*      => https://twitter.com/
>   https://maps.google.com/*            => https://maps.google.com/

Not true, desktop doesn't remove subdomais and path part.

maps.google.com 302 to http://maps.google.com/ which 302 to http://maps.google.com/maps and 302 to http://www.google.com/maps and 302 https://www.google.com/maps which loads https://www.google.pl/maps?source=tldso

https://www.google.pl/maps loads only https://www.google.pl/maps HTML.

> > Either I don't understand how it is relevant or it is not.
> 
> In short: if you don't like top sites, you can turn it off, which means top
> sites doesn't need to be all things to all people. That's not (yet) the case
> on iOS.

Not relevant then.

> > What desktop does is much better, clustering by domain isn't necessarily bad
> > but truncating URLs to tld in all cases creates poor experience even when
> > user finally lands on mostly good page.
> 
> What do you see desktop doing that's better? The iOS approach is based on
> desktop's.

Even if iOS approach was supposed to be based on desktop behavior it simply isn't - on desktop user lands on the page they wanted after one http request quite often and on iOS that is almost never true.

> Unless you tell me what you see different, I can't understand what's making
> you upset.

I already did and in this comment I did once again. I don't know if this problem with understanding comes from apparently wrong assumptions about desktop or failure to recognize messing with urls impact, sorry.
(In reply to Stefan Plewako [:stef] from comment #8)

> Not true, desktop doesn't remove subdomais and path part.

It absolutely does.

https://dxr.mozilla.org/mozilla-central/source/toolkit/modules/NewTabUtils.jsm#1414


> https://www.google.pl/maps loads only https://www.google.pl/maps HTML.

Yup. So either you're getting maps.google.com as a top site, and it's following that redirect chain, or you're getting google.pl/maps, and it's not. Which are you getting on each platform?


> Even if iOS approach was supposed to be based on desktop behavior it simply
> isn't - on desktop user lands on the page they wanted after one http request
> quite often and on iOS that is almost never true.

Are you complaining that on mobile sites often redirect from foo.com => m.foo.com?

Are you complaining that desktop sites are being shown as top sites on mobile?


> I already did and in this comment I did once again. I don't know if this
> problem with understanding comes from apparently wrong assumptions about
> desktop or failure to recognize messing with urls impact, sorry.

You haven't told me what you see and what you expect to see. You've told me the Google Maps redirect chain.

How about you start with a screenshot of your about:newtab on desktop with a hover (showing the destination URL) on a tile that particularly bothers you, and a screenshot of your home screen on iOS?
(In reply to Richard Newman [:rnewman] from comment #9)
> (In reply to Stefan Plewako [:stef] from comment #8)
> 
> > Not true, desktop doesn't remove subdomais and path part.
> 
> It absolutely does.
> 
> https://dxr.mozilla.org/mozilla-central/source/toolkit/modules/NewTabUtils.
> jsm#1414

I have no idea what that code is, what it represents or when it is called. It could be easily tested that it does not remove subdomain or path from sites displayed on a new tab page in desktop Firefox. 

> > https://www.google.pl/maps loads only https://www.google.pl/maps HTML.
> 
> Yup. So either you're getting maps.google.com as a top site, and it's
> following that redirect chain, or you're getting google.pl/maps, and it's
> not. Which are you getting on each platform?

No, not true. On iOS after redirects I get to  https://www.google.pl (from top site https://google.pl/) on desktop I get to https://www.google.pl/maps (from new tab site https://www.google.pl/maps).

> > Even if iOS approach was supposed to be based on desktop behavior it simply
> > isn't - on desktop user lands on the page they wanted after one http request
> > quite often and on iOS that is almost never true.
> 
> Are you complaining that on mobile sites often redirect from foo.com =>
> m.foo.com?

No.

> Are you complaining that desktop sites are being shown as top sites on
> mobile?

No.

> > I already did and in this comment I did once again. I don't know if this
> > problem with understanding comes from apparently wrong assumptions about
> > desktop or failure to recognize messing with urls impact, sorry.
> 
> You haven't told me what you see and what you expect to see. You've told me
> the Google Maps redirect chain.

I did, don't mess with urls to assert uniqueness of top sites urls (there are better ways to do that).

> How about you start with a screenshot of your about:newtab on desktop with a
> hover (showing the destination URL) on a tile that particularly bothers you,
> and a screenshot of your home screen on iOS?

Maybe you could read my comments more carefully first?
(In reply to Stefan Plewako [:stef] from comment #10)

> I have no idea what that code is, what it represents or when it is called.
> It could be easily tested that it does not remove subdomain or path from
> sites displayed on a new tab page in desktop Firefox. 

It does; that's exactly what it's for, so that 'noise' subdomains don't each appear in top sites. That code is why you don't get facebook.com, m.facebook.com, and www.facebook.com in your Top Sites.

For example, I have a desktop "https://reddit.com/" tile which loads and redirects to "https://www.reddit.com/".


> No, not true. On iOS after redirects I get to  https://www.google.pl (from
> top site https://google.pl/) on desktop I get to https://www.google.pl/maps
> (from new tab site https://www.google.pl/maps).

Okay, so your complaint is what I mentioned in Comment 1 and Comment 5, about extraction of greatest common sub-URLs.

Do you have two tiles on desktop, one of which is www.google.pl/ and one which is www.google.pl/maps, or only one?
(In reply to Richard Newman [:rnewman] from comment #11)
> (In reply to Stefan Plewako [:stef] from comment #10)
> 
> > I have no idea what that code is, what it represents or when it is called.
> > It could be easily tested that it does not remove subdomain or path from
> > sites displayed on a new tab page in desktop Firefox. 
> 
> It does; that's exactly what it's for, so that 'noise' subdomains don't each
> appear in top sites. That code is why you don't get facebook.com,
> m.facebook.com, and www.facebook.com in your Top Sites.
> 
> For example, I have a desktop "https://reddit.com/" tile which loads and
> redirects to "https://www.reddit.com/".

Then it doesn't work or maybe it works but not in a suggested way - maybe sites get grouped by domain and subdomain (with m, mobile, www and subdomains contributing to domain) for scoring purposes only. On the desktop new tab page I see sites with addresses not striped from "www." subdomains and path parts for sure.

> Okay, so your complaint is what I mentioned in Comment 1 and Comment 5,
> about extraction of greatest common sub-URLs.

My complain and suggestions are in my comments.

> Do you have two tiles on desktop, one of which is www.google.pl/ and one
> which is www.google.pl/maps, or only one?

I don't see two sites from the same subdomain appearing in a new tab page on desktop, just single one. In fact, I don't see them both even if they are only two sites in history and that may be good enough.
> On the desktop
> new tab page I see sites with addresses not striped from "www." subdomains
> and path parts for sure.

That's part of presentation, not clustering.

> I don't see two sites from the same subdomain appearing in a new tab page on
> desktop, just single one. In fact, I don't see them both even if they are
> only two sites in history and that may be good enough.

OK, good. So desktop is clustering in exactly the same way as iOS, and you are seeing that on your desktop profile.

Desktop is extracting a high-scoring common subdomain and path, while iOS is simply taking the common domain. You visit google.pl/maps more than any other individual google.pl sub-path, so that 'wins' as the representation for that domain's tile.

Extracting that high-scoring common subdomain is exactly what I mentioned in Comment 1 and Comment 5, and we already plan to do that at some point when it's feasible to do so, so I'm going to close this bug. INVALID, because this is working as designed for now.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → INVALID
(In reply to Richard Newman [:rnewman] from comment #13)
> > On the desktop
> > new tab page I see sites with addresses not striped from "www." subdomains
> > and path parts for sure.
> 
> That's part of presentation, not clustering.

It may be but but I highly doubt that is entirely true and that doesn't matter as far as that bug should be concerned.

> > I don't see two sites from the same subdomain appearing in a new tab page on
> > desktop, just single one. In fact, I don't see them both even if they are
> > only two sites in history and that may be good enough.
> 
> OK, good. So desktop is clustering in exactly the same way as iOS, and you
> are seeing that on your desktop profile.

This bug is not only about clustering despite your tries to force such interpretation.

> Desktop is extracting a high-scoring common subdomain and path, while iOS is
> simply taking the common domain. You visit google.pl/maps more than any
> other individual google.pl sub-path, so that 'wins' as the representation
> for that domain's tile.
>
> Extracting that high-scoring common subdomain is exactly what I mentioned in
> Comment 1 and Comment 5, and we already plan to do that at some point when
> it's feasible to do so, so I'm going to close this bug. INVALID, because
> this is working as designed for now.

iOS behavior is simply wrong and bad. Apparently you have some problem with admitting and fixing that as you invalidated bug that suggest what is supposedly planed.
You need to log in before you can comment on or make changes to this bug.