1332621 - Duplicate top sites due to multiple redirects being introduced in history

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Activity Stream, defect, P5)

Description

[Oana Horvath [:ohorvath]]

Attachment: desktop_history.png

Environment: 
Device: LG G4(Android 6.0) & Lenovo A536 Android (4.4.2);
Build: Nightly 53.0a1 (2017-20-1);

Steps to reproduce:
1. On a clean profile, on Firefox desktop browser, type in a website address, without http://www. E.g: bbc.com, gmail.com.
2. Sign in with the same profile on an mobile device and sync.
3. Check the history records synced on the mobile device. 

Expected result:
The synced history should be the same on both mobile and desktop browsers. The redirects made to open the final page should be hidden.

Actual result:
Redirects as from http to https or from bbc.com to http://www.bbc.com, etc, are hidden in Firefox desktop, but shown in Fennec. These multiple urls that lead to the same page, create duplicated top sites.

Notes: Attached screenshots.

Comment 1

[Oana Horvath [:ohorvath]]

Attachment: Screenshot_2017-01-20-14-28-41.png

Comment 2

[Oana Horvath [:ohorvath]]

Attachment: Screenshot_2017-01-20-14-01-08.png

Comment 3

[Oana Horvath [:ohorvath]]

Attachment: Screenshot_2017-01-19-09-04-30.png

Comment 5

[Kevin Brosnan [Ex-Mozilla]]

Normally the newer bug is duped to the older bug. In the future please follow this practice.

Comment 7

[Sorina Florean [:sflorean]]

Also, reproducible when sync is not performed.

Comment 8

[Nevin Chen(Not active on Bugzilla)]

Activity Stream will replace Top Site. So mark this as won't fix.

Comment 9

[Wesly Huang (EPM)]

[traige@0816] Max, would you help remove the tracking-fennec? since this is WONTFIX? Tks.

Comment 10

[:Grisha Kruglov]

This isn't a WONTFIX. Activity Stream isn't changing the top sites algorithm much (just how things are pinned and displayed, IIUC), neither did our underlying data change much from which Top Sites are generated. Some de-duping work was done in Bug 1382332, but I don't expect it to help here. 

As described in Comment 1 (history synced from desktop), this might not be as relevant in practice due to us heavily de-prioritizing remote history. For synced desktop visits, the visit type might help us exclude these bogus visits from our calculations, if we don't do this already. On Fennec though, all visits are recorded with the same default type, unfortunately.

I'd expect this to still be a practical problem for any visits/redirects generated locally on the device, e.g. see Comment 7.

NI mcomella as he's working on this currently.

Comment 11

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

(In reply to :Grisha Kruglov from comment #10)
> Activity Stream isn't changing the top sites algorithm much

This is true: Nevin & Taipei folks, if you see old top sites bugs with regard to the data (as opposed to the presentation), be sure to flag me!

> Some de-duping work was done in Bug 1382332, but I don't expect it to help here.

In that bug, I only:
- Dedupe "http" vs. "https" sites in highlights
- Dedupe multiple bookmarks making multiple top sites entries

While this bug looks like it's about:
- Dedupe "(www.)url" in top sites
- Dedupe "http(s)" in top sites
- (maybe) dedupe "news.google.com/#0" & "news.google.com"

Alternatively to ^: we can distinguish which visits are redirects and weight non-redirects higher in top sites

> I'd expect this to still be a practical problem for any visits/redirects
> generated locally on the device, e.g. see Comment 7.

Changed the bug title accordingly.

> NI mcomella as he's working on this currently.

Thanks for flagging me.

For what it's worth, I'd like to see how much this actually happens in the wild before we put in the effort to fix it. For new profiles, these sites appear in top sites but I don't think they'd be very prevalent for a used profile (so a new profile might be a little unpolished). Top sites reads from a Cursor directly so it's a pain to modify the data or skip entries.

Comment 12

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Maria thinks its important to make top sites look as good as possible - Aaron, how important do you think this is?

Roughly, my understanding of this issue is:
- If both www.<url> and <url> appear, we remove www.<url>
- If both

Comment 13

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Maria thinks its important to make top sites look as good as possible but we recognize this is an existing issue - Aaron, how important do you think this is?

Roughly, my understanding of what needs to be changed issue is:
- If both www.<url> and <url> appear, we remove www.<url>
- If both http<url> and https<url> appear, we remove http<url>

There is also the case of "news.google.com/#0" & "news.google.com" appearing in top sites but I don't know that redirecting <url>/#anchor -> <url> is always the best case for the user (e.g. if I intentionally bookmark an anchor tag in the page, I'd never have that in my top sites).

Comment 14

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

We prioritized this and no longer need UX feedback.

Comment 15

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

All open Activity Stream bugs are moving from the whiteboard tag, "[mobileAS]", to the Firefox for Android component, "Activity Stream", so that I can keep better track of these bugs as the new triage owner; I will send out an email shortly with additional details, caveats, etc.

Comment 17

[Mark]

Hi guys, I got directed here from bug 1418657 which apparently is a duplicate (I'm not so sure).

While you are de-duplicating can you think about preventing

moz-extension://
https://addons.mozilla.org

from appearing in Fennec's History and Top Sites?

At the moment I get all kinds of junky entries. It's a particular problem if you use uBlock Origin or similar, every time you open the uBO dashboard you get a moz-extension://[big hex number] entry in History which isn't really very useful.

cheers now

Comment 19

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 20

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.