1410180 - Not all typed urls get TYPED transition when opening many tabs

Status: NEW

(Firefox :: Address Bar, defect, P3)

Description

[Iulia Cristescu, QA [:JuliaC] (away, please needinfo? [e-mail])]

[Note]:
- I started from the assumption that, without any default top sites, the Top Sites section should always display the most visited websites, in descending order

[Affected versions]:
- 58.0a1 (2017-10-19)
- 57.0b9 (20171016185129)

[Affected platforms]:
- Windows 10x64
- Ubuntu 16.04 x64

[Steps to reproduce]:
1. Launch Firefox with a clean profile
2. Open the following websites (in separate tabs):
   - https://9gag.com/ (8 times)
   - https://imgur.com/ (9 times) 
   - http://www.psimadethis.com/ (12 times)
   - https://bugzilla.mozilla.org/ (4 times)
3. Restart the browser and access the AS New Tab page
4. Access http://www.imdb.com/ 11 times, restart the browser and reopen the AS New Tab page
5. Access https://github.com/ 13 times, restart the browser and reopen the AS New Tab page

[Expected result]:
- [step3] Top Sites section displays the above sites in the following order:
http://www.psimadethis.com/, https://imgur.com/, https://9gag.com/, https://bugzilla.mozilla.org/
- [step4] Top Sites section displays the above sites in the following order:
http://www.psimadethis.com/, http://www.imdb.com/, https://imgur.com/, https://9gag.com/, https://bugzilla.mozilla.org/
- [step5] Top Sites section displays the above sites in the following order:
https://github.com/, http://www.psimadethis.com/, http://www.imdb.com/, https://imgur.com/, https://9gag.com/, https://bugzilla.mozilla.org/

[Actual result]:
- [step3] Top Sites section displays the above sites in the following order: 
https://imgur.com/, http://www.psimadethis.com/, https://9gag.com/, https://bugzilla.mozilla.org/
- [step4] Top Sites section displays the above sites in the following order:
https://imgur.com/, http://www.imdb.com/, http://www.psimadethis.com/, https://9gag.com/, https://bugzilla.mozilla.org/
[step5] Top Sites section displays the above sites in the following order:
https://github.com/, https://imgur.com/, http://www.imdb.com/, http://www.psimadethis.com/, https://9gag.com/, https://bugzilla.mozilla.org/

[Additional notes]:
- The same steps provided the expected results using Mac OS 10.11.6

Comment 1

[Ed Lee :Mardak]

How did you "access" the pages? The ordering is based on frecency, which is based on visit count but not the same. In particular, if you type in a url, it will be much more highly weighted towards the beginning of the list.

Comment 2

[Iulia Cristescu, QA [:JuliaC] (away, please needinfo? [e-mail])]

(In reply to Ed Lee :Mardak from comment #1)
> How did you "access" the pages? The ordering is based on frecency, which is
> based on visit count but not the same. In particular, if you type in a url,
> it will be much more highly weighted towards the beginning of the list.

I accessed the pages by firstly typing the exact URL and then by sequentially opening different new tabs and pasting the same URL.

Comment 3

[Iulia Cristescu, QA [:JuliaC] (away, please needinfo? [e-mail])]

(In reply to Iulia Cristescu, QA [:JuliaC] from comment #2)
> (In reply to Ed Lee :Mardak from comment #1)
> > How did you "access" the pages? The ordering is based on frecency, which is
> > based on visit count but not the same. In particular, if you type in a url,
> > it will be much more highly weighted towards the beginning of the list.
> 
> I accessed the pages by firstly typing the exact URL and then by
> sequentially opening different new tabs and pasting the same URL.

I mean that only the first URL was typed and the rest pasted, for a particular site.

Comment 4

[Ed Lee :Mardak]

I seem to reproduce something similar if opening many tabs quickly with the url where the issue is that the "typed" / pasted in page doesn't associate as a typed page probably because so many tabs are being opened.
```
» NewTabUtils.activityStreamProvider.getTopFrecentSites().then(v => console.log(v.map(s => [s.frecency, s.url]).join("\n")))
19440,http://www.psimadethis.com/
14100,https://9gag.com/
10400,https://imgur.com/
8300,https://bugzilla.mozilla.org/
sqlite> SELECT url, frecency, v.* FROM moz_historyvisits v JOIN moz_places h ON h.id = v.place_id;
https://www.mozilla.org/privacy/firefox/|25|1|0|11|1508786614509189|1|0
https://www.mozilla.org/en-US/privacy/firefox/|100|2|1|12|1508786614571997|5|0
https://9gag.com/|14100|3|0|13|1508786622171639|2|0
https://9gag.com/|14100|4|0|13|1508786622203836|1|0
https://9gag.com/|14100|5|0|13|1508786623399590|2|0
https://9gag.com/|14100|6|0|13|1508786623745059|2|0
https://9gag.com/|14100|7|0|13|1508786624709272|2|0
https://9gag.com/|14100|8|0|13|1508786625790396|2|0
https://9gag.com/|14100|9|0|13|1508786626371267|2|0
https://9gag.com/|14100|10|0|13|1508786627296569|2|0
https://imgur.com/|10400|11|0|14|1508786632576312|2|0
https://imgur.com/|10400|12|0|14|1508786633244981|2|0
https://imgur.com/|10400|13|0|14|1508786634507353|2|0
https://imgur.com/|10400|14|0|14|1508786636050321|2|0
https://imgur.com/|10400|15|0|14|1508786636458137|1|0
https://imgur.com/|10400|16|0|14|1508786640134732|2|0
https://imgur.com/|10400|17|0|14|1508786640190539|1|0
https://imgur.com/|10400|18|0|14|1508786640412034|1|0
https://imgur.com/|10400|19|0|14|1508786641036418|1|0
http://www.psimadethis.com/|19440|20|0|15|1508786647004844|2|0
http://www.psimadethis.com/|19440|21|0|15|1508786647816430|2|0
http://www.psimadethis.com/|19440|22|0|15|1508786648481147|1|0
http://www.psimadethis.com/|19440|23|0|15|1508786649493700|2|0
http://www.psimadethis.com/|19440|24|0|15|1508786650361411|2|0
http://www.psimadethis.com/|19440|25|0|15|1508786651159309|2|0
http://www.psimadethis.com/|19440|26|0|15|1508786652085877|2|0
http://www.psimadethis.com/|19440|27|0|15|1508786652886799|2|0
http://www.psimadethis.com/|19440|28|0|15|1508786653831779|2|0
http://www.psimadethis.com/|19440|29|0|15|1508786654645928|2|0
http://www.psimadethis.com/|19440|30|0|15|1508786655501652|2|0
http://www.psimadethis.com/|19440|31|0|15|1508786656354243|1|0
https://bugzilla.mozilla.org/|8300|32|0|2|1508786662587968|2|0
https://bugzilla.mozilla.org/|8300|33|0|2|1508786662955018|2|0
https://bugzilla.mozilla.org/|8300|34|0|2|1508786663972252|2|0
https://bugzilla.mozilla.org/|8300|35|0|2|1508786664966663|2|0
```

However, I don't see the issue when typing in the url once and loading it again by pressing enter.
```
» NewTabUtils.activityStreamProvider.getTopFrecentSites().then(v => console.log(v.map(s => [s.frecency, s.url]).join("\n")))
24000,http://www.psimadethis.com/
18000,https://imgur.com/
16000,https://9gag.com/
8300,https://bugzilla.mozilla.org/
sqlite> SELECT url, frecency, v.* FROM moz_historyvisits v JOIN moz_places h ON h.id = v.place_id;
https://www.mozilla.org/privacy/firefox/|25|1|0|11|1508787251464412|1|0
https://www.mozilla.org/en-US/privacy/firefox/|100|2|1|12|1508787251512556|5|0
https://9gag.com/|16000|3|0|13|1508787432000880|2|0
https://9gag.com/|16000|4|0|13|1508787432286841|2|0
https://9gag.com/|16000|5|0|13|1508787433064777|2|0
https://9gag.com/|16000|6|0|13|1508787435048495|2|0
https://9gag.com/|16000|7|0|13|1508787436843112|2|0
https://9gag.com/|16000|8|0|13|1508787437376302|2|0
https://9gag.com/|16000|9|0|13|1508787438840822|2|0
https://9gag.com/|16000|10|0|13|1508787453554179|2|0
https://imgur.com/|18000|11|0|14|1508787462172479|2|0
https://imgur.com/|18000|12|0|14|1508787462865469|2|0
https://imgur.com/|18000|13|0|14|1508787463666589|2|0
https://imgur.com/|18000|14|0|14|1508787464602241|2|0
https://imgur.com/|18000|15|0|14|1508787465477303|2|0
https://imgur.com/|18000|16|0|14|1508787466335721|2|0
https://imgur.com/|18000|17|0|14|1508787467307475|2|0
https://imgur.com/|18000|18|0|14|1508787468194139|2|0
https://imgur.com/|18000|19|0|14|1508787476077778|2|0
http://www.psimadethis.com/|24000|20|0|15|1508787487760948|2|0
http://www.psimadethis.com/|24000|21|0|15|1508787489803958|2|0
http://www.psimadethis.com/|24000|22|0|15|1508787491964565|2|0
http://www.psimadethis.com/|24000|23|0|15|1508787492615127|2|0
http://www.psimadethis.com/|24000|24|0|15|1508787493596708|2|0
http://www.psimadethis.com/|24000|25|0|15|1508787512907797|2|0
http://www.psimadethis.com/|24000|26|0|15|1508787513788632|2|0
http://www.psimadethis.com/|24000|27|0|15|1508787514738562|2|0
http://www.psimadethis.com/|24000|28|0|15|1508787515571891|2|0
http://www.psimadethis.com/|24000|29|0|15|1508787516457251|2|0
http://www.psimadethis.com/|24000|30|0|15|1508787533970946|2|0
http://www.psimadethis.com/|24000|31|0|15|1508787535057143|2|0
https://bugzilla.mozilla.org/|8300|32|0|2|1508787549943460|2|0
https://bugzilla.mozilla.org/|8300|33|0|2|1508787550750170|2|0
https://bugzilla.mozilla.org/|8300|34|0|2|1508787551707928|2|0
https://bugzilla.mozilla.org/|8300|35|0|2|1508787552460847|2|0
```

Comment 5

[Ed Lee :Mardak]

Probably something related to multiple calls to markPageAsTyped https://searchfox.org/mozilla-central/search?q=markPageAsTyped but getting consumed by the first one that loads ?

Comment 6

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

I'd like to track this bug as a blocker for 57 until we better understand the severity. A problem in our top site candidate list determination algorithm can lead to a bad user experience.

Comment 7

[Ed Lee :Mardak]

I would say this is wontfix for 57. This is the same behavior as before 57, and it only happens if the user is opening and typing in a url before the previous one has finished loading. And it's only really noticeable if the user is typing in the same url many times, which seems unlikely in normal browser usage.

Comment 8

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Makes sense. I agree. That does sound like an edge case. Untracked for 57.

Comment 9

[Panos Astithas (he/him) [:past] (please ni?)]

Moving to P3 as we aren't supposed to be using P4 any more.