1388187 - AS Highlights query is still very slow

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Activity Stream, enhancement, P3)

Description

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

I tried to fix this but failed to do make a huge dent (results in bug 1369604 comment 70).

That being said, I think we should retriage (which is why I filed a follow-up): this is an optimization that I don't think it'll make a huge UX difference. As far as I know, once once the new design is implemented, Highlights is below the fold *and* below Pocket suggestions, which gets much more engagement on desktop.

---

Suggested path to follow for this bug, with my priority weighting from top to bottom:
- (alternative solution) Rather than making a raw query every time, we cache the results. Apparently, this is what iOS does (bug 1369604 comment 69).
- Profile on a slower device (see below) with a non-skewed profile: address found problems.

If the problem is still the post-processing, consider these approaches:
- Moving the data from the Cursor into Java objects takes the most time ^ so consider doing the ranking in the DB so we can avoid this step entirely (bug 1369604 comment 64)
- Parallelize HighlightsRanking.extractFeatures (54.4% of runtime; bug 1369604 comment 22). Unfortunately, we're blocked by what we can get out of the Cursor in parallel.
- Optimize Uri accesses: perhaps read String directly rather than using Uri API (bug 1369604 comment 24)

---

My findings from bug 1369604:

- On Chenxia's Moto X with a regular profile, the experience is terribly slow (both top sites & highlights). I think we should investigate more devices like Chenxia's – I wonder if the problem could be a slow disk (I think I made a mistake profiling on the devices I did, which were reasonably quick).

- For AS, we make two simultaneous disk queries (top sites, highlights; soon we may add Pocket), get screenshots from the disk, and then proceed to touch the disk more to pull results from the Cursor: this could be bad on devices with a slow disk.

- wrt ^, I also wonder about how much memory we're using: maybe we need to page too, which hits the disk harder, or we GC which would hit the CPU when post-processing already takes a long time (profile for GC?)

- On my profiling devices (GS5, N5), with skewed profiles (not enough history/bookmarks) the disk query is proportionally very fast but the post-processing is slow, particularly the process of reading from the Cursor to instantiate the max 500 objects we post-process on, from which only 5 are kept.

Comment 1

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

fwiw, I think this looks extra janky against the sleek new Photon UI.

Also, Kevin has bad perf on a synced (with 3 devices)/old profile on the 5X so perhaps we do need real profiles to test on.

Comment 2

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

After talking on Slack, rnewman concluded we should just cache it:

rnewman [15:36] 
It is very difficult to make fast the kinds of queries that AS runs without making significant changes to the way we store data.
(True for top sites, too.)
Caching is likely the only short-term solution.
Fast autocomplete requires alternative indexing strategies (FTS, for example)
FTS on iOS: https://bugzilla.mozilla.org/show_bug.cgi?id=1173164. Android: https://bugzilla.mozilla.org/show_bug.cgi?id=808872

Comment 3

[Richard Newman [:rnewman]]

My general opinion is that there are basically no remedies for this between short-term work on caching and long-term work on totally changing how we represent and store user activity.

We could make the feature less sophisticated, or store less data, but for the experience as designed there aren't a lot of options.

Comment 4

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

On my test device, we're throwing 192 exceptions each time this loads - I wonder if that is contributing to the slow performance. I filed bug 1390685.

Comment 5

[:Grisha Kruglov]

FWIW first run of a fresh install of the current m-c took around 5-6s to load the AS UI - top sites & highlights.

Comment 6

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

(In reply to :Grisha Kruglov from comment #5)
> FWIW first run of a fresh install of the current m-c took around 5-6s to
> load the AS UI - top sites & highlights.

I wonder if this includes the time to initially create the DBs (which would skew the measurement).

Comment 7

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

All open Activity Stream bugs are moving from the whiteboard tag, "[mobileAS]", to the Firefox for Android component, "Activity Stream", so that I can keep better track of these bugs as the new triage owner; I will send out an email shortly with additional details, caveats, etc.

Comment 8

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.