1027791 - Image-switch of high-quality downscale is visible, hurts UX, may hurt performance

Status: RESOLVED WONTFIX

(Firefox :: New Tab Page, defect)

Description

[Avi Halachmi (:avih)]

about:newtab page is preloaded. This means that the content is loaded into a hidden browser early, and later displayed to screen without "rendering from scratch" once the user opens a new tab - to achieve a smoother and more consistent user experience.

However, high quality image downsscaling (bug 486918) makes it render the preview thumbnails (probably directory tiles as well) in two stages:

1. Renders a low quality downscale to screen.
2. Usually about a frame later - switch the image with a higher-quality downscaled version.

This switch is visible and degrades the user experience. Also, it might also affect tab animation performance - especially on slower systems - because the second rendering and then the switch happens together with the animation, so both race for CPU cycles. Though I didn't try to measure it (I probably should).

This brought me to think if we can avoid this. And maybe going even further, and completely disable this system (rendering directly and only the high quality version) under certain conditions.

Such conditions could be, for example, that after rendering the first 2-stages NN downscaled images in a session and measuring their downscale rate (e.g. normalized to pixels/second), if the system is fast enough to render at more than some threshold speed, just completely disable the system and use only one version of downscale - high quality.

This could, for instance, also save CPU cycles while loading pages. Instead of spending CPU cycles rendering each downscaled image twice, just do it once, and it's going to be fast enough anyway on this system, possibly faster than rendering and then switching.

So I wanted to discuss possible approaches or ideas on this.

1. For newtab.
2. In general.

Jeff?

I know bug 486918 comment 221 (part 3) adds a pref to control this, but it's only to disable this 2-phase rendering, and I _think_ it leaves the user with low-quality only.

Comment 1

[Avi Halachmi (:avih)]

Does the downscale happen on the GPU? does that happen for both low and high quality scaling? I'm pretty sure the GPU can downscale high quality practically free compared to low quality, so that's another vector which might make this switch redundant.

Comment 2

[Elbart]

There's also this issue (bug 925611), which makes the switch more noticeable, because about:newtab is using downscaled images:

http://elbart.bplaced.net/bugzilla/925611_newtab.mp4
(repeatedly hitting ctrl+f5 on about:newtab)

The switch is also happening (and noticeable) when a new tile appears after removing the last tile in the bottom right corner.


May not happen for all resolutions/dpi.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Shouldn't about:newtab just use images that don't need to be scaled at draw time?

(In reply to Avi Halachmi (:avih) from comment #1)
> Does the downscale happen on the GPU? does that happen for both low and high
> quality scaling? I'm pretty sure the GPU can downscale high quality
> practically free compared to low quality, so that's another vector which
> might make this switch redundant.

The high quality downscale doesn't happen on the GPU. Implementing the high quality downscale on the GPU isn't easy because it uses an iterative implementation to avoid using a tremendous amount of memory bandwidth.

Comment 4

[Avi Halachmi (:avih)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #3)
> Shouldn't about:newtab just use images that don't need to be scaled at draw
> time?

That's a good option, like Drew did for the search logo with bug 1030232. I'm not sure however how feasible it is, especially if the images are scaled according to the window size (which I think they don't right now, but they used to).

> The high quality downscale doesn't happen on the GPU. Implementing the high
> quality downscale on the GPU isn't easy because it uses an iterative
> implementation to avoid using a tremendous amount of memory bandwidth.

That's a bit vague for me to fully understand, but regardless, I was asking about the idea in general that on fast enough systems we could drop the 2-steps system and instead just do one - and with high quality scaling, while gaining both better visual experience and possibly using less CPU cycles - regardless of the newtab page.

Comment 5

[Jessilyn Davis]

Hello!

This bug has been closed due to inactivity and/or the potential for this bug to no longer be an issue with the new Discovery Stream-powered New Tab experience.

Please help us triage by reopening if this issue still persists and should be addressed.

Thanks!