1491456 - When using the CoreAnimation present path, consider tiling the window into multiple CALayers for a weak form of partial present

Status: RESOLVED FIXED

(Core :: Widget: Cocoa, enhancement, P2)

Description

[Markus Stange [:mstange]]

When you attach an IOSurface to a CALayer, and the contents of that IOSurface change, the window server will only recomposite the part of the screen that is occupied by that CALayer if you call setContentsChanged on the layer.

This gives some level of control over how much of the screen the window server recomposites. We may be able to make use of this to implement some form of "partial present". For example, we could tile our window into 256x256 sized layers, and then only call setContentsChanged on the layers that contain pixels that we know have changed. This should allow us to save some power on for example throbber animations.

This proposal is a bit hacky. The more pure solution would be to teach the compositor to create CoreAnimation layers for Gecko layers, so that for example the throbber animation gets one CoreAnimation layer that contains all the animation frames in a film strip, plus an offset and clip that only shows the current frame. (That's how the Gecko layers are set up for the throbber animation.) Such a solution would be preferable but it would require much more work in the compositor. The tiling solution that I'm proposing in this bug is easier to implement and a bit more generic, because it treats CompositorOGL compositing as a black box that just produces a framebuffer filled with pixels, along with an invalid region.

Comment 1

[Markus Stange [:mstange]]

Try build for testing: target.dmg
This build should have the most efficient GPU usage so far.

Comment 2

[Markus Stange [:mstange]]

For example, we could tile our window into 256x256 sized layers, and then only call setContentsChanged on the layers that contain pixels that we know have changed.

The plan has changed a bit since I wrote that initial comment. We don't use setContentsChanged in the current implementation at all; we always call setContents with a new IOSurface when we update a layer, because we never draw to surfaces that are currently in use by the window server. (This avoids getting partially drawn frames on the screen.)
So in this bug we do actually have to create small surfaces, not just small layers. The implementation here will build on top of bug 1574586, which solves a very similar problem.

Comment 3

[Markus Stange [:mstange]]

The tile size of 1024x256 might be too small. Scrolling simple websites with such a tile size causes the WindowServer process's CPU usage to go to 50%. If I make the tiles 1024x1024 sized (using the restartless pref layers.compositing-tiles.height), WindowServer CPU usage is around 37% instead. That's better but still not great. This warrants more investigation.

If the WindowServer is busy, wheel events don't fire regularly enough, which causes scrolling to stutter even if the GPU is otherwise free.

Comment 4

[Markus Stange [:mstange]]

Scrolling the same web page in Chrome, with the same windows open, makes the WindowServer use 30% CPU. So maybe 37% is good enough.

Comment 5

[Mark]

Just dropped by to say this is looking ridiculously good. On my 2013 MBP @ 1680x1050 with a small throbber on screen, stock Nightly draws ~21 W from my battery (about 8 W GPU). The Try from Comment 1 draws about 9 W from the battery (<1 W GPU). I think there are some savings on CPU power as well as GPU. Scrolling power consumption seems better too but not looked into it much.

Thanks a lot!

[I know I said I was going to shut up about power consumption, but couldn't resist ;) ]

Comment 6

[Markus Stange [:mstange]]

I'm always happy to hear about things being ridiculously good :)

Comment 7

[Sam Johnson]

I would also like to report good news from the try build above. On my Retina MacBook Air, not only is energy usage reduced, I am getting a 10fps boost when scrolling in certain scenarios. Also, I am noticing a marked improvement in text rendering (no more grey fringe around text in certain scenarios).

Comment 8

[Markus Stange [:mstange]]

Attachment: Bug 1491456 - Split the window into "compositing tiles" sized to 1024x1024. r=mattwoodrow

The tile size is configurable with the prefs layers.compositing-tiles.width/height.

On macOS, whenever a CALayer is touched, the window server will recomposite the
entire layer to the screen. There is no API to mark parts of a layer as damaged.
So if we want the window server to only redraw a small part of the screen, we
need to only touch small layers. This patch achieves that using tiles; whenever
the compositor needs to redraw an area, all tiles that overlap this area will
be drawn to their layers and the window server will recomposite those layers.

On Intel GPUs, compositing in tiles should also help reduce GPU times if there
are multiple layers of overdraw: The overdraw will have better cache locality.
However, the magnitude of this effect is not known and requires further research.

Depends on D43880

Comment 9

[Markus Stange [:mstange]]

(In reply to Sam Johnson from comment #7)

I would also like to report good news from the try build above. On my Retina MacBook Air, not only is energy usage reduced, I am getting a 10fps boost when scrolling in certain scenarios.

Great! However, the following makes me wonder whether something else is in play:

Also, I am noticing a marked improvement in text rendering (no more grey fringe around text in certain scenarios).

Text rendering should not be affected by this work. Are you maybe comparing WebRender to non-WebRender? WebRender has more known cases where it uses grayscale anti-aliasing.

Comment 10

[Markus Stange [:mstange]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=55685f859d16410134469c581d72bba8e42532be
https://treeherder.mozilla.org/#/jobs?repo=try&revision=78dd2d9d3d39a75074d2ea56083b32da580f5aa4

Comment 11

[Sam Johnson]

(In reply to Markus Stange [:mstange] from comment #9)

(In reply to Sam Johnson from comment #7)

I would also like to report good news from the try build above. On my Retina MacBook Air, not only is energy usage reduced, I am getting a 10fps boost when scrolling in certain scenarios.

Great! However, the following makes me wonder whether something else is in play:

Also, I am noticing a marked improvement in text rendering (no more grey fringe around text in certain scenarios).

Text rendering should not be affected by this work. Are you maybe comparing WebRender to non-WebRender? WebRender has more known cases where it uses grayscale anti-aliasing.

You're exactly right-- I did not realize at the time the profile I was comparing against had WebRender enabled. Regardless, I performed another test with WebRender disabled on both profiles, and the try build did still have a lower energy usage, which is great!

Comment 12

[Liz Henry (:lizzard) (relman/hg->git project)]

Watching with interest for 70 as it will be great if we fix some of the long standing mac power issues!

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/83dba42b1853
Split the window into "compositing tiles" sized to 1024x1024. r=mattwoodrow

Comment 14

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/83dba42b1853

Comment 15

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/releases/mozilla-beta/rev/83dba42b1853