Reduce frame callback overhead
Categories
(Core :: Graphics: WebRender, enhancement)
Tracking
()
People
(Reporter: rmader, Assigned: rmader)
References
(Blocks 3 open bugs, )
Details
The current handling of frame callbacks requires us to create a lot of frame callbacks and thus surface commits. In order to reduce it to a sane level, we'd need a new Wayland protocol that:
- somehow allows us to group subsurfaces together, so we don't need to query callbacks for each tile
- optimally also replaces
gdk_wayland_window_[add|remove]_frame_callback_surface()
|
||
Comment 1•3 years ago
|
||
- somehow allows us to group subsurfaces together, so we don't need to query callbacks for each tile
Firefox should be able to do this on the client side without new protocol. All tiles which are synchronous sub-surfaces of the same parent surface will be updated together, so a single frame callback should suffice for them.
|
Assignee | |
Comment 2•3 years ago
|
||
(In reply to Michel Dänzer from comment #1)
- somehow allows us to group subsurfaces together, so we don't need to query callbacks for each tile
Firefox should be able to do this on the client side without new protocol. All tiles which are synchronous sub-surfaces of the same parent surface will be updated together, so a single frame callback should suffice for them.
I don't see how because:
- tiles don't cover the whole window area, so if e.g. only one tile is visible (the others obscured or off-screen), that one needs to have a callback setup.
- if we use a parent surface, covering the whole window, tiles above it could not be opaque. Because if a opaque subsurface covers an underlying one, the current Wayland spec allows and encourages not to send frame callbacks to it. Not using opaque-regions however implies extra GPU work.
I think this is a flaw in the original spec - when writing the frame callback section, people probably did not give enough attention to sub-surfaces.
|
||
Comment 3•3 years ago
|
||
Would it be possible to just create a special always-visible always-on-top-of-everything 1x1 pixel transparent subsurface specifically for the purpose of frame callbacks? This should avoid having to guess which one tile will for sure be visible on the next frame or whatever I think?
|
|
Assignee | |
Comment 4•3 years ago
|
||
(In reply to greg v [:unrelentingtech/myfreeweb] from comment #3)
Would it be possible to just create a special always-visible always-on-top-of-everything 1x1 pixel transparent subsurface specifically for the purpose of frame callbacks? This should avoid having to guess which one tile will for sure be visible on the next frame or whatever I think?
The closest to an always-on-top-of-everything surface is the top-most subsurface. No way to prevent other apps to hide it. And no way to position it globally, only in window coordinates. So no, doesn't work - and on purpose (i.e. that is basic Wayland philosophy).
|
|
||
Comment 5•3 years ago
|
||
(In reply to Robert Mader [:rmader] from comment #4)
The closest to an always-on-top-of-everything surface is the top-most subsurface. No way to prevent other apps to hide it. And no way to position it globally, only in window coordinates. So no, doesn't work - and on purpose (i.e. that is basic Wayland philosophy).
By "everything" I only meant "all other subsurfaces", not literally every surface in the system! Yes, a special transparent top-most subsurface that would have nothing rendered on it and would only be used for frame callbacks.
Oh, right, because these callbacks are wanted whenever any part of Firefox is visible, it would have to be full window size, not 1x1.
|
|
Assignee | |
Comment 6•3 years ago
|
||
(In reply to greg v [:unrelentingtech/myfreeweb] from comment #5)
Oh, right, because these callbacks are wanted whenever any part of Firefox is visible, it would have to be full window size, not 1x1.
In which case the Wayland compositor, which doesn't know the surface is all transparent, would need to do a window size extra blit and would have a hard time to use direct scan-out or overlay planes, increasing the GPU work significantly :/
So no, not a workable solution.
We know which nsWindow is focused (if any) so we can use that info and get frame callbacks from focused nsWindow only. Question is how it's reliable.
|
|
Assignee | |
Comment 8•3 years ago
•
|
||
(In reply to Martin Stránský [:stransky] (ni? me) from comment #7)
We know which nsWindow is focused (if any) so we can use that info and get frame callbacks from focused nsWindow only. Question is how it's reliable.
Hm, don't see how that would work. In the end it all comes down to Wayland surfaces - if they are visible and if they are opaque. We have a GTK surface, a moz_container subsurface and lots of tile subsurfaces. Each subsurface, if opaque (which we want whenever possible to allow optimizations in the Wayland compositor), can hide other surfaces below. That's why we currently need gdk_wayland_window_[add|remove]_frame_callback_surface(), which makes GTK query frame callbacks from the moz_container surface (in the normal Wayland backend) or from opaque tiles (in the compositor backend), because otherwise GTK would get stuck.
I really don't see any way how we can get away with a single frame callback with the current spec. Thanks for all the well meaning suggestions, but please note: I have thought about the issue for quite some time (and implemented the problematic frame callback optimization in Mutter, and implemented the GTK API as a workaround, and implemented that in Firefox, once for the default and once for the compositor backend), and I'm not the only one (otherwise the GTK API wouldn't have been accepted).
So most likely the tasks now is to come up with a good idea how to define that protocol :)
|
|
||
Comment 9•3 years ago
|
||
(In reply to Robert Mader [:rmader] from comment #6)
In which case the Wayland compositor, which doesn't know the surface is all transparent, would need to do a window size extra blit and would have a hard time to use direct scan-out or overlay planes, increasing the GPU work significantly :/
Good point… except what if you detach the buffer? There wouldn't be anything to blit, and there kinda shouldn't be any reason for the compositor to exclude the surface from receiving callbacks?
good idea how to define that protocol
I guess the simplest would be to have a special "recursive" frame request that returns a frame callback that would be notified when any subsurface (or the surface itself) should render. Then you listen on the parent without worrying about the parent being occluded by subsurfaces.
|
|
Assignee | |
Updated•3 years ago
|
| Comment hidden (off-topic) |
|
||
Comment 11•3 years ago
|
||
Sorry, there was a problem with the detection of inactive users. I'm reverting the change.
Should be fixed now as we use frame callback routing with timestamp in Bug 1934497.
Ahh, I see it's about not getting frame callbacks from particular surfaces.
Description
•