The clip rects in the WR display list appear to be constrained by the window size.
Categories
(Core :: Graphics: WebRender, defect, P3)
Tracking
()
People
(Reporter: gw, Unassigned)
Details
I'm not certain this is a bug - it might be a misunderstanding on my part.
Problem: The clip rects supplied in the WR display list appear to be constrained (or at least affected) by the size of the window and/or the display port. This definitely occurs on clips in the clip-chain, and I think also occurs on the local clip rect of the primitive in some cases.
Background: With the picture caching improvements I'm working on, we want to render areas outside the visible rect of the window in some cases. For example, we may want to render a 1024 x 1024 portion of a scroll root, that is only partially on screen, in order to cache that for use on subsequent frames and/or scrolling.
Example:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<style type="text/css">
#app {
position: absolute;
width: 100px;
height: 5000px;
background: blue;
}
</style>
</head>
<body>
<div id="app"></div>
</body>
</html>
Given the above input, I took two WR scene captures - one with a maximized window, and one with a window sized to a quarter of the screen resolution.
In the pasted fragments of those captures in this case, the clip chain for the blue rectangle changes based on the size of the window.
Maximized:
Clip((
id: Clip(4, (1, 2)),
parent_space_and_clip: (
spatial_id: (1, (1, 2)),
clip_id: Clip(0, (1, 2)),
),
clip_rect: ((0, 0), (1898, 1958)),
image_mask: None,
), [
]),// [20]
Clip((
id: Clip(5, (1, 2)),
parent_space_and_clip: (
spatial_id: (3, (1, 2)),
clip_id: Clip(3, (1, 2)),
),
clip_rect: ((0, 0), (1898, 2176)),
image_mask: None,
), [
]),// [23]
ClipChain((
id: (4, (1, 2)),
parent: None,
), [
Clip(5, (1, 2)),// [0]
Clip(4, (1, 2)),// [1]
]),// [24]
Rectangle((
common: (
clip_rect: ((12, 12), (150, 7500)),
clip_id: ClipChain((4, (1, 2))),
spatial_id: (3, (1, 2)),
hit_info: None,
is_backface_visible: true,
),
color: (
r: 0,
g: 0,
b: 1,
a: 1,
),
)),// [27]
Half size window:
Clip((
id: Clip(4, (1, 2)),
parent_space_and_clip: (
spatial_id: (1, (1, 2)),
clip_id: Clip(0, (1, 2)),
),
clip_rect: ((0, 0), (1898, 897)),
image_mask: None,
), [
]),// [20]
Clip((
id: Clip(5, (1, 2)),
parent_space_and_clip: (
spatial_id: (3, (1, 2)),
clip_id: Clip(3, (1, 2)),
),
clip_rect: ((0, 0), (1898, 1152)),
image_mask: None,
), [
]),// [23]
ClipChain((
id: (4, (1, 2)),
parent: None,
), [
Clip(5, (1, 2)),// [0]
Clip(4, (1, 2)),// [1]
]),// [24]
Rectangle((
common: (
clip_rect: ((12, 12), (150, 7500)),
clip_id: ClipChain((4, (1, 2))),
spatial_id: (3, (1, 2)),
hit_info: None,
is_backface_visible: true,
),
color: (
r: 0,
g: 0,
b: 1,
a: 1,
),
)),// [27]
Desired output:
In an ideal world, I think we want the clips in the DL to be completely independent of window size and/or display port size. If that was the case, then the only clips present would be due to content (e.g. scroll frames, iframes, explicit clips etc) and WR could internally choose how to size / clip things based on the area being rendered.
However, it may not be feasible to do this (e.g. if the incoming Gecko display lists already have this information and it's hard to remove during conversion) or desirable (if I'm overlooking something). If that's the case, another option may be to signal in the DL format whether a clip is content related, or window / display-port related. At the moment, WR can't distinguish between these so it doesn't know whether a clip is relevant when drawing an area that intersects with it.
Another benefit of removing any clip dependence on display port / window areas is that WR won't think it's a new primitive and re-intern it when the display port region changes (which then causes invalidation of an existing tile). We might be able to work around this in other ways though.
Questions:
- Are my interpretations above correct?
- If so, how much work is involved in changing the clips to the desired output above?
- Are there other options that we can think of if that's difficult?
- If we do make changes to this for Gecko, can we land them independently of the picture caching work? Or perhaps I can take a local patch to apply while working on this?
Reporter | ||
Updated•6 years ago
|
Comment 1•6 years ago
|
||
(In reply to Glenn Watson [:gw] from comment #0)
- Are my interpretations above correct?
Generally, yes. The clips are going to be restricted by the displayport rect, which in turn is affected by the window size. The displayport is the area that APZ determines would be "good to draw" outside of the visible area, and takes into account things like scroll direction, scroll speed, whether or not we're at a document edge, and so on. The computation is based on heuristics and a bunch of tuning, and may be inappropriate for WebRender. In particular, it assumes that a larger displayport uses up a correspondingly larger amount of memory, which was definitely true when everything was getting painted to textures and passed around pre-WR, and is still somewhat true but perhaps less so with WR. There is also code to try and prevent the displayport size/position changing too often, so e.g. on Windows it should always be on a 128px boundary and move by 128px periodically as the user scrolls.
- If so, how much work is involved in changing the clips to the desired output above?
- Are there other options that we can think of if that's difficult?
While getting rid of the displayport notion entirely is difficult, we can easily just set some prefs to make the displayport arbitrarily large so that WR's internal heuristics don't ever run into the computed displayport edge. However, I'd like to discuss this in a bit more detail because I think some of what the displayport provides (or can provide) in terms of accounting for scroll direction and speed can still be useful, and WR doesn't have that information currently. I think it might be better to modify the displayport that's computed to work better with WR's picture caching so that is basically "the right thing" in terms of what you want. If you can go a bit more into detail about what you're trying to do and why the current displayport is getting in the way maybe we can figure something out here.
You said that e.g. you may want to render a 1024x1024 portion of a scroll root regardless of window size, but it's not clear to me why that should be regardless of window size. With some scroll mechanisms (e.g. keyboard, autoscroll) the number of pixels they can scroll in a given amount of time is going to be proportional to window size. So it might make sense to keep the rendered area proportionally smaller than if they had a 2000x2000 window, because then they can easily scroll much faster. The main exception to this is if they scroll by dragging the scrollthumb - and in that case we actually trim the displayport to be much smaller with the expectation that they will be scrolling far and fast, and it's better to paint the visible area much faster than it is to waste time on a large displayport that's probably not going to be useful.
Another factor that the current displayport algorithm accounts for is that nested scrollframes can be arbitrarily large. This happens often on github, if the code view for a file is horizontally scrollable. Then there's a scrollframe that's super-tall vertically but we actually do still want to bound the height of the displayport by the window size (or more precise, what we refer to as the "root composition bounds" in the code) because painting the entire thing can use up unbounded amounts of memory in the worst case.
- If we do make changes to this for Gecko, can we land them independently of the picture caching work? Or perhaps I can take a local patch to apply while working on this?
We can certainly makes changes to this and land them independently, yes. For now if you just want to experiment with mostly getting rid of the displayport clip, you can increase these prefs to be much larger:
- apz.x_skate_size_multiplier
- apz.y_skate_size_multiplier
- apz.x_stationary_size_multiplier
- apz.y_stationary_size_multiplier
In case you want to look at the current code, here is a high-level explanation of where the code lives. APZ computes a displayport in this function (which uses the above-mentioned prefs). It then sends that displayport to the Gecko main thread as part of its repaint request, with the displayport stored in the form of margins around the visible area of the scrollframe. When the main thread goes to do a paint/DL build, it reads the margins and turns it back into a rect, mostly in this function. There's comments in both functions and helper methods that roughly explain the main transformation steps, but feel free to ask if you want more details.
Updated•6 years ago
|
Reporter | ||
Comment 2•3 years ago
|
||
We correctly handle this in WR now by detecting these changing clips and either intersecting them with tile boundary, or excluding them from content and applying to the picture cache tiles.
Description
•