1090323 - Speed up drawImage self-copies by making ExtractSubrect keep around a scratch DrawTarget

Status: RESOLVED WONTFIX

(Core :: Graphics: Canvas2D, defect)

Description

[Markus Stange [:mstange]]

Attachment: v1

This is an extension of bug 1083672.
On my machine this patch speeds up the A -> A microbenchmark in attachment 8505998 [details] by 70%, from around 25500 Op/s to around 44000 Op/s. That's 33% faster than the A -> B -> A copy, which is around 33000 Op/s over here.

I'm using nsExpirationTracker to free the DrawTarget after about four seconds of no use, and on memory pressure.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8512743 [details] [diff] [review]
v1

Review of attachment 8512743 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not sure this is worth doing. Self-copies are always going sort of slow and I'd rather they have predictable performance. Applications that care about this working well should just manage two buffers themselves.

Comment 2

[Markus Stange [:mstange]]

Okay.

Comment 3

[Michael Bebenita [:mbx]]

Managing multiple buffers is not always easy. Content developers would need to make sure they never allocate the destination of a blending operation in the same canvas as any of its source operands. That's not straight forward to implement, so I suspect they will either: allocate lots of canvases to ensure there are no such collisions, or make sure to copy one of the source operands into a temporary buffer before doing the blending operation.

Also, in content there is no easy way to free temporary buffers due to the lack of weak-refs. Thus, any workaround in content code is likely to be less than optimal.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

(In reply to Michael Bebenita [:mbx] from comment #3)
> Managing multiple buffers is not always easy. Content developers would need
> to make sure they never allocate the destination of a blending operation in
> the same canvas as any of its source operands. That's not straight forward
> to implement, so I suspect they will either: allocate lots of canvases to
> ensure there are no such collisions, or make sure to copy one of the source
> operands into a temporary buffer before doing the blending operation.

I'm struggling to imagine a scenario where the developer doesn't easily know whether there doing a self-copy or not. Perhaps you can help my imagination.
 
> Also, in content there is no easy way to free temporary buffers due to the
> lack of weak-refs. Thus, any workaround in content code is likely to be less
> than optimal.

Doesn't setting the size of the canvas to 0 effectively do this?

Comment 5

[Michael Bebenita [:mbx]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #4)
> (In reply to Michael Bebenita [:mbx] from comment #3)
> > Managing multiple buffers is not always easy. Content developers would need
> > to make sure they never allocate the destination of a blending operation in
> > the same canvas as any of its source operands. That's not straight forward
> > to implement, so I suspect they will either: allocate lots of canvases to
> > ensure there are no such collisions, or make sure to copy one of the source
> > operands into a temporary buffer before doing the blending operation.
> 
> I'm struggling to imagine a scenario where the developer doesn't easily know
> whether there doing a self-copy or not. Perhaps you can help my imagination.

Developers can easily detect a self-copy and use the JS workaround, but as this patch suggests, this is considerably slower than handling it natively. 

Suppose I wanted to evaluate the blending equation (A0 IN B1) OVER (C0 XOR D2), where A, B, C, D are source regions allocated across a pool of several large canvases. I'll mark them as A0, B1, C0, D1 (A in canvas 0, B in canvas 1, so on..) To evaluate it, I'll need to allocate two temporary regions, X and Y.

X = A0
X = X IN B1
Y = C0
Y = Y XOR D1
X = X OVER Y

Now, I need to make sure I allocate X and Y such that no self-copying occurs. In this case however, this is not possible without creating two new canvases. One possible such allocation is: X2, Y3

X2 = A0
X2 = X2 IN B1
Y3 = C0
Y3 = Y3 XOR D1
X2 = X2 OVER Y3

A naive allocation such as X0, Y0 would need the self-copy workaround and results in three additional copies, which is far from optimal.

X0 = A0          // T = A0, X0 = T
X0 = X0 IN B1  
Y0 = C0          // T = C0, Y0 = T
Y0 = Y0 XOR D1
X0 = X0 OVER Y0  // T = T0, X0 OVER T


I think the core of the problem is that there is no quick way to allocate and free temporary draw buffers in Canvas2D. As a result, developers need to manage their own allocation schemes and this can lead to unexpected performance problems.

> > Also, in content there is no easy way to free temporary buffers due to the
> > lack of weak-refs. Thus, any workaround in content code is likely to be less
> > than optimal.
> Doesn't setting the size of the canvas to 0 effectively do this?

Knowing when to set the size to 0 is the hard part. I would have to implement some kind of reference counting scheme that keeps track of which canvases have allocated regions and set their size to 0 after some time.