1467161 - Allow optimizing image surfaces more

Status: NEW

(Core :: Graphics: ImageLib, enhancement, P3)

Description

[Andrew Osmond [:aosmond] (he/him)]

There is currently a bug in imgFrame::Optimize which ignores the given DrawTarget. We should probably fix that instead of using the assumed preferred backend.

Additionally, we should consider calling Optimize if the image is layerized (e.g. uses GetImageContainer instead). We only optimize on the imgFrame::Draw path, not imgFrame::GetSourceSurface path. If we do the latter, we should ensure WebRender is unaffected by this since we don't want to optimize shared surfaces away accidently.

For animated images there is more work. If we are producing full frames off the main thread via bug 1337111, then we have the option to make imgFrame::Optimize actually work for the animated image frame. We currently can't do this because we store the individual frames via RawAccessFrameRef, but this will be changed in bug 1465619. We could remove the call to imgFrame::SetRawAccessOnly for retained frames, and guarantee volatile memory issues are gone another way (don't use volatile memory for animated images?). This could be a big win on Windows if we are retaining all the frames, because then we would only need to upload to a D2D texture once instead of on every frame change. With discarding/recycling, there is likely little to no benefit since the re-uploads are unavoidable.

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Attachment: [WIP] allow animated image frames to get optimized, v1

This is a first effort patch just to get the frames being optimized to verify what wins we could expect on Windows. This will put animated frames in heap memory so that we don't need to worry about mRawSurface releasing the frame because it is backed by a volatile buffer, as well as removing the SetRawAccessOnly calls for non-recycled, blended frames.

win64 pgo try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c000683066dba56ec6b4c574cf284c9eb501172a&selectedJob=182504745