Closed Bug 1653409 Opened 11 months ago Closed 11 months ago

Use IOSurface rather than Client Storage to upload software-decoded video on macOS

Categories

(Core :: Graphics: WebRender, task, P3)

All
macOS
task

Tracking

()

RESOLVED FIXED
81 Branch
Tracking Status
firefox81 --- fixed

People

(Reporter: mstange, Assigned: mattwoodrow)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files)

In order to do bug 1653048 for software-decoded video, we will need to have the video data in an IOSurface. Currently we have the data in a shmem, and the data gets uploaded to a GL texture via client storage.

Not all videos will be able to use the optimized path from bug 1653048. For example, videos that are subject to certain effects such as masks, 3d transforms or filters will not be able to be promoted to "compositor surfaces". For those videos we will still need a GL texture.

We can get a GL texture for an IOSurface. That means we can use the IOSurface upload path for all videos, and we can switch to it even before we do bug 1653048.

Some more relevant info from bug 1653048:

For software decoded video and 2D canvas we currently only have a shmem. As a first step, we can do a CPU-side copy into an IOSurface in the compositor. As a second step, we should move the IOSurface closer to where the data is being generated (e.g. software video decoding) so that we can skip that copy.

We'll also need to keep in mind the buffer lifetimes. Giving an IOSurface to the window server means that the IOSurface content must not be touched until the window server is done with it. In the past, WR was doing a copy during the composite, so the IOSurface could be unlocked right after that composite. But now it will need to stay locked for longer.

Blocks: wr-mac
Severity: -- → N/A
Priority: -- → P3
See Also: → 1403618
Assignee: nobody → matt.woodrow

This overrides the colorspace of hardware decoded videos to specify the monitor's color space, to prevent CoreAnimation from doing conversion.
This probably isn't the right long term answer, but it matches our current rendering and readback code paths.

A note for the future: IOSurfaceSetValue/IOSurfaceCopyValue silently fail if the IOSurface is 'in use'.

Duplicate of this bug: 1403618
Pushed by mwoodrow@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c2626b736f34
Upload software decoded video directly to IOSurface on OSX. r=mstange
https://hg.mozilla.org/integration/autoland/rev/7b57934c0765
Specify correct color space and YCbCr matrix on IOSurfaces, so that CoreAnimation converts them correctly. r=mstange
https://hg.mozilla.org/integration/autoland/rev/4558b3cfd6fe
Set ImageBitmap's picture rect using the picture rect of layers::Image, not the size. r=mstange
Regressions: 1657312
Regressions: 1657107
Regressions: 1657734
No longer regressions: 1657312
Regressions: 1659764
You need to log in before you can comment on or make changes to this bug.