1809026 - Fix VA-API -> WebGL texture path

Status: RESOLVED FIXED

(Core :: Graphics, enhancement, P3)

Description

[Stephen]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0

Steps to reproduce:

Play https://www.youtube.com/watch?v=FWYzdJJVpIQ at 2160s60 quality on Linux with VA-API enabled and nvidia-vaapi-driver installed.

Actual results:

Video playback framerate is very low, approx. 15fps, rather than the 60fps of the source video. Selecting 720s60 results in a much smoother playback experience.

I took some profiles during playback:
2160s60 profile: https://share.firefox.dev/3vLANPd
720s60 profile: https://share.firefox.dev/3CwjDJa

They show a lot of time spent copying the frame around, which would explain why the larger resolution is much slower.

Expected results:

The video should play back smoothly.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Martin Stránský [:stransky] (ni? me)]

Can you run it on terminal with MOZ_LOG="PlatformDecoderModule:5 Dmabuf:5" env variables and attach the output here? Looks like we copy the dmabuf frames instead of share them via IPC bridge.
Thanks.

Comment 3

[Stephen]

Attachment: 1809026-vr-bug.log

Log of VR play back at 2160s with MOZ_LOG="PlatformDecoderModule:5 Dmabuf:5" set.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

DMABufSurfaceRGBA::Create() doesn't look correct in the log. I wonder what calls it and why. Do you play it from YouTube or is that a plain video file?

Comment 5

[Martin Stránský [:stransky] (ni? me)]

Also can you try latest nightly?

Comment 6

[Martin Stránský [:stransky] (ni? me)]

Yes, I understand now. It's really about https://www.youtube.com/watch?v=FWYzdJJVpIQ clip which uses video clip as texture for WebGL. So it's not about NVIDIA but how we transfer textures from VA-API to WebGL.

Comment 7

[Stephen]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #6)

Yes, I understand now. It's really about https://www.youtube.com/watch?v=FWYzdJJVpIQ clip which uses video clip as texture for WebGL. So it's not about NVIDIA but how we transfer textures from VA-API to WebGL.

Correct. I've tried the video on the latest Firefox nightly, but that has the same issue.

Comment 8

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Implement fast DMABuf WebGL textures upload r?jgilbert

Comment 9

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Deallocate DMABufSurface surface descriptor used by GPUVideoTextureData r?alwu,emilio

DMABufSurface is exported/referenced by Serialize and the reference is carried over IPC bridge (by SurfaceDescriptor) to a new DMABufSurface object.
When the new DMABufSurface object is deleted the reference is also handled.

But GPUVideoTextureData does not create DMABufSurface from SurfaceDescriptor so we need to unref it at RemoteDecoderManagerParent::RecvDeallocateSurfaceDescriptorGPUVideo().

Depends on D167240

Comment 10

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Reference DMABufSurface when it's created/imported r?emilio

Recently we add DMABufSurface reference when DMABufSurface is exported over IPC bridge by Serialize().
That no longer works as Serialize() may be called more times due to WebGL integration.

In this patch remove ref add in Serialize() (GlobalRefCountExport) and add global ref when it's
created/imported by IPC bridge.

That better matches DMABufSurface cycle in compositor as DMABufSurface is referenced on create and undef when it's deleted on compositor side.

Also add logging to ref/unref and print warning if surface ref count is already zero (we're trying to unref already 'unused' surface).

Comment 11

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Move global DMABufSurface management from VideoFrameSurface to DMABUFSurfaceImage r?alwu

Right now we add global reference to DMABufSurface in VideoFrameSurface when it's allocated by decoder and release it when DMABUFSurfaceImage deleted by MediaSupervisor.
It's not veri obvious so in this patch we move DMABufSurface references to DMABUFSurfaceImage only, we add globak ref when DMABufSurface is put to DMABUFSurfaceImage
and unref when DMABUFSurfaceImage is deleted.

Depends on D170624

Comment 12

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Check and log FFMPEG HW surfaces ID so we're sure we don't use the same surface twice r?alwu

Depends on D170625

Comment 13

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1809026 [Linux] Set Debug log ffmpeg level for VA-API decoding r?alwu

Depends on D170626

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ce20888662f7
[Linux] Implement fast DMABuf WebGL textures upload r=jgilbert

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/31d23318230a
[Linux] Reference DMABufSurface when it's created/imported r=emilio
https://hg.mozilla.org/integration/autoland/rev/602247c99125
[Linux] Move global DMABufSurface management from VideoFrameSurface to DMABUFSurfaceImage r=alwu
https://hg.mozilla.org/integration/autoland/rev/77034005a42a
[Linux] Check and log FFMPEG HW surfaces ID so we're sure we don't use the same surface twice r=alwu
https://hg.mozilla.org/integration/autoland/rev/cf328273030f
[Linux] Set Debug log ffmpeg level for VA-API decoding r=alwu

Comment 18

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/ce20888662f7
https://hg.mozilla.org/mozilla-central/rev/31d23318230a
https://hg.mozilla.org/mozilla-central/rev/602247c99125
https://hg.mozilla.org/mozilla-central/rev/77034005a42a
https://hg.mozilla.org/mozilla-central/rev/cf328273030f

Comment 19

[Stephen]

Seems like the Vimeo VR player triggers the same memcpy issue, for instance this video: https://vimeo.com/channels/360vr/277503248

Comment 20

[Martin Stránský [:stransky] (ni? me)]

(In reply to Stephen from comment #19)

Seems like the Vimeo VR player triggers the same memcpy issue, for instance this video: https://vimeo.com/channels/360vr/277503248

Yes, it's the same. But do you still see the bug with latest Nightly?

Comment 21

[Stephen]

Yes, it's the same. But do you still see the bug with latest Nightly?

Yes, same bug, with build: 20230620094433.

Comment 22

[Martin Stránský [:stransky] (ni? me)]

(In reply to Stephen from comment #21)

Yes, it's the same. But do you still see the bug with latest Nightly?

Yes, same bug, with build: 20230620094433.

Please file a new bug for it.
Thanks.

Comment 23

[Martin Stránský [:stransky] (ni? me)]

btw. Looks like https://phabricator.services.mozilla.com/D167240 is broken somehow so we don't use fast path for dmabuf surfaces.