2085643 - FFmpegVideoDecoder leaks reference to ANativeWindow

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Andrew Osmond [:aosmond] (he/him)]

We supply AVMediaCodecDeviceContext an ANativeWindow pointer in native_window at initialization of a MediaCodec decoder:
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp#3205
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp#3223

It looks like we free it when we free the context:
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/media/ffvpx/libavutil/hwcontext_mediacodec.c#99-102

But that is misleading because it has a condition just above it:
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/media/ffvpx/libavutil/hwcontext_mediacodec.c#96

That is only satisfied if set libmedia in mc_device_init
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/media/ffvpx/libavutil/hwcontext_mediacodec.c#79

Which we won't due to the early return since we already set native_window
https://searchfox.org/firefox-main/rev/efd2d1e6f77570ae6bc5294731a68101e45c81a3/media/ffvpx/libavutil/hwcontext_mediacodec.c#71-72

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 2085643 - Release the ANativeWindow reference given to the MediaCodec device context.

FFmpeg only releases AVMediaCodecDeviceContext::native_window if it created
the window itself, which it never does for us because we always provide one.
Release our reference from the device context's free callback, which runs once
the last reference to the context is dropped, including on init failure.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/079338aef0f4
https://hg.mozilla.org/integration/autoland/rev/e739a0d07fd4
Release the ANativeWindow reference given to the MediaCodec device context. r=media-playback-reviewers,padenot

Comment 3

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e739a0d07fd4