1827703 - Use OpenH264 decoder's output timestamp for decoded frames

Status: RESOLVED FIXED

(Core :: Audio/Video: GMP, defect, P3)

Description

[Andrew Osmond [:aosmond] (he/him)]

ffmpeg supplies SBufferInfo::uiInBsTimeStamp to DecodeFrameNoDelay:
https://github.com/FFmpeg/FFmpeg/blob/1179bb703e35543e15401f05a4cc81cc389756ba/libavcodec/libopenh264dec.c#L146

and takes SBufferInfo::uiOutYuvTimeStamp as the frame's new timestamp:
https://github.com/FFmpeg/FFmpeg/blob/1179bb703e35543e15401f05a4cc81cc389756ba/libavcodec/libopenh264dec.c#L146

where Firefox supplies no SBufferInfo:
https://github.com/cisco/openh264/blob/openh264v2.3.1/module/gmp-openh264.cpp#L825

and reuses the input timestamp as the output timestamp:
https://github.com/cisco/openh264/blob/openh264v2.3.1/module/gmp-openh264.cpp#L913

This causes videos such as https://html5videoplayer.net/html5video/mp4-h-264-video-test/ to appear janky when decoding using the OpenH264 2.3.1 plugin.

To fix this, we require changes on the OpenH264 side and the gecko side. This bug tracks the Gecko changes.

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1827703 - Improve integration with OpenH264 decoder.

This patch bumps the API version for the OpenH264 decoder in order to
add new methods to GMPVideoi420Frame, SetDecodedTimestamp and
DecodedTimestamp. If the API version is sufficient, the OpenH264 library
will convert the encoded sample timestamp (SBufferInfo::uiInBsTimeStamp)
to the decoded frame timestamp (SBufferInfo::uiOutYuvTimeStamp).

If it is not supported, it will default to the old behaviour and use the
encoded sample timestamp instead.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2608d24551f9
Improve integration with OpenH264 decoder. r=media-playback-reviewers,padenot

Comment 3

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/2608d24551f9