1868403 - Implement FFmpegVideoEncoder based on modern APIs for H264

Status: RESOLVED FIXED

(Core :: Audio/Video, task, P1)

Description

[C.M.Chang[:chunmin]]

While bug 1860899 will bring an FFmpegVideoEncoder for libvpx, H264 support is required for webcodecs. FFmpegVideoEncoder should support H264 encoding via openH264 or libx264.

Comment 1

[C.M.Chang[:chunmin]]

I am trying to build ffvpx with libopenh264: https://github.com/ChunMinChang/gecko-dev/commits/b1868403/

Comment 2

[C.M.Chang[:chunmin]]

(In reply to C.M.Chang[:chunmin] from comment #1)
I probably should only use openh264 or libx264 when using dynamically-loading ffmpg, not in-tree libvpx

Comment 3

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Enable h264 via dynamically-linked ffmpeg

Depends on D196206

Comment 4

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Set H264 profile

Depends on D196251

Comment 5

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Set H264 level

Comment 6

[C.M.Chang[:chunmin]]

Attachment: WIP: Bug 1868403 - Set H264 level

Depends on D196252

Comment 7

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Set H264 encoding format

The default format for libx264 is annexb. To change the format, we need
to set x264_param_t's b_annexb to 0 via setting x264-params or
x264opts with annexb in X264_init [1,2,3,4,5,6]. For more details,
please read FFmpeg codecs document for libx264 options [7]

FFmpeg code references:
[1] https://github.com/FFmpeg/FFmpeg/blob/b51d9eb58eae046b08ef0a967ab2d3e863047d74/libavcodec/libx264.c#L1362
[2] https://github.com/FFmpeg/FFmpeg/blob/b51d9eb58eae046b08ef0a967ab2d3e863047d74/libavcodec/libx264.c#L1431
[3] https://github.com/FFmpeg/FFmpeg/blob/b51d9eb58eae046b08ef0a967ab2d3e863047d74/libavcodec/libx264.c#L1180
[4] https://github.com/FFmpeg/FFmpeg/blob/b51d9eb58eae046b08ef0a967ab2d3e863047d74/libavcodec/libx264.c#L1208

libx264 code refereces:
[5] https://code.videolan.org/videolan/x264/-/blob/c1c9931dc87289b8aeba78150467f17bdb97d019/common/base.c#L1385
[6] https://code.videolan.org/videolan/x264/-/blob/c1c9931dc87289b8aeba78150467f17bdb97d019/common/base.c#L886

FFmpeg libx264 options
[7] https://ffmpeg.org/ffmpeg-codecs.html#Options-35

Depends on D196674

Comment 8

[C.M.Chang[:chunmin]]

Attachment: WIP: Bug 1868403 - Add bit stream format to H264Specific

Comment 9

[C.M.Chang[:chunmin]]

Attachment: WIP: Bug 1868403 - Add level to H264Specific

Depends on D196680

Comment 10

[Phabricator Automation]

Comment on attachment 9369121 [details]
WIP: Bug 1868403 - Add bit stream format to H264Specific

Revision D196680 was moved to bug 1870519. Setting attachment 9369121 [details] to obsolete.

Comment 11

[Phabricator Automation]

Comment on attachment 9369122 [details]
WIP: Bug 1868403 - Add level to H264Specific

Revision D196681 was moved to bug 1870519. Setting attachment 9369122 [details] to obsolete.

Comment 12

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Set extra data in AVCC encoding

If the encoding format is AVCC and global-header is not set, every key
frame (I frame) comes with a header containing SPS and PPS data [1, 2].
This patch extract those data and put it into MediaRawData's extra data
field.

[1] https://github.com/FFmpeg/FFmpeg/blob/b51d9eb58eae046b08ef0a967ab2d3e863047d74/libavcodec/libx264.c#L1174-L1175
[2] https://code.videolan.org/videolan/x264/-/blob/c1c9931dc87289b8aeba78150467f17bdb97d019/encoder/encoder.c#L3665-L3691

Depends on D196675

Comment 13

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Test case for AVCC

Depends on D196684

Comment 14

[C.M.Chang[:chunmin]]

Attachment: Bug 1868403 - Add debugging logs

Depends on D196748

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/090ca90a322e
Enable h264 via dynamically-linked ffmpeg r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/cc02ea8a5d88
Set H264 profile r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/b5802f5738a0
Set H264 level r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/3f038df97fef
Set H264 encoding format r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/7cdf2a4559b6
Set extra data in AVCC encoding r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/38e39dfa0f67
Test case for AVCC r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/67cef656eb49
Add debugging logs r=media-playback-reviewers,padenot

Comment 16

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/090ca90a322e
https://hg.mozilla.org/mozilla-central/rev/cc02ea8a5d88
https://hg.mozilla.org/mozilla-central/rev/b5802f5738a0
https://hg.mozilla.org/mozilla-central/rev/3f038df97fef
https://hg.mozilla.org/mozilla-central/rev/7cdf2a4559b6
https://hg.mozilla.org/mozilla-central/rev/38e39dfa0f67
https://hg.mozilla.org/mozilla-central/rev/67cef656eb49