1876438 - Set scalability mode in FFmpegVideoEncoder for AV1

Status: RESOLVED FIXED

(Core :: Audio/Video: Web Codecs, task, P2)

Description

[C.M.Chang[:chunmin]]

This is required to implement the SVC setting for Webcodecs API when using AV1 codec

Comment 1

[C.M.Chang[:chunmin]]

Attachment: Bug 1876438 - Add SVC parameter support to libaomenc.c via av_opt_set

Comment 2

[C.M.Chang[:chunmin]]

Attachment: Bug 1876438 - Remove unnecessary lambda

Depends on D211951

Comment 3

[C.M.Chang[:chunmin]]

Attachment: Bug 1876438 - Enable SVC configuration in FFmpegVideoEncoder for libaom

Depends on D211952

Comment 4

[C.M.Chang[:chunmin]]

Attachment: Bug 1876438 - Update WPT expectations

Depends on D211953

Comment 5

[C.M.Chang[:chunmin]]

Attachment: WIP: Bug 1876438 - Apple SVC only in CBR mode

Depends on D211953

Comment 6

[C.M.Chang[:chunmin]]

Attachment: WIP: Bug 1876438 - Force to disable LAP for AV1 in SVC

Depends on D212170

Comment 7

[C.M.Chang[:chunmin]]

Note:

After studying the code in libaom, I found that SVC (Scalable Video Coding) in libaom only works if look-ahead processing (LAP) is disabled. This is because the LAYER_CONTEXT is only allocated for ctx->ppi->cpi->svc but not for ctx->ppi->cpi_lap->svc (svc in ctx->ppi->cpi_lap).

If LAP is enabled, running cpi_lap with av1_change_config then av1_update_layer_context_change_config will result in accessing the null cpi_lap->svc->layer_context (since layer_context is not allocated in cpi_lap), causing a segmentation fault.

Therefore, to use SVC, LAP must be disabled. LAP can be disabled in two cases: (1) if the bitrate is constant (CBR mode) or (2) if lag-in-frames is set to zero.

The libaom automatically disables LAP if the bitrate is constant. Otherwise, it calculates the num_lap_buffers value to determine if LAP should be enabled. By setting g_lag_in_frames to 0 (e.g., via av_opt_set(mCodecContext->priv_data, "lag-in-frames", "0", 0)), the num_lap_buffers will become 0 and lap_enabled will be set to false. Accordingly, no cpi_lap will be used.

Comment 8

[C.M.Chang[:chunmin]]

Attachment: Bug 1876438 - Extend timeout for temporal SVC encoding WPT

This patch extends the timeout duration in
temporal-svc-encoding.https.any.js WPT to accommodate the longer
processing time required for AV1 SVC encoding with the libaom encoder.
This adjustment is necessary to prevent timeout issues in our CI
environment.

Depends on D211954

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e0f0e274e21f
Remove unnecessary lambda r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/f62750343c4c
Add SVC parameter support to libaomenc.c via av_opt_set r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/f089bb5743b1
Enable SVC configuration in FFmpegVideoEncoder for libaom r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/b8acdb82bb51
Update WPT expectations r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/02e1ce846c4c
Extend timeout for temporal SVC encoding WPT r=media-playback-reviewers,padenot

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/46599 for changes under testing/web-platform/tests

Comment 11

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/e0f0e274e21f
https://hg.mozilla.org/mozilla-central/rev/f62750343c4c
https://hg.mozilla.org/mozilla-central/rev/f089bb5743b1
https://hg.mozilla.org/mozilla-central/rev/b8acdb82bb51
https://hg.mozilla.org/mozilla-central/rev/02e1ce846c4c

Comment 12

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot