1903582 - SVC encoding with libvpx/libaom doesn't work

Status: RESOLVED FIXED

(Core :: Audio/Video: Web Codecs, defect)

Description

[Paul Adenot (:padenot)]

It appears to work because gop_size is 0, and that means "disable keyframe", so of course we can decimate and it decodes fine.

Comment 1

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Improve CBR/VBR rate setting in FFmpegDataEncoder.cpp. r?chunmin

We can change this later, but this behave reasonably for now.

Comment 2

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Add a simple function to check if SVC is enabled in FFmpegVideoEncoder.cpp. r?chunmin

Comment 3

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Set reasonable color space defaults when encoding with FFmpegVideoEncoder. r?chunmin

Comment 4

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Prefer externally checking if SVC is enabled before calling related function, for clarity. r?chunmin

Comment 5

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Don't encode all key-frames when using FFmpegVideoEncoder. r?chunmin

Comment 6

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - When non-realtime, allow multi-threaded encoding. r?chunmin

We'll have to revisit this to see if it's optimal, but for now this will do.

Comment 7

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Set reasonable quantizer default in FFmpegVideoEncoder. r?chunmin

Comment 8

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - When either SVC or low-latency is used, orient the encoder parameters to real-time / low-latency encoding. r?chunmin

More settings will be added in next patches.

Comment 9

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - In SVC or low-latency, enable error resiliency in VP9. r?chunmin

This is necessary for VP9 SVC to work.

Comment 10

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - When encoding SVC AV1 or low-latency, set a number of settings to optimize for those scenario. r?chunmin

A mix of what I see used elsewhere and also based on what "An Overview of Core
Coding Tools in the AV1 Video Codec" says (DOI: 10.1109/PCS.2018.8456249).

Comments are on each setting when appropriate.

Comment 11

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Assert that GetSvcSetting() is only called if SVC is effectively enabled. r?chunmin

Comment 12

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Put the temporal scalability mode at the end of the svc parameter list. r?chunmin

See https://searchfox.org/mozilla-central/rev/74518d4f6979b088e28405ba7e6238f4707639cf/media/ffvpx/libavcodec/libvpxenc.c#695-696

Comment 13

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Allow setting temporal id on input frames when encoding AV1 with libaom through ffmpeg. r?chunmin

Comment 14

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Rework temporal id sequencing and set id on input frames when encoding with temporal SVC using ffmpeg. r?chunmin

libaomenc.c doesn't have ts_mode=2 that automatically sets the temporal id, it
needs to be done explicitly.

Comment 15

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Synthesize fake pts when encoding AV1 using ffmpeg. r?chunmin

Comment 16

[Paul Adenot (:padenot)]

Attachment: Bug 1903582 - Remove usage of Maybe around SVC config info. r?chunmin

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bb74d8ab837f
Improve CBR/VBR rate setting in FFmpegDataEncoder.cpp. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/a288a0c0fb27
Add a simple function to check if SVC is enabled in FFmpegVideoEncoder.cpp. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/1dbb31ba9f79
Set reasonable color space defaults when encoding with FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/e786f5aca21b
Prefer externally checking if SVC is enabled before calling related function, for clarity. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/c26a5af659ac
Don't encode all key-frames when using FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/183bdf3035c1
When non-realtime, allow multi-threaded encoding. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b09e5acc46c4
Set reasonable quantizer default in FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/9484beaa3f95
When either SVC or low-latency is used, orient the encoder parameters to real-time / low-latency encoding. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/f36a68a0feef
In SVC or low-latency, enable error resiliency in VP9. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/6f88facc78e6
When encoding SVC AV1 or low-latency, set a number of settings to optimize for those scenario. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/431328ea70a1
Assert that GetSvcSetting() is only called if SVC is effectively enabled. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/c059c4081bf0
Put the temporal scalability mode at the end of the svc parameter list. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/71a2340d6d65
Allow setting temporal id on input frames when encoding AV1 with libaom through ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/7aa49376b624
Rework temporal id sequencing and set id on input frames when encoding with temporal SVC using ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/528ccb76f57a
Synthesize fake pts when encoding AV1 using ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/2fdb48718ffa
Remove usage of Maybe around SVC config info. r=pehrsons

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6721d70b77ae
Improve CBR/VBR rate setting in FFmpegDataEncoder.cpp. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/255aac08382f
Add a simple function to check if SVC is enabled in FFmpegVideoEncoder.cpp. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/de816eddbf16
Set reasonable color space defaults when encoding with FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/71cdca6b4cec
Prefer externally checking if SVC is enabled before calling related function, for clarity. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/a4e6b3323c7c
Don't encode all key-frames when using FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/15c931df3d00
When non-realtime, allow multi-threaded encoding. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/d8ab62301f52
Set reasonable quantizer default in FFmpegVideoEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/0c7fe7104989
When either SVC or low-latency is used, orient the encoder parameters to real-time / low-latency encoding. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/7a53f0ee998b
In SVC or low-latency, enable error resiliency in VP9. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/49f517c8f1b0
When encoding SVC AV1 or low-latency, set a number of settings to optimize for those scenario. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/cee7d846efef
Assert that GetSvcSetting() is only called if SVC is effectively enabled. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/6a5c9400ac44
Put the temporal scalability mode at the end of the svc parameter list. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/54ffa8bf1cbf
Allow setting temporal id on input frames when encoding AV1 with libaom through ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/1ace4b331c34
Rework temporal id sequencing and set id on input frames when encoding with temporal SVC using ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/9d2f54b895df
Synthesize fake pts when encoding AV1 using ffmpeg. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/beb5a1e95bc2
Remove usage of Maybe around SVC config info. r=pehrsons

Comment 19

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/6721d70b77ae
https://hg.mozilla.org/mozilla-central/rev/255aac08382f
https://hg.mozilla.org/mozilla-central/rev/de816eddbf16
https://hg.mozilla.org/mozilla-central/rev/71cdca6b4cec
https://hg.mozilla.org/mozilla-central/rev/a4e6b3323c7c
https://hg.mozilla.org/mozilla-central/rev/15c931df3d00
https://hg.mozilla.org/mozilla-central/rev/d8ab62301f52
https://hg.mozilla.org/mozilla-central/rev/0c7fe7104989
https://hg.mozilla.org/mozilla-central/rev/7a53f0ee998b
https://hg.mozilla.org/mozilla-central/rev/49f517c8f1b0
https://hg.mozilla.org/mozilla-central/rev/cee7d846efef
https://hg.mozilla.org/mozilla-central/rev/6a5c9400ac44
https://hg.mozilla.org/mozilla-central/rev/54ffa8bf1cbf
https://hg.mozilla.org/mozilla-central/rev/1ace4b331c34
https://hg.mozilla.org/mozilla-central/rev/9d2f54b895df
https://hg.mozilla.org/mozilla-central/rev/beb5a1e95bc2