Closed Bug 1749047 Opened 3 years ago Closed 7 months ago

[WebCodecs] Implement VideoEncoder on Linux

Categories

(Core :: Audio/Video: Web Codecs, task, P1)

task

Tracking

()

RESOLVED FIXED
123 Branch
Tracking Status
firefox123 --- fixed

People

(Reporter: chunmin, Assigned: padenot)

References

(Blocks 4 open bugs, Regressed 1 open bug)

Details

(Keywords: dev-doc-complete)

Attachments

(55 files, 11 obsolete files)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
Depends on: VideoFrame
Keywords: dev-doc-needed
Component: Audio/Video → Audio/Video: Web Codecs
Alias: VideoEncoder
Depends on: 1858574
Depends on: 1860899
Depends on: FFmpegVideoEncoder
No longer depends on: 1860899

This is debugging-only.

Depends on D195696

Depends on D195697

They will be reused for the encoding side.

Depends on D195698

https://w3c.github.io/mediacapture-record/#bitratemode, we're not implementing
this on MediaRecorder, but are needed this for Web Codecs.

Depends on D195701

Depends on D195702

It intentionally more or less maps 1:1 since my refactoring.

Depends on D195705

It's a lot of code but there is no complexity. Messages are refptrs on this side
because it simplifies the code, remove lots of moves and the messages are const
anyways so uniqueptr don't bring anything.

There are two methods for encoding, because they don't have the same
signature: the video side can take options. Because I didn't want to refactor
yet, the audio side passes in empty options for now.

I'll reflect the changes bug 1865376 in a further patch, but they are similarly
needed.

Depends on D195707

If an EncoderAgent is already present, it means this is a reconfigure, and not
e.g. the first configure call after a reset, or the first configure call ever.

This attempts to set new parameters on a PEM. This can fail, it depends on what
the PEM can do. If it succeed, all is good and this is a success.

If it fails, or if we know that the reconfigure can't succeed (codec change, or
e.g. change between hw and software encoder), the encoder is flushed, destroyed,
and a new one is created with the new params.

Depends on D195708

No complexity there, just lots of glue.

Depends on D195709

I don't think it's terribly useful but it's there to call into the PEM to check
support. We'll see if we keep it.

Depends on D195710

Very similar to the decode side, except simpler because flushing's semantic
match.

Depends on D195711

This simplifies the rest of the code, in remaining patches.

Depends on D195712

There is a bit of overlap now conceptually between latency and bitstream format,
that will be cleaned up eventually. Web Codecs requires that latency and
bitstream format to be decoupled.

Depends on D195713

Depends on D195714

Similarly, this will be decoupled in the future, but this allows running and
passing tests for now. We don't ship it anyways.

Depends on D195716

Useful, supported, and required by Web Codecs.

Depends on D195718

From https://w3c.github.io/webcodecs/#enumdef-videocolorprimaries

bt709
Color primaries used by BT.709 and sRGB, as described by [H.273] section 8.1 table 2 value 1.

Depends on D195720

Assignee: cchang → padenot
Attachment #9367349 - Attachment description: Bug 1749047 - NOT READY - AudioEncoder implementation. r?chunmin → WIP: Bug 1749047 - NOT READY - AudioEncoder implementation.
Depends on: 1869332
Depends on: 1869487
Depends on: 1869583
Depends on: 1869560
Attachment #9367336 - Attachment description: Bug 1749047 - Make AudioData::ToString public, add VideoFrame::ToString. r?chunmin → Bug 1749047 - Add VideoFrame::ToString. r?chunmin
Attachment #9367339 - Attachment description: Bug 1749047 - Add a way to print VideoData and AudioData objects for debugging. r?chunmin → Bug 1749047 - Add a way to print VideoData object for debugging. r?chunmin
Attachment #9367340 - Attachment description: Bug 1749047 - Allow comparing config and colorspace objects. r?chunmin → Bug 1749047 - Allow comparing video config and colorspace objects. r?chunmin
Attachment #9367343 - Attachment description: Bug 1749047 - Introduce the encoder types. r?chunmin → Bug 1749047 - Introduce the encoder types for the video side. r?chunmin
Attachment #9367353 - Attachment description: Bug 1749047 - Convert from MediaRawData to Encoded{Audio,Video}Chunk one by one. r?chunmin → Bug 1749047 - Convert from MediaRawData to EncodedVideoChunk one by one. r?chunmin

The previous code was causing issues when the frame rate wasn't available.

I've opened https://bugzilla.mozilla.org/show_bug.cgi?id=1869560 to revisit
this if feel we need to.

Depends on D196109

This is very useful to understand in the log what config is output in what
encoder output callback, with respect to configure(...) calls.

Depends on D196209

This logs when new configuration is output, and also logs the timestamp, to be
able to better match encode calls and output callbacks when logging.

Depends on D196210

When we a regular configure call happens, either it's the first configure call
after a reset (or at all), or we have the guarantee that the previous decoder
has been flushed, and so there are no packets in flight.

When we allow the FFmpegVideoEncoder to reconfigure on the fly, we should keep
track of what config each output should use.

Additionally, it's important to destroy / recreate the new EncoderAgent after
the packets have been output, so that ordering is correct. For this, we can
simply dispatch to the current thread, so that the runnable is after all the
output runnables. This was triggered by the WPT
reconfiguring-encoder.https.any.worker.html?vp9_p0, that doesn't output any
packet for the 16 first encode message, and the outputs 16 packets when the
encoder is flushed while reconfiguring.

Depends on D196211

Per FFmepg API's it has to be allocated on the heap and freed manually.
MakeScopeExit instead of uniqueptr + free policy is nicer I think because we
need to capture the FFmpegLibWrapper and all that.

Depends on D196212

Attachment #9367335 - Attachment is obsolete: true
Attachment #9367336 - Attachment is obsolete: true
Attachment #9367342 - Attachment is obsolete: true
Attachment #9367336 - Attachment is obsolete: false
Attachment #9367335 - Attachment is obsolete: false
Attachment #9367335 - Attachment description: Bug 1749047 - Add IsClosed on VideoFrame and AudioData. r?chunmin → Bug 1749047 - Add IsClosed on VideoFrame. r?chunmin

There is a bit of a guess there as well, but a lot less than before.

https://bugzilla.mozilla.org/show_bug.cgi?id=1869825 has been filed to
investigate this, but there's various bits to it, in gfx and media.

Depends on D196338

Depends on: 1869986

Depends on D196472

This is ffmpeg 60 only, we might have to do it ourselves for other version. A
simple queue will work, since for all codecs we intend to support, it's one
packet per video frame.

Depends on D196475

This would error out later anyways, but causes an assertion in ASAN build
because the integer isn't part of the enum.

Depends on D196871

Attachment #9367355 - Attachment is obsolete: true
Attachment #9368502 - Attachment is obsolete: true

Depends on D196872

AV_PIX_FMT_YUV420P is only defined for ffmpeg version >= 54. For
version < 54, it's PIX_FMT_YUV420P.

ffmpeg::FFMPEG_PIX_FMT_YUV420P is the type automatically mapping those
pixel types for us.

Depends on D197019

AV_PIX_FMT_YUV420P is only defined for ffmpeg version >= 54. For
version < 54, it's PIX_FMT_YUV420P.

ffmpeg::FFMPEG_PIX_FMT_YUV420P is the type automatically mapping those
pixel types for us.

Depends on D197020

mozilla::layers::Image is incomplete type since it's forward declared
in MediaData.h, unless ImageContainer.h is included.

Depends on D197021

Using a template type, ConfigType, in EncoderAgent::Configure requires a
template implementation. Instead of defining the implementation
somewehre, taking a EncoderConfig in EncoderAgent makes it easier to
add a EncoderTemplate implementation in the future, as long as the
ConfigTypeInternal comes with a ToEncoderConfig() method.

Depends on D197022

Depends on D197023

Attachment #9369755 - Attachment is obsolete: true
Attachment #9367354 - Attachment is obsolete: true
Attachment #9369756 - Attachment description: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes → WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes
Attachment #9369757 - Attachment description: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes → WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes
Attachment #9369758 - Attachment description: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp → WIP: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp
Attachment #9369756 - Attachment description: WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes → Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes
Attachment #9369757 - Attachment description: WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes → Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes
Attachment #9369758 - Attachment description: WIP: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp → Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp

Depends on D197024

Depends on: 1871457
Attachment #9369756 - Attachment description: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes → WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes
Attachment #9369757 - Attachment description: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes → WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes
Attachment #9369758 - Attachment description: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp → WIP: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp
Attachment #9369759 - Attachment description: Bug 1749047 - Take EncoderConfig in EncoderAgent::Configure → WIP: Bug 1749047 - Take EncoderConfig in EncoderAgent::Configure
Attachment #9369760 - Attachment description: Bug 1749047 - Remove extra empty line in EncoderTraits.cpp → WIP: Bug 1749047 - Remove extra empty line in EncoderTraits.cpp
Depends on: 1871520
Attachment #9369756 - Attachment description: WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes → Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes
Attachment #9369757 - Attachment description: WIP: Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes → Bug 1749047 - Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes
Attachment #9369758 - Attachment description: WIP: Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp → Bug 1749047 - Include ImageContainer.h in VideoEncoder.cpp
Attachment #9369759 - Attachment description: WIP: Bug 1749047 - Take EncoderConfig in EncoderAgent::Configure → Bug 1749047 - Take EncoderConfig in EncoderAgent::Configure
Attachment #9369760 - Attachment description: WIP: Bug 1749047 - Remove extra empty line in EncoderTraits.cpp → Bug 1749047 - Remove extra empty line in EncoderTraits.cpp

Depends on D197095

GetExtraData, which is removed in D196209, must be added back to
generate the extra data for H264 AVCC format, so the VideoEncoder can
forward those info with the encoded data.

Depends on D197137

This patch disables the WebCodecs encoding support on non-linux
platforms and adds some checks in WPTs to ensure that the execution
occurs only when the encoding is supported on the platform.

  • Add checkEncoderSupport-like code to
    video-encoder-content-hint.https.any.js
  • Add checkEncoderSupport to video-encoder-flush.https.any.js
  • Add checkEncoderSupport to video-encoder.https.any.js
    and update video-encoder.https.any.js.ini

Depends on D197138

  • For H264 encoding, FFmpegVideoEncoder requires a dynamically loaded
    ffmpeg with version > 57. However, the ffmpeg verions is 57 in ubuntu
    18.04, so video encoder module returns an "unsupported" for h264
  • The av1 encoding of FFmpegVideoEncoder relies on the in-tree libaom,
    but the libaom is out-of-date. It's vendered in 2018, and it's pretty
    slow, so it's likely av1 encoding will get timeout in some situations.

Depends on D197139

Depends on: 1871587

Run:

./mach clang-format --path \
  dom/media/platforms/ffmpeg/FFmpegVideoEncoder.cpp

Depends on D196872

Depends on: 1871485
Attachment #9370041 - Attachment is obsolete: true
Attachment #9369962 - Attachment is obsolete: true
Attachment #9369961 - Attachment is obsolete: true
Pushed by rvandermeulen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3695862104e5
Add IsClosed on VideoFrame. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/9430aed8659c
Make Clone const on VideoFrame. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/029f5470314e
Add VideoFrame::ToString. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/e903ac081227
Move and deduplicate some utility functions to WebCodecUtils. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/2d5bc871fb9e
Add a way to print VideoData object for debugging. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/700cb49e7c8a
Allow comparing video config and colorspace objects. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b6d97ebd3dda
Add BitrateMode to MediaRecorder.webidl. r=webidl,smaug
https://hg.mozilla.org/integration/autoland/rev/ee31e639b046
Introduce the encoder types for the video side. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/bc4f18f14bd3
Implement the boilerplate for VideoEncoder. r=chunmin,webidl,smaug,saschanaz
https://hg.mozilla.org/integration/autoland/rev/e3395cf67e5c
Allow converting VideoEncoderConfigInternal to something a PEM can understand. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/00a94805df47
Allow diffing and producing a configuration change list between two encoder configuration. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/5a2cef0729cc
Add an EncoderTemplate. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/3ce5091256fd
Implement encoder reconfiguration. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/a65dfea63fc7
Add EncoderTraits. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/e59a8dc5add5
Implement an EncoderAgent to bridge Web Codecs and the PEM. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/da42eb3f9664
Convert from MediaRawData to EncodedVideoChunk one by one. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/23a3b0995f32
Switch EncoderAgent::StateChange to Verbose logging. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/d8501e23246c
Allow forcing AVCC format in the AppleVTEncoder. r=jolin
https://hg.mozilla.org/integration/autoland/rev/272ecdefa3db
Handle h264 bitstream format in WebCodecs. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b32edc2659a6
Teach the AppleEncoderModule to encode in h264 High profile. r=jolin
https://hg.mozilla.org/integration/autoland/rev/88d92a0509cd
Fix enable constant bitrate mode on an AppleVTEncoder. r=jolin
https://hg.mozilla.org/integration/autoland/rev/0595d51073c6
sRGB and BT709 use the same primaries. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/a78bb8799ef4
Set the color spaces in more cases on VideoFrame. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/bcb45500ba22
Wire up more codecs from VideoEncoder to the new FFmpeg PEM. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b9234a7fb938
The encoders behind the new FFmpegVideoEncoder only support YUV420 input, convert. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/be191f9ac293
Simply use microseconds in FFmpegVideoEncoder.cpp. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/2e334c8a1160
Allow printing a VideoDecoderConfig and a VideoDecoderConfigInternal. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/2214654fd856
Improve logging around encoder output callbacks. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/12768abca8ac
Properly handle reconfiguration and configuration changes. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/7cbc5704562d
Don't leak AVPacket. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/c781f30e5605
Allow some decoder to take their color info from the codec string. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/d43f0c82f214
Guess more colorspace information when missing, based on the information present. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/279a5522919a
Add the ability to print a VideoInfo and a VideoDecoderConfigInternal. r=chunmin,media-playback-reviewers
https://hg.mozilla.org/integration/autoland/rev/ce3fa8ca07ec
Attach color space information to decoded video frame going over IPC. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/b7d997c42167
Set colorspace information on VideoFrame constructed from a GPUImage. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/26c23ac47959
Add a logging statement when outputing video frames, with the timestamp. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/f0ac27321cf8
Tell FFmpegVideoEncoder that it can encode AV1. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/ea4d0db67f89
Implement plumbing for SVC. r=chunmin,webidl,smaug
https://hg.mozilla.org/integration/autoland/rev/e99c66838796
Handle encoders having been reset while outputing encoded data. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/4d35297ae4bb
Consolidate some codec-specific checks from PEMs and add more checks for VP8 and VP9.r r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/0e68a1cc5f26
Ask ffmpeg to pass duration through. r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/7a47354c58c4
Resize at the input of the decoder if the source video frame is different from the resolution given to the encoder config. r=chunmin,media-playback-reviewers
https://hg.mozilla.org/integration/autoland/rev/cc1b04d5f70e
Clear out the colorspace member of a VideoFrame when closing it. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/05a066dcfb52
Adjust WPT expectations. r=chunmin,media-playback-reviewers
https://hg.mozilla.org/integration/autoland/rev/44fac5680126
Remove unused function to avoid breaking the build. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/d3c7d90b2150
Use a typical time base in FFmpegVideoEncoder if the framerate isn't known. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/f55b735f283a
Clearly mark the Reconfigure method for FFmpegVideoEncoder as not implemented. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/53c348265bb8
Validate H264 profile right after parsing. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/51dc14d395c6
Use FFMPEG_PIX_FMT_YUV420P for D196109 pixel format changes r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/6ecbca91effe
Use FFMPEG_PIX_FMT_YUV420P for D196477 pixel-format changes r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/8436bfea59b9
Include ImageContainer.h in VideoEncoder.cpp r=padenot
https://hg.mozilla.org/integration/autoland/rev/db7a0f8d337d
Take EncoderConfig in EncoderAgent::Configure r=padenot
https://hg.mozilla.org/integration/autoland/rev/306ad5c722c8
Remove extra empty line in EncoderTraits.cpp r=padenot
https://hg.mozilla.org/integration/autoland/rev/98d2ecb3f476
Bring GetExtraData back to ToMediaRawData r=media-playback-reviewers,padenot
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/43779 for changes under testing/web-platform/tests

This patch uses RootedDictionary around EncodedVideoChunkMetadata,
VideoDecoderConfig and VideoColorSpaceInit in OutputEncodedData to
work around the JS hazard (unrooted) issues.

https://hg.mozilla.org/mozilla-central/rev/3695862104e5
https://hg.mozilla.org/mozilla-central/rev/9430aed8659c
https://hg.mozilla.org/mozilla-central/rev/029f5470314e
https://hg.mozilla.org/mozilla-central/rev/e903ac081227
https://hg.mozilla.org/mozilla-central/rev/2d5bc871fb9e
https://hg.mozilla.org/mozilla-central/rev/700cb49e7c8a
https://hg.mozilla.org/mozilla-central/rev/b6d97ebd3dda
https://hg.mozilla.org/mozilla-central/rev/ee31e639b046
https://hg.mozilla.org/mozilla-central/rev/bc4f18f14bd3
https://hg.mozilla.org/mozilla-central/rev/e3395cf67e5c
https://hg.mozilla.org/mozilla-central/rev/00a94805df47
https://hg.mozilla.org/mozilla-central/rev/5a2cef0729cc
https://hg.mozilla.org/mozilla-central/rev/3ce5091256fd
https://hg.mozilla.org/mozilla-central/rev/a65dfea63fc7
https://hg.mozilla.org/mozilla-central/rev/e59a8dc5add5
https://hg.mozilla.org/mozilla-central/rev/da42eb3f9664
https://hg.mozilla.org/mozilla-central/rev/23a3b0995f32
https://hg.mozilla.org/mozilla-central/rev/d8501e23246c
https://hg.mozilla.org/mozilla-central/rev/272ecdefa3db
https://hg.mozilla.org/mozilla-central/rev/b32edc2659a6
https://hg.mozilla.org/mozilla-central/rev/88d92a0509cd
https://hg.mozilla.org/mozilla-central/rev/0595d51073c6
https://hg.mozilla.org/mozilla-central/rev/a78bb8799ef4
https://hg.mozilla.org/mozilla-central/rev/bcb45500ba22
https://hg.mozilla.org/mozilla-central/rev/b9234a7fb938
https://hg.mozilla.org/mozilla-central/rev/be191f9ac293
https://hg.mozilla.org/mozilla-central/rev/2e334c8a1160
https://hg.mozilla.org/mozilla-central/rev/2214654fd856
https://hg.mozilla.org/mozilla-central/rev/12768abca8ac
https://hg.mozilla.org/mozilla-central/rev/7cbc5704562d
https://hg.mozilla.org/mozilla-central/rev/c781f30e5605
https://hg.mozilla.org/mozilla-central/rev/d43f0c82f214
https://hg.mozilla.org/mozilla-central/rev/279a5522919a
https://hg.mozilla.org/mozilla-central/rev/ce3fa8ca07ec
https://hg.mozilla.org/mozilla-central/rev/b7d997c42167
https://hg.mozilla.org/mozilla-central/rev/26c23ac47959
https://hg.mozilla.org/mozilla-central/rev/f0ac27321cf8
https://hg.mozilla.org/mozilla-central/rev/ea4d0db67f89
https://hg.mozilla.org/mozilla-central/rev/e99c66838796
https://hg.mozilla.org/mozilla-central/rev/4d35297ae4bb
https://hg.mozilla.org/mozilla-central/rev/0e68a1cc5f26
https://hg.mozilla.org/mozilla-central/rev/7a47354c58c4
https://hg.mozilla.org/mozilla-central/rev/cc1b04d5f70e
https://hg.mozilla.org/mozilla-central/rev/05a066dcfb52
https://hg.mozilla.org/mozilla-central/rev/44fac5680126
https://hg.mozilla.org/mozilla-central/rev/d3c7d90b2150
https://hg.mozilla.org/mozilla-central/rev/f55b735f283a
https://hg.mozilla.org/mozilla-central/rev/53c348265bb8
https://hg.mozilla.org/mozilla-central/rev/51dc14d395c6
https://hg.mozilla.org/mozilla-central/rev/6ecbca91effe
https://hg.mozilla.org/mozilla-central/rev/8436bfea59b9
https://hg.mozilla.org/mozilla-central/rev/db7a0f8d337d
https://hg.mozilla.org/mozilla-central/rev/306ad5c722c8
https://hg.mozilla.org/mozilla-central/rev/98d2ecb3f476
https://hg.mozilla.org/mozilla-central/rev/b1577155be1c

Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
Target Milestone: --- → 123 Branch
Regressions: 1871736
Regressions: 1871739
Upstream PR merged by moz-wptsync-bot
Upstream PR merged by moz-wptsync-bot
Alias: VideoEncoder
Blocks: VideoEncoder

This bug implemented WebCodecs' video encoder but mainly focused on Linux for interop 2023. Although most of the code would be the same for other platforms, the implementations on other platforms should be tracked. I've opened a meta bug tracking this in bug 1872733, and I am going to move some dependencies there.

Blocks: 1871587
No longer blocks: webcodecs, VideoEncoder
Summary: [WebCodecs] Implement VideoEncoder → [WebCodecs] Implement VideoEncoder on Linux
Blocks: VideoEncoder
Blocks: 1867066
No longer depends on: 1858574
Duplicate of this bug: 1868404
Regressions: 1878300

FF123 docs for this can be tracked in https://github.com/mdn/content/issues/31913

Can you please clarify what this actually delivers?

  1. What desktop/mobile platforms?
    • The title says "Implement VideoEncoder on Linux" so from that I would assume that this fully supports whatever the spec says for VideoEncoder on FF123, but only on Linux desktop - not Windows, macOS or Firefox for android. However when I look at the gating preference dom.media.webcodecs.enabled I see that this is enabled on nightly for all versions, not restricted at all.
  2. Does this match the specification for VideoEncoder (and EncodedVideoChunk, VideoFrame, VideoColorSpace) or are there some known/obvious missing parts?
    • On quick scan it looks like MDN IDL matches the spec, so I guess I'm asking confirmation/things I might have missed.
    • VideoDecoder appears to be gated by same preference, but I assume does not yet need docs.
  3. Are there any special things that need to be said about this in terms of compatibility/what is delivered?
    • What I mean is that this is that the docs indicate you can use this to encode specific frames to formats supported by browser encoders, but presumably FF only supports some formats. Do we need to say what they are, or is this something people should query when needed?
Flags: needinfo?(padenot)

This delivers nothing in release, we'll handle the release notes, there seem to have been something going on.

Flags: needinfo?(padenot)

Thanks Paul, it's not just release notes. Even though this is not released we would usually record support for the feature behind pref dom.media.webcodecs.enabled in the compatibility data too. Unless there is a reason not to. Hence the question about what platforms and anything that we don't expect to be supported by this. If you "actively don't want" any support in the compatibility data or experimental features page at this point then we'd replace dev-doc-needed flag with dev-doc-complete and move on.

Flags: needinfo?(padenot)

It's a bit bumpy at the minute, it only really works on Linux desktop, but support for all platforms should come in the next few weeks.

Answering your questions now that I understand what this is about:

  1. Linux desktop only right now. I can take care of updating the browser compat thing MDN uses, I often do it myself when I implement / ship new Web API. This is Nightly-only, and is slated to ride the train and reach release in a few cycles.
  2. This matches the spec, and the decoder side also implements the spec
  3. The only thing I can think of is that encoding video frames the h264 codec isn't going to work on 5-6 years old linux, such as ubuntu 18.04 LTS, but this will fixed before release

Thanks a lot for handling this, lmk here on in chat if you need more info!

Flags: needinfo?(padenot) → needinfo?(hamishwillee)

Hi Paul,

Yes, please do update the compatibility data! When you do that, can you please cross link from your PR to https://github.com/mdn/content/issues/31913 for my tracking. I assume you are going to take care of that in the near future (i.e. while it is still behind the pref)?

I'll add entry to the MDN Experimental features page for this - from what you've said I don't think there is much else to do now.

Thanks a lot for handling this, lmk here on in chat if you need more info!

I certainly will! From what I can tell the docs look pretty complete so not much else for me to ask at this point.
But if you think there is something I should doing for this you can tag me here or on the MDN github topic with @hamishwillee.

Flags: needinfo?(hamishwillee)

OK I've added a note about this in MDN experimental features/release notes. The docs look broadly good and I don't propose to touch them.
This is waiting on compatibility to be provided by PaulA.
Work for me complete for now, so have marked this as "dev-doc-complete".

Hey! Let us know if you need any help with the compat data in BCD for this feature. It’s less urgent since it’s an initial release on Linux only and the wider support is coming soon but we always try to keep the compact data up to date with the Firefox releases.

FYI I've added a BCD showing current progress https://github.com/mdn/browser-compat-data/pull/22289

Attachment #9367349 - Attachment is obsolete: true
Regressions: 1887554
Attachment #9369959 - Attachment is obsolete: true
Attachment #9369896 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: