Closed Bug 1867542 Opened 1 year ago Closed 1 year ago

Overhaul PlatformEncoderModules to prepare for Web Codecs

Categories

(Core :: Audio/Video: Web Codecs, defect)

defect

Tracking

()

RESOLVED FIXED
122 Branch
Tracking Status
firefox122 --- fixed

People

(Reporter: padenot, Assigned: padenot)

References

Details

Attachments

(21 files)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
No description provided.

This change the configuration interface, simplifying it. It moves hardware codec
preference to config, move the task queue outside of the config, adds a source
pixel format config, removes usage of TrackInfo (that is really a playback side
thing), adds a bitrate mode configuration option (variable, constant,
quantizer).

This patch is not intended to contain any functional changes.

Depends on D195134

Depends on D195135

There contains lots of changes but no functional changes is intended.

Depends on D195136

The intent is that this doesn't not change the defaults, but allows configuring
more aspects of the encoding.

Depends on D195142

This is essential to encode e.g. <canvas> readbacks.

Depends on D195143

A configuration change is an array of sum types that each describe a
configuration change. Sometimes the configuration can be the empty type, in
which case the PEMs should attempt to set the value to a default that is
maybe platform-specific.

We will add to this list of possible changes over time but this is a good first
set.

Depends on D195145

No functional changes intended, it should behave the same.

Depends on D195148

A change: attempting to create an H264 encoder without specifying the profile is
expected to fail.

Depends on D195151

Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8046ebc92c89 Remove unused include in H264.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/7b8ab99e9e8f Fix static-analysis warnings in H264.cpp and H264.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/e0eee03289ec Simplify PlatformEncoderModule interface, allow asynchronous creation. r=jolin https://hg.mozilla.org/integration/autoland/rev/71b3d9090b65 Reflect PEM changes to PEMFactory. r=jolin https://hg.mozilla.org/integration/autoland/rev/d6b9fef929fc Reflect PEM changes to AndroidEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/6eac4790b134 Fix static-analysis warnings in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/1d2eca5c1554 Fix static-analysis warnings in PlatformDecoderModule.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/231ef5da7785 Fix static-analysis warning in AppleDecoderModule.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/2317688d59db Remove unused header in AppleVTDecoder.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/cba1ad0e322f Reflect PEM changes to AppleEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/19c2ba439ebe Teach more configuration options to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/12fc139ae866 Teach AppleVTEncoder to have RGB8 surfaces as input. r=jolin https://hg.mozilla.org/integration/autoland/rev/148d6cbcd5da Introduce a generic configuration change interface to the PEM inteface. r=jolin https://hg.mozilla.org/integration/autoland/rev/0b4b4c4a5dcd Adds an empty Reconfigure method in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/b2526be85618 Add a functional Reconfigure implementation to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/a36a47cdcb51 Reflect PlatformEncoderModule changes to WebRTCMediaDataEncoderCodec. r=jolin https://hg.mozilla.org/integration/autoland/rev/4dbe699e3b18 Fix static-analysis warnings in AppleVTDecoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/b705d069361f Fix static-analysis warnings in PlatformDecoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/8258253d875d Update TestMediaDataEncoder.cpp to reflect changes in PlatformEncoderModule and associated APIs. r=jolin

Backed out for causing build bustages on WMFEncoderModule.h.
This affected only Windows 2012 platforms.

[task 2023-12-07T14:21:16.817Z] 14:21:16     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/obj-build/dom/media/platforms'
[task 2023-12-07T14:21:16.818Z] 14:21:16     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang-cl -fms-compatibility-version=19.29 -Xclang -std=c++17 -m32 -Xclang -ivfsoverlay -Xclang /builds/worker/fetches/vs/overlay.yaml -FoUnified_cpp_dom_media_platforms0.obj -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -guard:cf -DNDEBUG=1 -DTRIMMED=1 -DUNICODE -D_UNICODE -D_CRT_RAND_S -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -D_SECURE_ATL -DCHROMIUM_BUILD -DU_STATIC_IMPLEMENTATION -DWIN32 -D_WIN32 -D_WINDOWS -DWIN32_LEAN_AND_MEAN -DWINAPI_NO_BUNDLED_LIBRARIES -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/dom/media/platforms -I/builds/worker/workspace/obj-build/dom/media/platforms -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -MD -FI /builds/worker/workspace/obj-build/mozilla-config.h -DMOZILLA_CLIENT -fcrash-diagnostics-dir=/builds/worker/artifacts -TP -Zc:sizedDealloc- -D_HAS_EXCEPTIONS=0 -Gy -Zc:inline -arch:SSE2 -Gw -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -GR- -Z7 -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -Oy- -Werror -W3 -Wbitfield-enum-conversion -Wdeprecated-this-capture -Wempty-body -Wformat-type-confusion -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtautological-constant-in-range-compare -Wtype-limits -Wno-error=tautological-type-limit-compare -Wunreachable-code -Wunreachable-code-return -Wunused-but-set-parameter -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wno-range-loop-analysis -Wc++2a-compat -Wenum-compare-conditional -Wenum-float-conversion -Wvolatile -Wno-error=deprecated -Wno-error=deprecated-anon-enum-enum-conversion -Wno-error=deprecated-enum-enum-conversion -Wno-error=deprecated-pragma -Wno-error=deprecated-this-capture -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wno-error=atomic-alignment -Wno-error=deprecated-builtins -Wno-unknown-pragmas -Wno-ignored-pragmas -Wno-deprecated-declarations -Wno-microsoft-enum-value -Wno-microsoft-include -Wno-invalid-noreturn -Wno-inconsistent-missing-override -Wno-implicit-exception-spec-mismatch -Wno-microsoft-exception-spec -Wno-unused-local-typedef -Wno-ignored-attributes -Wno-used-but-marked-unused -Wno-psabi -Wthread-safety -Wno-error=builtin-macro-redefined -Wno-vla-cxx-extension -Wno-unknown-warning-option -fno-strict-aliasing -Xclang -ffp-contract=off  -Xclang -MP -Xclang -dependency-file -Xclang .deps/Unified_cpp_dom_media_platforms0.obj.pp -Xclang -MT -Xclang Unified_cpp_dom_media_platforms0.obj   Unified_cpp_dom_media_platforms0.cpp
[task 2023-12-07T14:21:16.819Z] 14:21:16     INFO -  In file included from Unified_cpp_dom_media_platforms0.cpp:29:
[task 2023-12-07T14:21:16.819Z] 14:21:16     INFO -  In file included from /builds/worker/checkouts/gecko/dom/media/platforms/PEMFactory.cpp:20:
[task 2023-12-07T14:21:16.820Z] 14:21:16    ERROR -  /builds/worker/workspace/obj-build/dist/include/WMFEncoderModule.h(15,60): error: only virtual member functions can be marked 'override'
[task 2023-12-07T14:21:16.820Z] 14:21:16     INFO -     15 |   bool SupportsMimeType(const nsACString& aMimeType) const override;
[task 2023-12-07T14:21:16.821Z] 14:21:16     INFO -        |                                                            ^~~~~~~~
[task 2023-12-07T14:21:16.821Z] 14:21:16    ERROR -  /builds/worker/workspace/obj-build/dist/include/WMFEncoderModule.h(18,13): error: unknown type name 'CreateEncoderParams'; did you mean 'CreateDecoderParams'?
[task 2023-12-07T14:21:16.821Z] 14:21:16     INFO -     18 |       const CreateEncoderParams& aParams,
[task 2023-12-07T14:21:16.822Z] 14:21:16     INFO -        |             ^~~~~~~~~~~~~~~~~~~
[task 2023-12-07T14:21:16.822Z] 14:21:16     INFO -        |             CreateDecoderParams
[task 2023-12-07T14:21:16.822Z] 14:21:16     INFO -  /builds/worker/checkouts/gecko/dom/media/platforms/PDMFactory.h(29,8): note: 'CreateDecoderParams' declared here
[task 2023-12-07T14:21:16.823Z] 14:21:16     INFO -     29 | struct CreateDecoderParams;
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -        |        ^
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -  In file included from Unified_cpp_dom_media_platforms0.cpp:29:
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -  In file included from /builds/worker/checkouts/gecko/dom/media/platforms/PEMFactory.cpp:20:
[task 2023-12-07T14:21:16.826Z] 14:21:16    ERROR -  /builds/worker/workspace/obj-build/dist/include/WMFEncoderModule.h(19,45): error: non-virtual member function marked 'override' hides virtual member function
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -     19 |       const bool aHardwareNotAllowed) const override;
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -        |                                             ^
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -  /builds/worker/checkouts/gecko/dom/media/platforms/PlatformEncoderModule.h(104,46): note: hidden overloaded virtual function 'mozilla::PlatformEncoderModule::CreateVideoEncoder' declared here: type mismatch at 1st parameter ('const EncoderConfig &' vs 'const CreateDecoderParams &')
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -    104 |   virtual already_AddRefed<MediaDataEncoder> CreateVideoEncoder(
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -        |                                              ^
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -  In file included from Unified_cpp_dom_media_platforms0.cpp:29:
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -  In file included from /builds/worker/checkouts/gecko/dom/media/platforms/PEMFactory.cpp:20:
[task 2023-12-07T14:21:16.826Z] 14:21:16    ERROR -  /builds/worker/workspace/obj-build/dist/include/WMFEncoderModule.h(13,7): error: abstract class is marked 'final' [-Werror,-Wabstract-final-class]
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -     13 | class WMFEncoderModule final : public PlatformEncoderModule {
[task 2023-12-07T14:21:16.826Z] 14:21:16     INFO -        |       ^
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -  /builds/worker/checkouts/gecko/dom/media/platforms/PlatformEncoderModule.h(118,16): note: unimplemented pure virtual method 'Supports' in 'WMFEncoderModule'
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -    118 |   virtual bool Supports(const EncoderConfig& aConfig) const = 0;
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -        |                ^
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -  /builds/worker/checkouts/gecko/dom/media/platforms/PlatformEncoderModule.h(119,16): note: unimplemented pure virtual method 'SupportsCodec' in 'WMFEncoderModule'
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -    119 |   virtual bool SupportsCodec(CodecType aCodecType) const = 0;
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -        |                ^
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -  /builds/worker/checkouts/gecko/dom/media/platforms/PlatformEncoderModule.h(122,23): note: unimplemented pure virtual method 'GetName' in 'WMFEncoderModule'
[task 2023-12-07T14:21:16.827Z] 14:21:16     INFO -    122 |   virtual const char* GetName() const = 0;
[task 2023-12-07T14:21:16.828Z] 14:21:16     INFO -        |                       ^
[task 2023-12-07T14:21:16.828Z] 14:21:16     INFO -  In file included from Unified_cpp_dom_media_platforms0.cpp:29:
[task 2023-12-07T14:21:16.828Z] 14:21:16    ERROR -  /builds/worker/checkouts/gecko/dom/media/platforms/PEMFactory.cpp(45,3): error: use of undeclared identifier 'mModules'
[task 2023-12-07T14:21:16.828Z] 14:21:16     INFO -     45 |   mModules.AppendElement(new WMFEncoderModule());
<...>
Flags: needinfo?(padenot)
Flags: needinfo?(padenot)
Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/0f9376cb7b68 Remove unused include in H264.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/4534cfd92775 Fix static-analysis warnings in H264.cpp and H264.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/af86ae97ca08 Simplify PlatformEncoderModule interface, allow asynchronous creation. r=jolin https://hg.mozilla.org/integration/autoland/rev/b31389120219 Reflect PEM changes to PEMFactory. r=jolin https://hg.mozilla.org/integration/autoland/rev/8a0c3faaeb96 Reflect PEM changes to AndroidEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/ba45280ca500 Fix static-analysis warnings in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/21d7101ba9a3 Fix static-analysis warnings in PlatformDecoderModule.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/5f7689ac0357 Fix static-analysis warning in AppleDecoderModule.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/94f6b561b4d5 Remove unused header in AppleVTDecoder.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/7abe1f239b4f Reflect PEM changes to AppleEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/ce2e32e30761 Teach more configuration options to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/1e79f4a01297 Teach AppleVTEncoder to have RGB8 surfaces as input. r=jolin https://hg.mozilla.org/integration/autoland/rev/b56ec46251a9 Introduce a generic configuration change interface to the PEM inteface. r=jolin https://hg.mozilla.org/integration/autoland/rev/e87e1bb6bcf8 Adds an empty Reconfigure method in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/63f403a0f1ff Add a functional Reconfigure implementation to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/afb09d4a02f9 Reflect PlatformEncoderModule changes to WebRTCMediaDataEncoderCodec. r=jolin https://hg.mozilla.org/integration/autoland/rev/7af4d3fe37ba Fix static-analysis warnings in AppleVTDecoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/46c83c6cf9bb Fix static-analysis warnings in PlatformDecoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/6ea19cc6a14b Update TestMediaDataEncoder.cpp to reflect changes in PlatformEncoderModule and associated APIs. r=jolin https://hg.mozilla.org/integration/autoland/rev/4a7b2f9ad22b Reflect PEM changes to WMFEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/a092c177e959 apply code formatting via Lando

(In reply to Sandor Molnar[:smolnar] from comment #24)

Failure log -> TEST-UNEXPECTED-FAIL | MediaDataEncoderTest.InvalidSize | Expected: (e0x0) != (nullptr), actual: 0x0 vs (nullptr)

From the code, it looks like AppleEncoderModule::Supports rejects 0 width or 0 height, so PEMFactory::FindPEM and CreateEncoder return nullptr, then CreateVideoEncoder/CreateH264Encoder got a nullptr and EXPECT_NE(e0x0, nullptr) failed.

arg I pushed the verison of the patch without this fix.

Flags: needinfo?(padenot)
Attachment #9367738 - Attachment description: Bug 1867542 - Update the new ffmpeg PEM with the new PEM interface. r?chunmin → Bug 1867542 - Update the new ffmpeg PEM with the new PEM interface. r?chunmin
Pushed by cchang@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/75a513213553 Remove unused include in H264.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/a18b841b42ec Fix static-analysis warnings in H264.cpp and H264.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/0e46d39d417e Simplify PlatformEncoderModule interface, allow asynchronous creation. r=jolin https://hg.mozilla.org/integration/autoland/rev/2c27edf81401 Reflect PEM changes to PEMFactory. r=jolin https://hg.mozilla.org/integration/autoland/rev/4e5ad412c86d Reflect PEM changes to AndroidEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/ddb637201de9 Fix static-analysis warnings in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/0b4ad45024d3 Fix static-analysis warnings in PlatformDecoderModule.h. r=jolin https://hg.mozilla.org/integration/autoland/rev/c125453bdb25 Fix static-analysis warning in AppleDecoderModule.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/f22739070fe7 Remove unused header in AppleVTDecoder.cpp. r=jolin https://hg.mozilla.org/integration/autoland/rev/49c8a5f17da6 Reflect PEM changes to AppleEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/9c246a20c9f4 Teach more configuration options to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/cd76148ccea6 Teach AppleVTEncoder to have RGB8 surfaces as input. r=jolin https://hg.mozilla.org/integration/autoland/rev/509a405ae33e Introduce a generic configuration change interface to the PEM inteface. r=jolin https://hg.mozilla.org/integration/autoland/rev/4cc3144411cf Adds an empty Reconfigure method in AndroidDataEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/fb9c7d8765ae Add a functional Reconfigure implementation to AppleVTEncoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/541592e33d98 Reflect PlatformEncoderModule changes to WebRTCMediaDataEncoderCodec. r=jolin https://hg.mozilla.org/integration/autoland/rev/fc1cf5f53b1f Fix static-analysis warnings in AppleVTDecoder. r=jolin https://hg.mozilla.org/integration/autoland/rev/e45477257958 Fix static-analysis warnings in PlatformDecoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/61b624916ac3 Update TestMediaDataEncoder.cpp to reflect changes in PlatformEncoderModule and associated APIs. r=jolin https://hg.mozilla.org/integration/autoland/rev/ced184732f69 Reflect PEM changes to WMFEncoderModule. r=jolin https://hg.mozilla.org/integration/autoland/rev/d7f89f5b4970 Update the new ffmpeg PEM with the new PEM interface. r=chunmin https://hg.mozilla.org/integration/autoland/rev/19967878c4b0 apply code formatting via Lando
Blocks: 1869186
Regressions: 1917627
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: