1823135 - Support NVIDIA RTX Video Super Resolution for video overlay on Windows

Status: RESOLVED FIXED

(Core :: Graphics, enhancement)

Description

[Marco Castelluccio [:marco]]

See the FAQ at https://nvidia.custhelp.com/app/answers/detail/a_id/5448, only Chromium-based browsers are supported.

Comment 1

[Oliver Medhurst [:canadahonk]]

Attachment: Bug 1823135 - Initial support for NVIDIA RTX Video Super Resolution

Added initial support for NVIDIA's "RTX Video Super Resolution" for
video overlays on Windows, which uses hardware accelerated AI to
upscale instead of traditional upscaling methods.

Currently behind new pref "gfx.webrender.super-resolution.nvidia"
(disabled by default), allowing for prefs for other vendors later.

NVIDIA also have a setting in their own driver settings. Currently
Chromium always enable it on their end if a new enough driver and not on
battery power. This patch just checks if we are using an NVIDIA GPU and
tries to use it if the pref is enabled.

Intel also have had similar tech for a while, see Bug 1780824.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/93db8f4d488c
Initial support for NVIDIA RTX Video Super Resolution r=gfx-reviewers,sotaro

Comment 3

[Iulian Moraru]

Backed out for causing assertion failures on StaticPrefList_gfx.h.

• Push with failures

• Failure log

• Backout link

[task 2023-05-01T07:38:29.569Z] 07:38:29     INFO - [Child 8172, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005 (NS_ERROR_FAILURE): file /builds/worker/checkouts/gecko/toolkit/xre/nsXREDirProvider.cpp:475
[task 2023-05-01T07:38:29.743Z] 07:38:29     INFO - ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to C:\Users\task_168292532099759\AppData\Local\Temp\tmpxkmkqzh0.mozrunner\runreftest_leaks_rdd_pid5136.log
[task 2023-05-01T07:38:29.744Z] 07:38:29     INFO - [5136, Main Thread] WARNING: XPCOM_MEM_BLOAT_LOG is set, disabling native allocations.: file /builds/worker/checkouts/gecko/tools/profiler/core/platform.cpp:345
[task 2023-05-01T07:38:29.792Z] 07:38:29     INFO - [Parent 8792, Main Thread] WARNING: IPC message 'PContent::Msg_UpdateMediaCodecsSupported' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:504
[task 2023-05-01T07:38:29.792Z] 07:38:29     INFO - [Parent 8792, Main Thread] WARNING: IPC message 'PContent::Msg_UpdateMediaCodecsSupported' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:504
[task 2023-05-01T07:38:29.793Z] 07:38:29     INFO - [Parent 8792, Main Thread] WARNING: IPC message 'PContent::Msg_UpdateMediaCodecsSupported' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:504
[task 2023-05-01T07:38:30.008Z] 07:38:30     INFO - Assertion failure: IsAtomic<bool>::value || NS_IsMainThread() (Non-atomic static pref 'gfx.webrender.super-resolution.nvidia' being accessed on background thread by getter), at /builds/worker/workspace/obj-build/dist/include/mozilla/StaticPrefList_gfx.h:1191
[task 2023-05-01T07:38:30.011Z] 07:38:30     INFO - REFTEST INFO | drawWindow flags = DRAWWINDOW_DRAW_CARET | DRAWWINDOW_DRAW_VIEW | DRAWWINDOW_USE_WIDGET_LAYERS; window size = 816,1039; test browser size = 800,1000
[task 2023-05-01T07:38:30.287Z] 07:38:30     INFO -  Initializing stack-fixing for the first stack frame, this may take a while...
[task 2023-05-01T07:38:54.829Z] 07:38:54     INFO - #01: mozilla::StaticPrefs::gfx_webrender_super_resolution_nvidia() [s3:gecko-generated-sources:d605e08fb9ea862981b8a2f539e39dae7d81b6e9b65d1356bffd5f2622f7a78562ed815c8a4a258c4d848af846339d0a082a945627dd7ca2912134fa7fa27b76/modules/libpref/init/StaticPrefList_gfx.h::1186]
[task 2023-05-01T07:38:54.829Z] 07:38:54     INFO - #02: mozilla::wr::DCSurfaceVideo::CallVideoProcessorBlt() [gfx/webrender_bindings/DCLayerTree.cpp:1527]
[task 2023-05-01T07:38:54.830Z] 07:38:54     INFO - #03: mozilla::wr::DCSurfaceVideo::PresentVideo() [gfx/webrender_bindings/DCLayerTree.cpp:1212]
[task 2023-05-01T07:38:54.831Z] 07:38:54     INFO - #04: mozilla::wr::DCExternalSurfaceWrapper::PresentExternalSurface(mozilla::gfx::BaseMatrix<float>&) [gfx/webrender_bindings/DCLayerTree.cpp:732]
[task 2023-05-01T07:38:54.832Z] 07:38:54     INFO - #05: mozilla::wr::DCLayerTree::AddSurface(mozilla::wr::NativeSurfaceId, mozilla::wr::ScaleOffset const&, mozilla::wr::Box2D<int,mozilla::wr::DevicePixel>, mozilla::wr::ImageRendering) [gfx/webrender_bindings/DCLayerTree.cpp:768]
[task 2023-05-01T07:38:54.832Z] 07:38:54     INFO - #06: wr_compositor_add_surface(void*, mozilla::wr::NativeSurfaceId, mozilla::wr::ScaleOffset const*, mozilla::wr::Box2D<int,mozilla::wr::DevicePixel>, mozilla::wr::ImageRendering) [gfx/webrender_bindings/RenderCompositor.cpp:46]
[task 2023-05-01T07:38:54.833Z] 07:38:54     INFO - #07: webrender_bindings::bindings::impl$21::add_surface(webrender_bindings::bindings::WrCompositor*, webrender::device::gl::Device*, webrender::composite::NativeSurfaceId, webrender::util::ScaleOffset, euclid::box2d::Box2D<i32,webrender_api::units::DevicePixel>, webrender_api::display_item::ImageRendering) [gfx/webrender_bindings/src/bindings.rs:1373]
[task 2023-05-01T07:38:54.834Z] 07:38:54     INFO - #08: webrender::renderer::Renderer::draw_frame(webrender::frame_builder::Frame*, enum2$<core::option::Option<euclid::size::Size2D<i32,webrender_api::units::DevicePixel> > >, unsigned int, webrender::renderer::RenderResults*) [gfx/wr/webrender/src/renderer/mod.rs:4444]
[task 2023-05-01T07:38:54.834Z] 07:38:54     INFO - #09: webrender::renderer::Renderer::render_impl(webrender_api::DocumentId, webrender::internal_types::RenderedDocument*, enum2$<core::option::Option<euclid::size::Size2D<i32,webrender_api::units::DevicePixel> > >, unsigned int) [gfx/wr/webrender/src/renderer/mod.rs:1522]
[task 2023-05-01T07:38:54.835Z] 07:38:54     INFO - #10: webrender::renderer::Renderer::render(euclid::size::Size2D<i32,webrender_api::units::DevicePixel>, unsigned int) [gfx/wr/webrender/src/renderer/mod.rs:1239]
[task 2023-05-01T07:38:54.836Z] 07:38:54     INFO - #11: webrender_bindings::bindings::wr_renderer_render(webrender::renderer::Renderer*, int, int, unsigned int, webrender::renderer::RendererStats*, thin_vec::ThinVec<euclid::box2d::Box2D<i32,webrender_api::units::DevicePixel> >*) [gfx/webrender_bindings/src/bindings.rs:619]
[task 2023-05-01T07:38:54.836Z] 07:38:54     INFO - #12: mozilla::wr::RendererOGL::UpdateAndRender(mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&, bool*, mozilla::wr::RendererStats*) [gfx/webrender_bindings/RendererOGL.cpp:190]
[task 2023-05-01T07:38:54.838Z] 07:38:54     INFO - #13: mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, bool, mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&, bool*) [gfx/webrender_bindings/RenderThread.cpp:786]
[task 2023-05-01T07:38:54.838Z] 07:38:54     INFO - #14: mozilla::wr::RenderThread::HandleFrameOneDocInner(mozilla::wr::WrWindowId, bool, bool, mozilla::Maybe<mozilla::wr::FramePublishId>) [gfx/webrender_bindings/RenderThread.cpp:637]
[task 2023-05-01T07:38:54.839Z] 07:38:54     INFO - #15: mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId, bool, bool, mozilla::Maybe<mozilla::wr::FramePublishId>) [gfx/webrender_bindings/RenderThread.cpp:576]
[task 2023-05-01T07:38:54.840Z] 07:38:54     INFO - #16: mozilla::wr::RenderThread::WrNotifierEvent_HandleNewFrameReady(mozilla::wr::WrWindowId, bool, mozilla::wr::FramePublishId) [gfx/webrender_bindings/RenderThread.cpp:537]
[task 2023-05-01T07:38:54.840Z] 07:38:54     INFO - #17: mozilla::wr::RenderThread::HandleWrNotifierEvents(mozilla::wr::WrWindowId) [gfx/webrender_bindings/RenderThread.cpp:499]
[task 2023-05-01T07:38:54.841Z] 07:38:54     INFO - #18: mozilla::detail::RunnableMethodImpl<mozilla::wr::RenderThread *,void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId) __attribute__((thiscall)),1,0,mozilla::wr::WrWindowId>::Run() [xpcom/threads/nsThreadUtils.h:1216]
[task 2023-05-01T07:38:54.841Z] 07:38:54     INFO - #19: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1234]
[task 2023-05-01T07:38:54.842Z] 07:38:54     INFO - #20: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:479]
[task 2023-05-01T07:38:54.843Z] 07:38:54     INFO - #21: mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:330]
[task 2023-05-01T07:38:54.843Z] 07:38:54     INFO - #22: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:369]
[task 2023-05-01T07:38:54.843Z] 07:38:54     INFO - #23: MessageLoop::RunHandler() [ipc/chromium/src/base/message_loop.cc:363]
[task 2023-05-01T07:38:54.844Z] 07:38:54     INFO - #24: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:345]
[task 2023-05-01T07:38:54.845Z] 07:38:54     INFO - #25: nsThread::ThreadFunc(void*) [xpcom/threads/nsThread.cpp:393]
[task 2023-05-01T07:38:55.038Z] 07:38:55     INFO - #26: _PR_NativeRunThread(void*) [nsprpub/pr/src/threads/combined/pruthr.c:399]
[task 2023-05-01T07:38:55.038Z] 07:38:55     INFO - #27: pr_root(void*) [nsprpub/pr/src/md/windows/w95thred.c:139]
[task 2023-05-01T07:38:55.042Z] 07:38:55     INFO -  fix-stacks: error: failed to read debug info file `ucrtbase.pdb` for `C:\Windows\System32\ucrtbase.dll`
[task 2023-05-01T07:38:55.042Z] 07:38:55     INFO -  fix-stacks: note: this is expected and harmless for all PDB files on opt automation runs
[task 2023-05-01T07:38:55.042Z] 07:38:55     INFO -  fix-stacks: The system cannot find the file specified. (os error 2)
[task 2023-05-01T07:38:55.043Z] 07:38:55     INFO - #28: time32 [C:\Windows\System32\ucrtbase.dll + 0x459d3]
[task 2023-05-01T07:38:55.045Z] 07:38:55     INFO -  fix-stacks: error: failed to read debug info file `wkernel32.pdb` for `C:\Windows\System32\KERNEL32.DLL`
[task 2023-05-01T07:38:55.046Z] 07:38:55     INFO -  fix-stacks: note: this is expected and harmless for all PDB files on opt automation runs
[task 2023-05-01T07:38:55.046Z] 07:38:55     INFO -  fix-stacks: The system cannot find the file specified. (os error 2)
[task 2023-05-01T07:38:55.047Z] 07:38:55     INFO - #29: BaseThreadInitThunk [C:\Windows\System32\KERNEL32.DLL + 0x17d69]
[task 2023-05-01T07:38:55.156Z] 07:38:55     INFO - #30: DllBlocklist_Initialize [Z:\task_168292532099759\build\application\firefox\mozglue.dll + 0x7d6a3]
[task 2023-05-01T07:38:55.188Z] 07:38:55     INFO -  fix-stacks: error: failed to read debug info file `wntdll.pdb` for `C:\Windows\SYSTEM32\ntdll.dll`
[task 2023-05-01T07:38:55.189Z] 07:38:55     INFO -  fix-stacks: note: this is expected and harmless for all PDB files on opt automation runs
[task 2023-05-01T07:38:55.189Z] 07:38:55     INFO -  fix-stacks: The system cannot find the file specified. (os error 2)
[task 2023-05-01T07:38:55.190Z] 07:38:55     INFO - #31: RtlInitializeExceptionChain [C:\Windows\SYSTEM32\ntdll.dll + 0x6bb9b]
[task 2023-05-01T07:38:55.190Z] 07:38:55     INFO - #32: RtlClearBits [C:\Windows\SYSTEM32\ntdll.dll + 0x6bb1f]
[task 2023-05-01T07:38:55.191Z] 07:38:55     INFO - [Parent 8792, IPC I/O Parent] WARNING: [1.1]: Ignoring message 'EVENT_MESSAGE' to unknown peer 3F7F4CECC256840E.EF23FE9813665951: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:344

Comment 4

[Oliver Medhurst [:canadahonk]]

Fixed in patch, pref was set to incorrect mirror value which didn't come up in local tests.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0cc9bdf506f3
Initial support for NVIDIA RTX Video Super Resolution r=gfx-reviewers,sotaro

Comment 6

[Sandor Molnar[:smolnar]]

Backed out for causing build bustages in gfx/webrender_bindings/DCLayerTree.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/68131c211e7c412e25bcd448432d546c7e07c691

Push with failures

Failure log -> ERROR - /builds/worker/checkouts/gecko/gfx/webrender_bindings/DCLayerTree.cpp(1528,20): error: no member named 'gfx_webrender_super_resolution_nvidia' in namespace 'mozilla::StaticPrefs'

Comment 7

[Oliver Medhurst [:canadahonk]]

Apologies, wasn't expecting a build error from the trivial change. Patch fixed again.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8bdd3602183b
Initial support for NVIDIA RTX Video Super Resolution r=gfx-reviewers,sotaro

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/8bdd3602183b

Comment 10

[Jim Mathies [:jimm]]

Thanks for the contribution CanadaHonk!