1912543 - Assertion failure: !frameSize.IsEmpty(), at /builds/worker/checkouts/gecko/dom/html/HTMLVideoElement.cpp:759

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Tyson Smith [:tsmith]]

Found with m-c 20240802-c38029641964 (--enable-debug --enable-fuzzing)

This was found by visiting a live website with a debug build.

STR:

• Launch browser and visit site

This issue was triggered by visiting http://dealerinspire.com/.

This has been triggered by visiting a few different sites so far:

• http://www.stjude.org/
• http://service-now.com/
• http://www.motorolasolutions.com/

Assertion failure: !frameSize.IsEmpty(), at /builds/worker/checkouts/gecko/dom/html/HTMLVideoElement.cpp:759

0|0|xul.dll|mozilla::dom::HTMLVideoElement::TakeVideoFrameRequestCallbacks(mozilla::TimeStamp const&, mozilla::Maybe<mozilla::TimeStamp> const&, mozilla::dom::VideoFrameCallbackMetadata&, nsTArray<mozilla::dom::RequestCallbackEntry<mozilla::dom::VideoFrameRequestCallback> >&)|hg:hg.mozilla.org/mozilla-central:dom/html/HTMLVideoElement.cpp:c38029641964591e518535856e2d7b3038b134ad|759|0x3f8
0|1|xul.dll|nsRefreshDriver::RunVideoFrameCallbacks(nsTArray<RefPtr<mozilla::dom::Document> > const&, mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:c38029641964591e518535856e2d7b3038b134ad|2415|0x2e3
0|2|xul.dll|nsRefreshDriver::RunVideoAndFrameRequestCallbacks(mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:c38029641964591e518535856e2d7b3038b134ad|2537|0x358
0|3|xul.dll|nsRefreshDriver::Tick(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::TimeStamp, nsRefreshDriver::IsExtraTick)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:c38029641964591e518535856e2d7b3038b134ad|2772|0x949
0|4|xul.dll|mozilla::InactiveRefreshDriverTimer::TickOne()|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:c38029641964591e518535856e2d7b3038b134ad|1151|0x114
0|5|xul.dll|mozilla::InactiveRefreshDriverTimer::TimerTickOne(nsITimer*, void*)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:c38029641964591e518535856e2d7b3038b134ad|1160|0x4c
0|6|xul.dll|nsTimerImpl::Fire(int)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsTimerImpl.cpp:c38029641964591e518535856e2d7b3038b134ad|675|0x745
0|7|xul.dll|nsTimerEvent::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TimerThread.cpp:c38029641964591e518535856e2d7b3038b134ad|515|0x6ff
0|8|xul.dll|mozilla::RunnableTask::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:c38029641964591e518535856e2d7b3038b134ad|618|0x1b
0|9|xul.dll|mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex &> const&)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:c38029641964591e518535856e2d7b3038b134ad|945|0xa00
0|10|xul.dll|mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex &> const&)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:c38029641964591e518535856e2d7b3038b134ad|768|0x56
0|11|xul.dll|mozilla::TaskController::ProcessPendingMTTask(bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:c38029641964591e518535856e2d7b3038b134ad|554|0x51
0|12|xul.dll|mozilla::detail::RunnableFunction<`lambda at /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:271:7'>::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.h:c38029641964591e518535856e2d7b3038b134ad|548|0x13
0|13|xul.dll|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:c38029641964591e518535856e2d7b3038b134ad|1204|0x753
0|14|xul.dll|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:c38029641964591e518535856e2d7b3038b134ad|480|0x63
0|15|xul.dll|mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:c38029641964591e518535856e2d7b3038b134ad|107|0x126
0|16|xul.dll|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|370|0x82
0|17|xul.dll|MessageLoop::RunHandler()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|363|0x70
0|18|xul.dll|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|345|0x55
0|19|xul.dll|nsBaseAppShell::Run()|hg:hg.mozilla.org/mozilla-central:widget/nsBaseAppShell.cpp:c38029641964591e518535856e2d7b3038b134ad|148|0x24
0|20|xul.dll|nsAppShell::Run()|hg:hg.mozilla.org/mozilla-central:widget/windows/nsAppShell.cpp:c38029641964591e518535856e2d7b3038b134ad|655|0x15e
0|21|xul.dll|XRE_RunAppShell()|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:c38029641964591e518535856e2d7b3038b134ad|714|0x60
0|22|xul.dll|mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:c38029641964591e518535856e2d7b3038b134ad|235|0x35
0|23|xul.dll|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|370|0x82
0|24|xul.dll|MessageLoop::RunHandler()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|363|0x70
0|25|xul.dll|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:c38029641964591e518535856e2d7b3038b134ad|345|0x55
0|26|xul.dll|XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:c38029641964591e518535856e2d7b3038b134ad|649|0x83e
0|27|xul.dll|mozilla::BootstrapImpl::XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/Bootstrap.cpp:c38029641964591e518535856e2d7b3038b134ad|71|0x10
0|28|firefox.exe|NS_internal_main(int, char**, char**)|hg:hg.mozilla.org/mozilla-central:browser/app/nsBrowserApp.cpp:c38029641964591e518535856e2d7b3038b134ad|378|0x292

Comment 1

[Alastor Wu [:alwu]]

It's an issue related with requestVideoFrameCallback, Andrew could you take a look on this? Thanks!

Comment 2

[Tyson Smith [:tsmith]]

A Pernosco session is available here: https://pernos.co/debug/ioh1G3YQ5nVDqvkglNI95w/index.html

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

We appear to have created a dummy frame:
https://searchfox.org/mozilla-central/rev/3f193a8bb0759058c2f9a9d7761ad73bc6ffd4b1/dom/media/platforms/agnostic/NullDecoderModule.cpp#17

This happens when we have suspended video playback, typically when the tab is backgrounded. This does beg the question, what should we do in this case? We already throttle requestAnimationFrame callbacks for backgrounded tabs, and by extension, requestVideoFrameCallback. We definitely should not trigger the callback for dummy/empty frames, which is an easy enough fix, but presumably we should consider preventing video suspension and/or unsuspending the video if we have a pending requestVideoFrameCallback request.

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1912543 - Fix requestVideoFrameCallback with suspended/invisible videos.

When a tab is backgrounded, or on some platforms, when the window is
fully covered, full decoding of a video for playback is disabled. The
video timeline still advances, but there are no valid frames for
presentation. Similarly, if an HTMLVideoElement is in the DOM tree but
marked as invisible (e.g. 'display: none;' is set), we also cease full
decoding of a video. This is the VideoDecodeMode::Suspend mode.

Chrome and Safari both continue to honour requestVideoFrameCallback when
the video element is invisible in a foreground tab. Conversely, when we
are in a backgrounded tab, Chrome suspends rVFC callbacks, while Safari
continues.

Given that we suspend requestAnimationFrame callbacks similar to Chrome
for backgrounded tabs, this patch matches our behaviour with Chrome.

The standard does not discuss the implications of visibility and
background/foreground on rVFC. We have filed an issue requesting for
clarification, and that can tracked at:

https://github.com/WICG/video-rvfc/issues/92

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

Marking as disabled in 131, as the feature is nightly only right now. I expect to ship this in 132 once we fix the remaining gaps.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7e3e16f06a4e
Fix requestVideoFrameCallback with suspended/invisible videos. r=media-playback-reviewers,padenot

Comment 7

[tszentpeteri]

https://hg.mozilla.org/mozilla-central/rev/7e3e16f06a4e