1839264 - Assertion failure: false (MOZ_ASSERT_UNREACHABLE: unexpected to be called), at /builds/worker/checkouts/gecko/gfx/layers/RemoteTextureMap.cpp:667

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

Found while fuzzing m-c 20230519-d97636946466 (--enable-debug --enable-fuzzing)

To reproduce via Grizzly Replay:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch -d --fuzzing --cpu x86 -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.html

Assertion failure: false (MOZ_ASSERT_UNREACHABLE: unexpected to be called), at /builds/worker/checkouts/gecko/gfx/layers/RemoteTextureMap.cpp:667

6|0|xul.dll|mozilla::layers::RemoteTextureMap::GetRemoteTextureForDisplayList(mozilla::layers::RemoteTextureHostWrapper*, std::function<void (const mozilla::layers::RemoteTextureInfo &)>&&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/RemoteTextureMap.cpp:a9e342522ebdb9602671208a06e988a285d3af18|667|0x711
6|1|xul.dll|mozilla::layers::WebRenderImageHost::UseRemoteTexture()|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderImageHost.cpp:a9e342522ebdb9602671208a06e988a285d3af18|220|0x2bc
6|2|xul.dll|mozilla::layers::CompositableParentManager::ReceiveCompositableUpdate(mozilla::layers::CompositableOperationDetail const&, mozilla::NotNull<mozilla::layers::CompositableHost *>, mozilla::layers::CompositableHandle const&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/ipc/CompositableTransactionParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|95|0x298
6|3|xul.dll|mozilla::layers::CompositableParentManager::ReceiveCompositableUpdate(mozilla::layers::CompositableOperation const&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/ipc/CompositableTransactionParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|35|0x82
6|4|xul.dll|mozilla::layers::WebRenderBridgeParent::ProcessWebRenderParentCommands(nsTArray<mozilla::layers::WebRenderParentCommand> const&, mozilla::wr::TransactionBuilder&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1517|0x21b
6|5|xul.dll|mozilla::layers::WebRenderBridgeParent::ProcessDisplayListData(mozilla::layers::DisplayListData&, mozilla::wr::Epoch, mozilla::TimeStamp const&, bool, bool)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1169|0xd9
6|6|xul.dll|mozilla::layers::WebRenderBridgeParent::RecvSetDisplayList(mozilla::layers::DisplayListData&&, nsTArray<mozilla::layers::OpDestroy>&&, unsigned long long const&, mozilla::layers::BaseTransactionId<mozilla::layers::TransactionIdType> const&, bool const&, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, nsTSubstring<char> const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::CompositionPayload>&&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1233|0x23a
6|7|xul.dll|mozilla::layers::PWebRenderBridgeParent::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:777c843b67635508ddfb6f2f0da1fcd0804deebbce07f33d176dfc439d033df5d067223ae92833d61dbe5a69d74b75e53f8a48e300fe4cf7f528fed615b288a2/ipc/ipdl/PWebRenderBridgeParent.cpp:|489|0x1995
6|8|xul.dll|mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:8a5c9cc165d7d668ca5410663e4f25e5dc6ad7d155ae9883734d6883bbd6597f509d7c66e9fdd7365d896d91f348d7e3de10400d7c699126ef48a3442327480e/ipc/ipdl/PCompositorManagerParent.cpp:|194|0x28c
6|9|xul.dll|mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1811|0x128
6|10|xul.dll|mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message,mozilla::DefaultDelete<IPC::Message> >)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1736|0x1f3
6|11|xul.dll|mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::ipc::MessageChannel::MessageTask&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1536|0x155
6|12|xul.dll|mozilla::ipc::MessageChannel::MessageTask::Run()|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1634|0xba
6|13|xul.dll|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1193|0x9ae
6|14|xul.dll|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:a9e342522ebdb9602671208a06e988a285d3af18|479|0x41
6|15|xul.dll|mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:a9e342522ebdb9602671208a06e988a285d3af18|300|0xe0
6|16|xul.dll|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|370|0x82
6|17|xul.dll|MessageLoop::RunHandler()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|363|0x72
6|18|xul.dll|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|345|0x55
6|19|xul.dll|nsThread::ThreadFunc(void*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:a9e342522ebdb9602671208a06e988a285d3af18|391|0x18e
6|20|nss3.dll|_PR_NativeRunThread(void*)|hg:hg.mozilla.org/mozilla-central:nsprpub/pr/src/threads/combined/pruthr.c:a9e342522ebdb9602671208a06e988a285d3af18|399|0xe3
6|21|nss3.dll|pr_root(void*)|hg:hg.mozilla.org/mozilla-central:nsprpub/pr/src/md/windows/w95thred.c:a9e342522ebdb9602671208a06e988a285d3af18|139|0x15
6|22|ucrtbase.dll||||
6|23|kernel32.dll||||
6|24|mozglue.dll|patched_BaseThreadInitThunk(int, void*, void*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/dllservices/mozglue/WindowsDllBlocklist.cpp:a9e342522ebdb9602671208a06e988a285d3af18|617|0x72
6|25|ntdll.dll||||
6|26|ntdll.dll||||

Comment 1

[Glenn Watson [:gw]]

Sotaro, is this a part of the code that you would know about?

Comment 2

[Jason Kratzer [:jkratzer]]

Bugmon is currently only supported on Linux.

Comment 3

[Sotaro Ikeda [:sotaro]]

:tsmith, does the problem still happen with latest nightly?

I do not have an environment of "--cpu x86", but tried with "python -m fuzzfetch -d --fuzzing -n firefox" on Linux, the problem did not happen. And testcase.html did not hit assert failure with local build.

Comment 4

[Tyson Smith [:tsmith]]

(In reply to Sotaro Ikeda [:sotaro] from comment #3)

:tsmith, does the problem still happen with latest nightly?

Yes this is reproducible with m-c 20230628-3b7b3970a884.

I do not have an environment of "--cpu x86"

A 32-bit x86 build just works on Windows x64_86. So all you need is a machine running Windows . I am not able to reproduce this on Linux.

Comment 5

[Sotaro Ikeda [:sotaro]]

I looked into source code, ClientWebGLContext, WebGLParent and WebGLContext handle the default frame buffer size as expected. Then it seemed that an unexpected error caused the problem.

• when size of WebGL canvas is resized, ClientWebGLContext::SetDimensions() is called, it set ClientWebGLContext::mResetLayer to true. And WebGLParent is requested to resize by sync for async way.
• When ClientWebGLContext::UpdateWebRenderCanvasData() is called, mResetLayer = true requests to call ClientWebGLContext::DrawingBufferSize(). child->SendDrawingBufferSize() is sync IPC. It end up to call WebGLContext::DrawingBufferSize()
• WebGLContext::DrawingBufferSize() calls EnsureDefaultFB(). WebGLContext::EnsureDefaultFB() allocates default frame buffer when it is not allocated yet.

Therefore ClientWebGLContext::UpdateWebRenderCanvasData() get actual valid default frame buffer size from WebGLContext.

Comment 6

[Sotaro Ikeda [:sotaro]]

I succeeded to reproduce the assertion failure with Attachment 9339917 [details] and with local build(64bit build) only once. When it happened, Pbuffer surface allocation was failed at CreatePBufferSurface().

Comment 7

[Sotaro Ikeda [:sotaro]]

Actually Comment 6 is a problem of ANGLE.

Comment 8

[Sotaro Ikeda [:sotaro]]

:tsmith, can you get logout to console with With pref security.sandbox.gpu.level = 0? It seemed that several error might happen before the assertion failure.

Comment 9

[Tyson Smith [:tsmith]]

Attachment: stderr.txt

Comment 10

[Sotaro Ikeda [:sotaro]]

Thank you for the log! From the following, Texture2D allocation was failed in ANGLE and device reset happened. Then WebGL context was lost.

allocate:404. Internal D3D11 error: HRESULT: 0x887A0005 (removal reason: HRESULT: 0x887A0020): Error allocating Texture2D
WebGL(07388900)::LoseContext(0)

bug 1841214 seems to address the problem.

Comment 11

[Sotaro Ikeda [:sotaro]]

:tsmith, can you check if the problem is addressed?

Comment 12

[Tyson Smith [:tsmith]]

(In reply to Sotaro Ikeda [:sotaro] from comment #11)

:tsmith, can you check if the problem is addressed?

The attached test case now triggers bug 1845554.

Comment 13

[Sotaro Ikeda [:sotaro]]

With Bug 1841214 fix, a problem that is related to RemoteTexture is addressed.
Since bug 1845554 exists, this bug could be closed.