2014892 - Assertion failure: self->mState != CodecState::Closed, at /dom/media/webcodecs/DecoderTemplate.cpp:660

Status: VERIFIED FIXED

(Core :: Audio/Video, defect, P2)

Description

[Jesse Schwartzentruber (:truber)]

Reproduced on mozilla-central rev ed1c0b4f0bbe (built with: --enable-debug --enable-fuzzing).

Testcase can be reproduced using the following commands:

# From within a firefox checkout
$ pipx install fuzzfetch
$ fuzzfetch --build ed1c0b4f0bbe --debug --fuzzing  -n tc-firefox
$ cp testcase.html testing/web-platform/tests/dom/foo-crash.html
$ ./mach web-platform-tests testing/web-platform/tests/dom/foo-crash.html --binary ./tc-firefox --headless --repeat 100

Assertion failure: self->mState != CodecState::Closed, at /dom/media/webcodecs/DecoderTemplate.cpp:660

    ==2098155==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fc62cf864ae bp 0x7ffd2cfb8100 sp 0x7ffd2cfb8100 T2098155)
    ==2098155==The signal is caused by a WRITE memory access.
    ==2098155==Hint: address points to the zero page.
        #0 0x7fc62cf864ae in MOZ_CrashSequence /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:237:3
        #1 0x7fc62cf864ae in operator() /dom/media/webcodecs/DecoderTemplate.cpp:660:24
        #2 0x7fc62cf864ae in mozilla::detail::RunnableFunction<mozilla::dom::DecoderTemplate<mozilla::dom::VideoDecoderTraits>::ProcessDecodeMessage(std::unique_ptr<mozilla::dom::DecoderTemplate<mozilla::dom::VideoDecoderTraits>::ControlMessage, std::default_delete<mozilla::dom::DecoderTemplate<mozilla::dom::VideoDecoderTraits>::ControlMessage>>&)::'lambda'(mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData>>, mozilla::MediaResult, true>::ResolveOrRejectValue&&)::operator()(mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData>>, mozilla::MediaResult, true>::ResolveOrRejectValue&&)::'lambda'()>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:549:5
        #3 0x7fc628544497 in mozilla::RunnableTask::Run() /xpcom/threads/TaskController.cpp:705:16
        #4 0x7fc62853eb64 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) /xpcom/threads/TaskController.cpp:1325:20
        #5 0x7fc62853d7e7 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) /xpcom/threads/TaskController.cpp:1148:15
        #6 0x7fc62853dc65 in mozilla::TaskController::ProcessPendingMTTask(bool) /xpcom/threads/TaskController.cpp:641:36
        #7 0x7fc62854d776 in operator() /xpcom/threads/TaskController.cpp:333:37
        #8 0x7fc62854d776 in mozilla::detail::RunnableFunction<mozilla::TaskController::TaskController()::$_0>::Run() /xpcom/threads/nsThreadUtils.h:549:5
        #9 0x7fc62855fcf3 in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1168:16
        #10 0x7fc62856594f in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:461:10
        #11 0x7fc629155287 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:85:21
        #12 0x7fc6290ae691 in RunHandler /ipc/chromium/src/base/message_loop.cc:366:3
        #13 0x7fc6290ae691 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:348:3
        #14 0x7fc62e265a68 in nsBaseAppShell::Run() /widget/nsBaseAppShell.cpp:152:27
        #15 0x7fc62e331484 in nsAppShell::Run() /widget/gtk/nsAppShell.cpp:555:33
        #16 0x7fc62f3794eb in XRE_RunAppShell() /toolkit/xre/nsEmbedFunctions.cpp:652:20
        #17 0x7fc629156134 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:235:9
        #18 0x7fc6290ae691 in RunHandler /ipc/chromium/src/base/message_loop.cc:366:3
        #19 0x7fc6290ae691 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:348:3
        #20 0x7fc62f378c4f in XRE_InitChildProcess(int, char**, XREChildData const*) /toolkit/xre/nsEmbedFunctions.cpp:590:34
        #21 0x558f9f577f8c in main /browser/app/nsBrowserApp.cpp:465:22
        #22 0x7fc639c33f74 in __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
        #23 0x7fc639c34026 in __libc_start_main ./csu/../csu/libc-start.c:360:3
        #24 0x558f9f54bde8 in _start ??:0:0
    
    ==2098155==Register values:
    rax = 0x0000000000000000  rbx = 0x00007fc57c22ec40  rcx = 0x0000000000000294  rdx = 0x0000000000000000
    rdi = 0x00007fc639df3780  rsi = 0x0000000000000000  rbp = 0x00007ffd2cfb8100  rsp = 0x00007ffd2cfb8100
     r8 = 0x0000000000000000   r9 = 0x0000000000000000  r10 = 0x0000000000000000  r11 = 0x00007fc639bfa5c0
    r12 = 0x000000000000001c  r13 = 0x00007fc6372da561  r14 = 0x00007ffd2cfb8678  r15 = 0x00007fc57c22ec40
    UndefinedBehaviorSanitizer can not provide additional info.
    SUMMARY: UndefinedBehaviorSanitizer: SEGV (/home/truber/builds/m-c-20260205163736-fuzzing-debug/libxul.so+0x93864ae) (BuildId: 0b411186bb9521ab7e302998bc6191dc011c2d12)
    ==2098155==ABORTING

Comment 1

[Jesse Schwartzentruber (:truber)]

Attachment: Detailed Crash Information

Comment 2

[Jesse Schwartzentruber (:truber)]

Attachment: Testcase

Comment 3

[Bugmon [:jkratzer for issues]]

Verified bug as reproducible on mozilla-central 20260205213633-279b57ef4bf0.
Unable to bisect testcase (Testcase reproduces on start build!):

Start: ce0f77cae5a91e2cb4b1ec859117e08aa1182c52 (20250207164009)
End: ed1c0b4f0bbef7e458685d5f4743c73ae66cc3fd (20260205163736)
BuildFlags: BuildFlags(asan=False, tsan=False, debug=True, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False, nyx=False, searchfox=False, afl=False)

Comment 4

[Paul Adenot (:padenot)]

Attachment: (secure)

This fixes a race condition where async error handling tasks attempt to close
a codec that the user has already explicitly closed, causing an assertion
failure: MOZ_ASSERT(self->mState != CodecState::Closed).

The race occurs as follows:

1. An async decode/encode operation fails and queues a task to call CloseInternal()
2. Before that queued task runs, the user explicitly calls decoder.close()
3. The queued error task runs and hits the assertion

The assertions at call sites don't hold due to this race condition. The fix
removes these assertions and instead adds a check in CloseInternal() to return
early if already closed, making close operations idempotent.

Per the WebCodecs spec, the Close algorithm calls Reset, and Reset is specified
to throw InvalidStateError if the state is already "closed". Our ResetInternal()
implements this check (line 283-285). By checking in CloseInternal() first, we
avoid calling Reset on an already-closed codec, which respects the spec's
invariant while handling the race gracefully.

This approach is spec-compliant because:

• It respects the invariant that Reset should not operate on closed codecs
• Close operations are inherently idempotent (closing an already-closed codec
  is a no-op, as the codec is already in the terminal "closed" state)
• It prevents error callbacks from firing on codecs the user has already closed

Note: This race is extremely rare and timing-dependent. Attempts to reproduce
included: 10 normal runs, 20 runs with stress-ng (--cpu 4 --vm 2), and 20 runs
with MOZ_CHAOSMODE=0xfb. None triggered the assertion. The fuzzer found it,
confirming it exists, but the race window is exceptionally narrow. I still think
the fix is correct but I can't prove it.

Comment 5

[Andrew McCreight [:mccr8]]

Is this any kind of security problem? If not we could mark it sec-other. Thanks.

Comment 6

[Paul Adenot (:padenot)]

No, this is just invariant that we want to hold, but they didn't hold in valid cases. sec-other.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/5e71a87db055
https://hg.mozilla.org/integration/autoland/rev/28c3e24d6d4b
Prevent assertion failure when closing already-closed WebCodecs decoders/encoders. r=media-playback-reviewers,alwu

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg-edge.mozilla.org/mozilla-central/rev/28c3e24d6d4b

Comment 9

[Bugmon [:jkratzer for issues]]

Verified bug as fixed on rev mozilla-central 20260213044212-cabf73b570b5.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.