2016504 - Assertion failure: isSome(), at /builds/worker/workspace/obj-build/dist/include/mozilla/Maybe.h:1026

Status: VERIFIED FIXED

(Core :: Audio/Video: Web Codecs, defect)

Description

[Tyson Smith [:tsmith] (PTO -> Sept/8)]

Attachment: testcase.html

Found while fuzzing m-c 20260211-7df40d9149a3 (--enable-address-sanitizer --enable-fuzzing)

To reproduce via Grizzly Replay:

$ pip install fuzzfetch grizzly-framework --upgrade
$ python -m fuzzfetch -a --fuzzing -n firefox
$ python -m grizzly.replay.bugzilla ./firefox/firefox <bugid>

Assertion failure: isSome(), at /builds/worker/workspace/obj-build/dist/include/mozilla/Maybe.h:1026

==737425==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x7638dd47124b bp 0x7638196dce70 sp 0x7638196dce10 T245)
==737425==The signal is caused by a WRITE memory access.
==737425==Hint: address points to the zero page.
    #0 0x7638dd47124b in MOZ_CrashSequence /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:237:3
    #1 0x7638dd47124b in ref /builds/worker/workspace/obj-build/dist/include/mozilla/Maybe.h:1026:3
    #2 0x7638dd47124b in operator() /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:300:46
    #3 0x7638dd47124b in matchN<const mozilla::Variant<mozilla::StrongTypedef<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::DimensionsChangeType>, mozilla::StrongTypedef<mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> >, mozilla::DisplayDimensionsChangeType>, mozilla::StrongTypedef<mozilla::BitrateMode, mozilla::BitrateModeChangeType>, mozilla::StrongTypedef<mozilla::Maybe<unsigned int>, mozilla::BitrateChangeType>, mozilla::StrongTypedef<mozilla::Maybe<double>, mozilla::FramerateChangeType>, mozilla::StrongTypedef<mozilla::Usage, mozilla::UsageChangeType>, mozilla::StrongTypedef<mozilla::Maybe<nsTString<char16_t> >, mozilla::ContentHintTypeType>, mozilla::StrongTypedef<unsigned int, mozilla::SampleRateChangeType>, mozilla::StrongTypedef<unsigned int, mozilla::NumberOfChannelsChangeType> > &, (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:305:16
    #4 0x7638dd47124b in matchN<const mozilla::Variant<mozilla::StrongTypedef<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::DimensionsChangeType>, mozilla::StrongTypedef<mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> >, mozilla::DisplayDimensionsChangeType>, mozilla::StrongTypedef<mozilla::BitrateMode, mozilla::BitrateModeChangeType>, mozilla::StrongTypedef<mozilla::Maybe<unsigned int>, mozilla::BitrateChangeType>, mozilla::StrongTypedef<mozilla::Maybe<double>, mozilla::FramerateChangeType>, mozilla::StrongTypedef<mozilla::Usage, mozilla::UsageChangeType>, mozilla::StrongTypedef<mozilla::Maybe<nsTString<char16_t> >, mozilla::ContentHintTypeType>, mozilla::StrongTypedef<unsigned int, mozilla::SampleRateChangeType>, mozilla::StrongTypedef<unsigned int, mozilla::NumberOfChannelsChangeType> > &, (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:295:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:314:14
    #5 0x7638dd47124b in matchN<const mozilla::Variant<mozilla::StrongTypedef<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::DimensionsChangeType>, mozilla::StrongTypedef<mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> >, mozilla::DisplayDimensionsChangeType>, mozilla::StrongTypedef<mozilla::BitrateMode, mozilla::BitrateModeChangeType>, mozilla::StrongTypedef<mozilla::Maybe<unsigned int>, mozilla::BitrateChangeType>, mozilla::StrongTypedef<mozilla::Maybe<double>, mozilla::FramerateChangeType>, mozilla::StrongTypedef<mozilla::Usage, mozilla::UsageChangeType>, mozilla::StrongTypedef<mozilla::Maybe<nsTString<char16_t> >, mozilla::ContentHintTypeType>, mozilla::StrongTypedef<unsigned int, mozilla::SampleRateChangeType>, mozilla::StrongTypedef<unsigned int, mozilla::NumberOfChannelsChangeType> > &, (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:294:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:295:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:314:14
    #6 0x7638dd47124b in matchN<const mozilla::Variant<mozilla::StrongTypedef<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::DimensionsChangeType>, mozilla::StrongTypedef<mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> >, mozilla::DisplayDimensionsChangeType>, mozilla::StrongTypedef<mozilla::BitrateMode, mozilla::BitrateModeChangeType>, mozilla::StrongTypedef<mozilla::Maybe<unsigned int>, mozilla::BitrateChangeType>, mozilla::StrongTypedef<mozilla::Maybe<double>, mozilla::FramerateChangeType>, mozilla::StrongTypedef<mozilla::Usage, mozilla::UsageChangeType>, mozilla::StrongTypedef<mozilla::Maybe<nsTString<char16_t> >, mozilla::ContentHintTypeType>, mozilla::StrongTypedef<unsigned int, mozilla::SampleRateChangeType>, mozilla::StrongTypedef<unsigned int, mozilla::NumberOfChannelsChangeType> > &, (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:293:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:294:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:295:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:314:14
    #7 0x7638dd47124b in matchN<const mozilla::Variant<mozilla::StrongTypedef<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::DimensionsChangeType>, mozilla::StrongTypedef<mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> >, mozilla::DisplayDimensionsChangeType>, mozilla::StrongTypedef<mozilla::BitrateMode, mozilla::BitrateModeChangeType>, mozilla::StrongTypedef<mozilla::Maybe<unsigned int>, mozilla::BitrateChangeType>, mozilla::StrongTypedef<mozilla::Maybe<double>, mozilla::FramerateChangeType>, mozilla::StrongTypedef<mozilla::Usage, mozilla::UsageChangeType>, mozilla::StrongTypedef<mozilla::Maybe<nsTString<char16_t> >, mozilla::ContentHintTypeType>, mozilla::StrongTypedef<unsigned int, mozilla::SampleRateChangeType>, mozilla::StrongTypedef<unsigned int, mozilla::NumberOfChannelsChangeType> > &, (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:293:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:294:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:295:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:905:12
    #8 0x7638dd47124b in match<(lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:293:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:294:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:295:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:296:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:307:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:308:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:309:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:310:9), (lambda at /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:311:9)> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:848:12
    #9 0x7638dd47124b in mozilla::FFmpegDataEncoder<60>::ProcessReconfigure(RefPtr<mozilla::EncoderConfigurationChangeList const> const&) /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataEncoder.cpp:291:22
    #10 0x7638dd4a7733 in operator()<StoreCopyPassByRRef<RefPtr<const mozilla::EncoderConfigurationChangeList> > &> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1083:18
    #11 0x7638dd4a7733 in __invoke_impl<RefPtr<mozilla::MozPromise<bool, mozilla::MediaResult, true> >, (lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1082:9), StoreCopyPassByRRef<RefPtr<const mozilla::EncoderConfigurationChangeList> > &> /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14
    #12 0x7638dd4a7733 in __invoke<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1082:9), StoreCopyPassByRRef<RefPtr<const mozilla::EncoderConfigurationChangeList> > &> /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14
    #13 0x7638dd4a7733 in __apply_impl<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1082:9), std::tuple<StoreCopyPassByRRef<RefPtr<const mozilla::EncoderConfigurationChangeList> > > &, 0UL> /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/tuple:1740:14
    #14 0x7638dd4a7733 in apply<(lambda at /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1082:9), std::tuple<StoreCopyPassByRRef<RefPtr<const mozilla::EncoderConfigurationChangeList> > > &> /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/tuple:1751:14
    #15 0x7638dd4a7733 in apply<mozilla::FFmpegDataEncoder<60>, RefPtr<mozilla::MozPromise<bool, mozilla::MediaResult, true> > (mozilla::FFmpegDataEncoder<60>::*)(const RefPtr<const mozilla::EncoderConfigurationChangeList> &)> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1081:12
    #16 0x7638dd4a7733 in Invoke /builds/worker/workspace/obj-build/dist/include/mozilla/MozPromise.h:1711:47
    #17 0x7638dd4a7733 in mozilla::detail::ProxyRunnable<mozilla::MozPromise<bool, mozilla::MediaResult, true>, RefPtr<mozilla::MozPromise<bool, mozilla::MediaResult, true>> (mozilla::FFmpegDataEncoder<60>::*)(RefPtr<mozilla::EncoderConfigurationChangeList const> const&), mozilla::FFmpegDataEncoder<60>, StoreCopyPassByRRef<RefPtr<mozilla::EncoderConfigurationChangeList const>>>::Run() /builds/worker/workspace/obj-build/dist/include/mozilla/MozPromise.h:1731:42
    #18 0x7638d4e06a05 in mozilla::TaskQueue::Runner::Run() /builds/worker/checkouts/gecko/xpcom/threads/TaskQueue.cpp:311:20
    #19 0x7638d4e4f007 in nsThreadPool::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadPool.cpp:446:14
    #20 0x7638d4e42230 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1162:16
    #21 0x7638d4e4b089 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:461:10
    #22 0x7638d657faa1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:299:20
    #23 0x7638d645d1f4 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:373:10
    #24 0x7638d645d1f4 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:366:3
    #25 0x7638d645d1f4 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:348:3
    #26 0x7638d4e3b130 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:373:10
    #27 0x7a38faee1aff in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:191:3
    #28 0x616619575676 in asan_thread_start(void*) _asan_rtl_:28
    #29 0x7a38fac9caa3 in start_thread ./nptl/pthread_create.c:447:8

Comment 1

[Bugmon [:jkratzer for issues]]

Verified bug as reproducible on mozilla-central 20260212213836-06d48ece4edf.
Unable to bisect testcase (Testcase reproduces on start build!):

Start: 8e442c6e776594983158fd04b5331f1378f5f0dd (20250214215152)
End: 7df40d9149a3d5135ccdcf8602299a76bf7f0081 (20260211162125)
BuildFlags: BuildFlags(asan=True, tsan=False, debug=False, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False, nyx=False, searchfox=False, afl=False)

Comment 2

[Paul Adenot (:padenot)]

Attachment: Bug 2016504 - Fix VideoEncoderConfigInternal::Diff() producing stale config values r?chunmin

Comment 3

[Paul Adenot (:padenot)]

Attachment: Bug 2016504 - Fix VideoEncoderConfigInternal::Diff() producing stale config values r?chunmin

Diff(aOther) is called as newConfig->Diff(oldConfig), so this is the new
config and aOther is the old one. All change entries were incorrectly
populated from aOther instead of this, causing the encoder to receive
stale values on reconfigure.

In the crashing case, a first configure() without bitrate followed by a
configure() with bitrate produces a BitrateChange{Nothing()} instead of
BitrateChange{Some(bitrate)}, which then hits a MOZ_ASSERT in
FFmpegDataEncoder when unwrapping the empty Maybe.

Replace the assert with an isNothing() guard that returns false, causing the
encoder to drain and reinitialize cleanly.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/9eb29ce4944c
https://hg.mozilla.org/integration/autoland/rev/b9f9fbb8e7bf
Fix VideoEncoderConfigInternal::Diff() producing stale config values r=chunmin

Comment 5

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/b9f9fbb8e7bf

Comment 6

[Bugmon [:jkratzer for issues]]

Verified bug as fixed on rev mozilla-central 20260221090124-501ded2d35df.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.