1734008 - Assertion failure: !mImpl->IsConnected(), at /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293

Reporter

Description

•

2 years ago

Testcase found while fuzzing mozilla-central rev 9790289bfed7 (built with: --enable-address-sanitizer --enable-fuzzing).

Testcase can be reproduced using the following commands:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch --build 9790289bfed7 --asan --fuzzing -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.zip

Assertion failure: !mImpl->IsConnected(), at /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293

    =================================================================
    ==248851==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x7fad074267e3 bp 0x7faceca4cce0 sp 0x7faceca4ccd0 T35)
    ==248851==The signal is caused by a WRITE memory access.
    ==248851==Hint: address points to the zero page.
        #0 0x7fad074267e3 in mozilla::Mirror<nsMainThreadPtrHandle<nsIPrincipal> >::~Mirror() /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293:5
        #1 0x7fad07a0613a in mozilla::DecodedStream::~DecodedStream() /dom/media/mediasink/DecodedStream.cpp:471:1
        #2 0x7fad07a0649d in mozilla::DecodedStream::~DecodedStream() /dom/media/mediasink/DecodedStream.cpp:469:33
        #3 0x7fad07a123b9 in Release /dom/media/mediasink/MediaSink.h:40:3
        #4 0x7fad07a123b9 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:50:40
        #5 0x7fad07a123b9 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:381:36
        #6 0x7fad07a123b9 in ~RefPtr /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:81:7
        #7 0x7fad07a123b9 in mozilla::VideoSink::~VideoSink() /dom/media/mediasink/VideoSink.cpp:92:1
        #8 0x7fad07a1251d in mozilla::VideoSink::~VideoSink() /dom/media/mediasink/VideoSink.cpp:88:25
        #9 0x7fad0742b777 in Release /dom/media/mediasink/MediaSink.h:40:3
        #10 0x7fad0742b777 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:50:40
        #11 0x7fad0742b777 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:381:36
        #12 0x7fad0742b777 in assign_assuming_AddRef /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:69:7
        #13 0x7fad0742b777 in RefPtr<mozilla::MediaSink>& RefPtr<mozilla::MediaSink>::operator=<mozilla::MediaSink>(already_AddRefed<mozilla::MediaSink>&&) /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:206:5
        #14 0x7fad0743748e in mozilla::MediaDecoderStateMachine::ResumeMediaSink() /dom/media/MediaDecoderStateMachine.cpp:3901:14
        #15 0x7fad0760108d in applyImpl<mozilla::MediaDecoderStateMachine, void (mozilla::MediaDecoderStateMachine::*)()> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1147:12
        #16 0x7fad0760108d in apply<mozilla::MediaDecoderStateMachine, void (mozilla::MediaDecoderStateMachine::*)()> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1153:12
        #17 0x7fad0760108d in mozilla::detail::RunnableMethodImpl<mozilla::MediaDecoderStateMachine*, void (mozilla::MediaDecoderStateMachine::*)(), true, (mozilla::RunnableKind)0>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1200:13
        #18 0x7fad013926a0 in mozilla::AutoTaskDispatcher::TaskGroupRunnable::Run() /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:227:35
        #19 0x7fad013a1cbd in mozilla::TaskQueue::Runner::Run() /xpcom/threads/TaskQueue.cpp:208:20
        #20 0x7fad013c9b3f in nsThreadPool::Run() /xpcom/threads/nsThreadPool.cpp:303:14
        #21 0x7fad013bbbfb in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1142:16
        #22 0x7fad013c63dc in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:467:10
        #23 0x7fad0286a0d5 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:300:20
        #24 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #25 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #26 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #27 0x7fad013b5488 in nsThread::ThreadFunc(void*) /xpcom/threads/nsThread.cpp:390:10
        #28 0x7fad1f431cce in _pt_root /nsprpub/pr/src/pthreads/ptthread.c:201:5
        #29 0x7fad23548608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
        #30 0x7fad23110292 in clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293:5 in mozilla::Mirror<nsMainThreadPtrHandle<nsIPrincipal> >::~Mirror()
    Thread T35 (MediaDe~hine #1) created by T0 (Web Content) here:
        #0 0x5575e7eb10cc in pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cpp:205:3
        #1 0x7fad1f421d34 in _PR_CreateThread /nsprpub/pr/src/pthreads/ptthread.c:458:14
        #2 0x7fad1f41320e in PR_CreateThread /nsprpub/pr/src/pthreads/ptthread.c:533:12
        #3 0x7fad013b7e9a in nsThread::Init(nsTSubstring<char> const&) /xpcom/threads/nsThread.cpp:602:18
        #4 0x7fad013c3dff in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) /xpcom/threads/nsThreadManager.cpp:576:12
        #5 0x7fad013cf0d1 in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadUtils.cpp:163:57
        #6 0x7fad013c8740 in NS_NewNamedThread /xpcom/threads/nsThreadUtils.cpp:155:10
        #7 0x7fad013c8740 in nsThreadPool::PutEvent(already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadPool.cpp:118:17
        #8 0x7fad013ca9bf in nsThreadPool::Dispatch(already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadPool.cpp:354:5
        #9 0x7fad0139f93b in mozilla::TaskQueue::DispatchLocked(nsCOMPtr<nsIRunnable>&, unsigned int, mozilla::AbstractThread::DispatchReason) /xpcom/threads/TaskQueue.cpp:68:26
        #10 0x7fad013d3043 in mozilla::TaskQueue::Dispatch(already_AddRefed<nsIRunnable>, mozilla::AbstractThread::DispatchReason) /builds/worker/workspace/obj-build/dist/include/mozilla/TaskQueue.h:87:14
        #11 0x7fad0139215a in mozilla::AutoTaskDispatcher::DispatchTaskGroup(mozilla::UniquePtr<mozilla::AutoTaskDispatcher::PerThreadTaskGroup, mozilla::DefaultDelete<mozilla::AutoTaskDispatcher::PerThreadTaskGroup> >) /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:275:20
        #12 0x7fad013915a6 in mozilla::AutoTaskDispatcher::~AutoTaskDispatcher() /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:121:7
        #13 0x7fad013936d9 in reset /builds/worker/workspace/obj-build/dist/include/mozilla/Maybe.h:639:19
        #14 0x7fad013936d9 in mozilla::XPCOMThreadWrapper::MaybeFireTailDispatcher() /xpcom/threads/AbstractThread.cpp:195:23
        #15 0x7fad0138f46c in AfterProcessNextEvent /xpcom/threads/AbstractThread.cpp:133:5
        #16 0x7fad0138f46c in non-virtual thunk to mozilla::XPCOMThreadWrapper::AfterProcessNextEvent(nsIThreadInternal*, bool) /xpcom/threads/AbstractThread.cpp
        #17 0x7fad013bb726 in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1172:3
        #18 0x7fad013c63dc in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:467:10
        #19 0x7fad02868a24 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:107:5
        #20 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #21 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #22 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #23 0x7fad0911f417 in nsBaseAppShell::Run() /widget/nsBaseAppShell.cpp:137:27
        #24 0x7fad0d95419f in XRE_RunAppShell() /toolkit/xre/nsEmbedFunctions.cpp:917:20
        #25 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #26 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #27 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #28 0x7fad0d95336e in XRE_InitChildProcess(int, char**, XREChildData const*) /toolkit/xre/nsEmbedFunctions.cpp:749:34
        #29 0x5575e7efa60d in content_process_main(mozilla::Bootstrap*, int, char**) /browser/app/../../ipc/contentproc/plugin-container.cpp:57:28
        #30 0x5575e7efaa3d in main /browser/app/nsBrowserApp.cpp:327:18
        #31 0x7fad230150b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
    
    ==248851==ABORTING

Jason Kratzer [:jkratzer]

Reporter

Comment 1

•

2 years ago

Attached file Testcase — Details

Bugmon [:jkratzer for issues]

Comment 2

•

2 years ago

Bugmon Analysis
Verified bug as reproducible on mozilla-central 20211004215121-cc37b1400a58.
The bug appears to have been introduced in the following build range:

Start: 1130661c79c222fb1acd29b7ec5dc5202cdd0d2d (20201211213049)
End: a3add3f43cbcbfdf053d901689882975479842eb (20201211175457)
Pushlog: https://hg.mozilla.org/mozilla-unified/pushloghtml?fromchange=1130661c79c222fb1acd29b7ec5dc5202cdd0d2d&tochange=a3add3f43cbcbfdf053d901689882975479842eb

Whiteboard: [bugmon:confirm] → [bugmon:bisected,confirmed]

John Lin [:jhlin][:jolin]

Comment 3

•

2 years ago

The Mirror value in DecodedStream[1] is supposed to be disconnected at Stop()[2] or Shutdown()[3]. From the stack it looks like the object is VideoSink::mAudioSink, which should be disconnected in VideoSink::Stop() or VideoSink::Shutdown().

Also, I couldn't see any related changes in the mentioned commits that possibly causes it.

Jason, when try to run the replay command locally on my Mac I got the following error. Do you know how to fix it? Thanks a lot!

[2021-10-11 01:29:37] Starting Grizzly Replay
[2021-10-11 01:29:37] Ignoring: log-limit, timeout
[2021-10-11 01:29:37] Using time limit: 15s, timeout: 30s
[2021-10-11 01:29:37] Repeat: 1, Minimum crashes: 1, Relaunch 1
[2021-10-11 01:29:42] Running test (1/1)...
[2021-10-11 01:29:46] Test case was not served
[2021-10-11 01:29:46] Delayed startup failure detected
[2021-10-11 01:29:46] Result: Different signature: Hit MOZ_CRASH(Attempting to connect to non-local address!) at /builds/worker/checkouts/gecko/netwerk/base/nsSocketTransport2.cpp:1236 (8b52d4d6:2aeaeb0f)
[2021-10-11 01:29:46] Failed to reproduce results
[2021-10-11 01:29:46] Shutting down...
[2021-10-11 01:29:46] Done.

[1] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.h#113
[2] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.cpp#623
[3] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.cpp#640

Flags: needinfo?(jkratzer)

Jason Kratzer [:jkratzer]

Reporter

Comment 4

•

2 years ago

John, that error typically indicates that you are using a prefs file that does not have the network.proxy.autoconfig_url set. The attached testcase however, includes a pref file with this set so I'm not entirely sure why you're seeing that error. Just out of curiosity, are you seeing this with a local build or one from Fuzzfetch?

Flags: needinfo?(jkratzer)

Alastor Wu [:alwu]

Assignee

Comment 5

•

1 year ago

ResumeMediaSink is triggered asynchrously, and it assumes that we have already stopped the media sink in the SuspendMedia call

However, resumming the sink is an asychrous task, it's possible that we start the sink before running that task. If so, we would destroy the started sink without properly calling its stop method.

Assignee: nobody → alwu

Severity: -- → S3

Component: Audio/Video → Audio/Video: Playback

Priority: -- → P3

Alastor Wu [:alwu]

Assignee

Comment 6

•

1 year ago

BTW I wonder if it's possible to let grizzly.replay to print the debug log while it's running the test case?

Flags: needinfo?(jkratzer)

Release mgmt bot [:suhaib / :marco/ :calixte]

Comment 7

•

1 year ago

:alwu, since this bug contains a bisection range, could you fill (if possible) the regressed_by field?
For more information, please visit auto_nag documentation.

Flags: needinfo?(alwu)

Jason Kratzer [:jkratzer]

Reporter

Comment 8

•

1 year ago

:alwu, currently there is no way to enable grizzly.replay to report stdout/stderr while the testcase is running. The only other option available is --dump which will print the contents of stderr/stdout to the console once the browser is closed.

Flags: needinfo?(jkratzer)

Alastor Wu [:alwu]

Assignee

Comment 9

•

1 year ago

I found that --dump isn't defined in my grizzly.replay, but I can use -l to define the log path and works well.

Flags: needinfo?(alwu)

Alastor Wu [:alwu]

Assignee

Comment 10

•

1 year ago

Attached file Bug 1734008 - part1 : avoid resuming the media sink which is already running. — Details

This situation happens when we try to resume the media sink which is already running.

The resume calls are called from HTMLMediaElement [1][2] and media element doesn't know the internal status of the decoder. When [2] is called after [1], then it triggers resuming twice in MediaDecoderStateMachine.

Therefore, we should check mIsMediaSinkSuspended first to see if we really need to perform the suspend/resume on the media sink.

[1] https://searchfox.org/mozilla-central/rev/483cfde5a54f6c1cd94c6295564993aeb4f10980/dom/html/HTMLMediaElement.cpp#6391
[2] https://searchfox.org/mozilla-central/rev/483cfde5a54f6c1cd94c6295564993aeb4f10980/dom/html/HTMLMediaElement.cpp#4344

Alastor Wu [:alwu]

Assignee

Comment 11

•

1 year ago

Attached file Bug 1734008 - part2 : add a crashtest. — Details

Jason Kratzer [:jkratzer]

Reporter

Comment 12

•

1 year ago

(In reply to Alastor Wu [:alwu] from comment #9)

I found that --dump isn't defined in my grizzly.replay, but I can use -l to define the log path and works well.

Ah, my apologies. That argument actually belongs to ffpuppet which is what grizzly uses to drive firefox.

Pulsebot

Comment 13

•

1 year ago

Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/33082d9250aa
part1 : avoid resuming the media sink which is already running. r=bryce
https://hg.mozilla.org/integration/autoland/rev/ac232c892ea5
part2 : add a crashtest. r=bryce

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Comment 14

•

1 year ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/33082d9250aa
https://hg.mozilla.org/mozilla-central/rev/ac232c892ea5

Status: NEW → RESOLVED

Closed: 1 year ago

status-firefox96: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → 96 Branch

Bugmon [:jkratzer for issues]

Comment 15

•

1 year ago

Bugmon Analysis
Verified bug as fixed on rev mozilla-central 20211101215926-08eb1047d841.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.

Status: RESOLVED → VERIFIED

status-firefox96: fixed → verified

Keywords: bugmon

Testcase 2 years ago Jason Kratzer [:jkratzer] 11.75 KB, application/octet-stream		Details
Bug 1734008 - part1 : avoid resuming the media sink which is already running. 1 year ago Alastor Wu [:alwu] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1734008 - part2 : add a crashtest. 1 year ago Alastor Wu [:alwu] 48 bytes, text/x-phabricator-request		Details \| Review

Bugzilla

Quick Search

Assertion failure: !mImpl->IsConnected(), at /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293

Categories

(Core :: Audio/Video: Playback, defect, P3)

Tracking

()

People

(Reporter: jkratzer, Assigned: alwu)

References

(Blocks 1 open bug)

Details

(Keywords: testcase, Whiteboard: [bugmon:bisected,confirmed])

Crash Data

Security

(public)

User Story

Attachments

(3 files)

Description

Comment 1

Comment 2

Comment 3

Comment 4

Comment 5

Comment 6

Comment 7

Comment 8

Comment 9

Comment 10

Comment 11

Comment 12

Comment 13

Comment 14

Comment 15