Closed Bug 1687597 Opened 5 years ago Closed 5 years ago

heap-use-after-free in [@ mozilla::dom::WorkerPrivate::ResetWorkerPrivateInWorkerThread]

Categories

(Core :: DOM: Workers, defect, P1)

defect

Tracking

()

RESOLVED FIXED
87 Branch
Tracking Status
firefox-esr78 86+ fixed
firefox84 --- wontfix
firefox85 --- wontfix
firefox86 + fixed
firefox87 --- fixed

People

(Reporter: tsmith, Assigned: sg)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: csectype-uaf, regression, sec-high, Whiteboard: [sec-survey][adv-main86+r][adv-esr78.8+r])

Crash Data

Attachments

(2 files)

A test case in not available due to size and reliability but a Pernosco session will be added shortly.

==27596==ERROR: AddressSanitizer: heap-use-after-free on address 0x61b000032c90 at pc 0x1a3823b6a371 bp 0x5e2f15d010f0 sp 0x5e2f15d010e8
READ of size 8 at 0x61b000032c90 thread T21 (DOM Worker)
    #0 0x1a3823b6a370 in RefPtr<mozilla::dom::SharedMutex::RefCountedMutex>::get() const src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:286:27
    #1 0x1a3823b6a331 in RefPtr<mozilla::dom::SharedMutex::RefCountedMutex>::operator*() const src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:359:13
    #2 0x1a3823b6a304 in mozilla::dom::SharedMutex::operator mozilla::Mutex&() src/objdir-ff-ubsan/dist/include/mozilla/dom/WorkerPrivate.h:99:30
    #3 0x1a3823bb5cf0 in mozilla::dom::WorkerPrivate::ResetWorkerPrivateInWorkerThread() src/dom/workers/WorkerPrivate.cpp:5000:22
    #4 0x1a3823b2bb98 in mozilla::dom::workerinternals::(anonymous namespace)::WorkerThreadPrimaryRunnable::Run()::SetThreadHelper::~SetThreadHelper() src/dom/workers/RuntimeService.cpp:2184:25
    #5 0x1a3823b2add4 in mozilla::dom::workerinternals::(anonymous namespace)::WorkerThreadPrimaryRunnable::Run() src/dom/workers/RuntimeService.cpp:2289:1
    #6 0x1a38156e45fb in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1200:14
    #7 0x1a38156eff19 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:548:10
    #8 0x1a3817d41b54 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:302:20
    #9 0x1a3817a2be3f in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
    #10 0x1a3817a2bd94 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:327:3
    #11 0x1a3817a2bd01 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3
    #12 0x1a38156dc159 in nsThread::ThreadFunc(void*) src/xpcom/threads/nsThread.cpp:441:10
    #13 0x492166439ff9 in _pt_root src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #14 0x6b4a3ff706da in start_thread /build/glibc-2ORdQG/glibc-2.27/nptl/pthread_create.c:463
    #15 0x718e52f65a3e in clone /build/glibc-2ORdQG/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x61b000032c90 is located 16 bytes inside of 1448-byte region [0x61b000032c80,0x61b000033228)
freed by thread T0 (Web Content) here:
    #0 0x557328ee9dbd in free /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:123:3
    #1 0x1a3823b770ba in operator delete(void*) src/objdir-ff-ubsan/dist/include/mozilla/cxxalloc.h:51:10
    #2 0x1a3823b770ba in mozilla::dom::WorkerPrivate::Release() src/objdir-ff-ubsan/dist/include/mozilla/dom/WorkerPrivate.h:122:3
    #3 0x1a3823b77024 in mozilla::RefPtrTraits<mozilla::dom::WorkerPrivate>::Release(mozilla::dom::WorkerPrivate*) src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:50:40
    #4 0x1a3823b77004 in RefPtr<mozilla::dom::WorkerPrivate>::ConstRemovingRefPtrTraits<mozilla::dom::WorkerPrivate>::Release(mozilla::dom::WorkerPrivate*) src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:381:36
    #5 0x1a3823b76fe0 in RefPtr<mozilla::dom::WorkerPrivate>::assign_assuming_AddRef(mozilla::dom::WorkerPrivate*) src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:69:7
    #6 0x1a3823b64116 in RefPtr<mozilla::dom::WorkerPrivate>::operator=(std::nullptr_t) src/objdir-ff-ubsan/dist/include/mozilla/RefPtr.h:168:5
    #7 0x1a3823becd12 in mozilla::dom::WorkerPrivate::ClearSelfAndParentEventTargetRef() src/objdir-ff-ubsan/dist/include/mozilla/dom/WorkerPrivate.h:145:14
    #8 0x1a3823bd5a4f in mozilla::dom::(anonymous namespace)::TopLevelWorkerFinishedRunnable::Run() src/dom/workers/WorkerPrivate.cpp:294:22
    #9 0x1a3815738df0 in mozilla::ThrottledEventQueue::Inner::ExecuteRunnable() src/xpcom/threads/ThrottledEventQueue.cpp:254:22
    #10 0x1a3815722285 in mozilla::ThrottledEventQueue::Inner::Executor::Run() src/xpcom/threads/ThrottledEventQueue.cpp:81:15
    #11 0x1a38156a0ee9 in mozilla::RunnableTask::Run() src/xpcom/threads/TaskController.cpp:459:16
    #12 0x1a38156853e2 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:739:26
    #13 0x1a3815682220 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:598:15
    #14 0x1a3815682629 in mozilla::TaskController::ProcessPendingMTTask(bool) src/xpcom/threads/TaskController.cpp:382:36
    #15 0x1a381568818d in mozilla::TaskController::InitializeInternal()::$_4::operator()() const src/xpcom/threads/TaskController.cpp:126:37
    #16 0x1a38156880fd in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_4>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:534:5
    #17 0x1a38156e45fb in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1200:14
    #18 0x1a38156eff19 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:548:10
    #19 0x1a3817d3f58b in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:109:5
    #20 0x1a3817d41684 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:270:30

previously allocated by thread T0 (Web Content) here:
    #0 0x557328eea03d in malloc /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
    #1 0x557328f2cbb4 in moz_xmalloc src/memory/mozalloc/mozalloc.cpp:52:15
    #2 0x1a3823b99d86 in operator new(unsigned long) src/objdir-ff-ubsan/dist/include/mozilla/cxxalloc.h:33:10
    #3 0x1a3823b99d86 in mozilla::dom::WorkerPrivate::Constructor(JSContext*, nsTSubstring<char16_t> const&, bool, mozilla::dom::WorkerType, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, mozilla::dom::WorkerLoadInfo*, mozilla::ErrorResult&, nsTString<char16_t>) src/dom/workers/WorkerPrivate.cpp:2413:7
    #4 0x1a3823bfd97f in mozilla::dom::RemoteWorkerChild::ExecWorkerOnMainThread(mozilla::dom::RemoteWorkerData&&) src/dom/workers/remoteworkers/RemoteWorkerChild.cpp:437:41
    #5 0x1a3823c1d72e in mozilla::dom::RemoteWorkerChild::ExecWorker(mozilla::dom::RemoteWorkerData const&)::$_2::operator()() src/dom/workers/remoteworkers/RemoteWorkerChild.cpp:298:29
    #6 0x1a3823c1d5ed in mozilla::detail::RunnableFunction<mozilla::dom::RemoteWorkerChild::ExecWorker(mozilla::dom::RemoteWorkerData const&)::$_2>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:534:5
    #7 0x1a381567a637 in mozilla::SchedulerGroup::Runnable::Run() src/xpcom/threads/SchedulerGroup.cpp:146:20
    #8 0x1a38156a0ee9 in mozilla::RunnableTask::Run() src/xpcom/threads/TaskController.cpp:459:16
    #9 0x1a38156853e2 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:739:26
    #10 0x1a3815682220 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:598:15
    #11 0x1a3815682629 in mozilla::TaskController::ProcessPendingMTTask(bool) src/xpcom/threads/TaskController.cpp:382:36
    #12 0x1a381568818d in mozilla::TaskController::InitializeInternal()::$_4::operator()() const src/xpcom/threads/TaskController.cpp:126:37
    #13 0x1a38156880fd in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_4>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:534:5
    #14 0x1a38156e45fb in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1200:14
    #15 0x1a38156eff19 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:548:10
    #16 0x1a3817d3f58b in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:109:5
    #17 0x1a3817d41684 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:270:30
    #18 0x1a3817a2be3f in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
    #19 0x1a3817a2bd94 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:327:3
    #20 0x1a3817a2bd01 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3

Thread T21 (DOM Worker) created by T0 (Web Content) here:
    #0 0x557328ed4aaa in pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
    #1 0x49216641c279 in _PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:458:14
    #2 0x4921663ff455 in PR_CreateThread src/nsprpub/pr/src/pthreads/ptthread.c:533:12
    #3 0x1a38156dece1 in nsThread::Init(nsTSubstring<char> const&) src/xpcom/threads/nsThread.cpp:658:8
    #4 0x1a3823bd1651 in mozilla::dom::WorkerThread::Create(mozilla::dom::WorkerThreadFriendKey const&) src/dom/workers/WorkerThread.cpp:102:7
    #5 0x1a3823b02c94 in mozilla::dom::workerinternals::RuntimeService::ScheduleWorker(mozilla::dom::WorkerPrivate&) src/dom/workers/RuntimeService.cpp:1364:14
    #6 0x1a3823b01193 in mozilla::dom::workerinternals::RuntimeService::RegisterWorker(mozilla::dom::WorkerPrivate&) src/dom/workers/RuntimeService.cpp:1231:19
    #7 0x1a3823b99f86 in mozilla::dom::WorkerPrivate::Constructor(JSContext*, nsTSubstring<char16_t> const&, bool, mozilla::dom::WorkerType, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, mozilla::dom::WorkerLoadInfo*, mozilla::ErrorResult&, nsTString<char16_t>) src/dom/workers/WorkerPrivate.cpp:2429:24
    #8 0x1a3823bfd97f in mozilla::dom::RemoteWorkerChild::ExecWorkerOnMainThread(mozilla::dom::RemoteWorkerData&&) src/dom/workers/remoteworkers/RemoteWorkerChild.cpp:437:41
    #9 0x1a3823c1d72e in mozilla::dom::RemoteWorkerChild::ExecWorker(mozilla::dom::RemoteWorkerData const&)::$_2::operator()() src/dom/workers/remoteworkers/RemoteWorkerChild.cpp:298:29
    #10 0x1a3823c1d5ed in mozilla::detail::RunnableFunction<mozilla::dom::RemoteWorkerChild::ExecWorker(mozilla::dom::RemoteWorkerData const&)::$_2>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:534:5
    #11 0x1a381567a637 in mozilla::SchedulerGroup::Runnable::Run() src/xpcom/threads/SchedulerGroup.cpp:146:20
    #12 0x1a38156a0ee9 in mozilla::RunnableTask::Run() src/xpcom/threads/TaskController.cpp:459:16
    #13 0x1a38156853e2 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:739:26
    #14 0x1a3815682220 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:598:15
    #15 0x1a3815682629 in mozilla::TaskController::ProcessPendingMTTask(bool) src/xpcom/threads/TaskController.cpp:382:36
    #16 0x1a381568818d in mozilla::TaskController::InitializeInternal()::$_4::operator()() const src/xpcom/threads/TaskController.cpp:126:37
    #17 0x1a38156880fd in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_4>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:534:5
    #18 0x1a38156e45fb in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1200:14
    #19 0x1a38156eff19 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:548:10
    #20 0x1a3817d3f58b in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:109:5
    #21 0x1a3817d41684 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:270:30
    #22 0x1a3817a2be3f in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
    #23 0x1a3817a2bd94 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:327:3
    #24 0x1a3817a2bd01 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3
    #25 0x1a38247dc77b in nsBaseAppShell::Run() src/widget/nsBaseAppShell.cpp:137:27
    #26 0x1a382da8d463 in XRE_RunAppShell() src/toolkit/xre/nsEmbedFunctions.cpp:902:20
    #27 0x1a3817d414ad in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:237:9
    #28 0x1a3817a2be3f in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
    #29 0x1a3817a2bd94 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:327:3
    #30 0x1a3817a2bd01 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3
    #31 0x1a382da8c4b9 in XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/nsEmbedFunctions.cpp:733:34
    #32 0x1a382daacba6 in mozilla::BootstrapImpl::XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/Bootstrap.cpp:67:12
    #33 0x557328f1c98e in content_process_main(mozilla::Bootstrap*, int, char**) src/browser/app/../../ipc/contentproc/plugin-container.cpp:57:28
    #34 0x557328f1d212 in main src/browser/app/nsBrowserApp.cpp:306:18
    #35 0x718e52e65b96 in __libc_start_main /build/glibc-2ORdQG/glibc-2.27/csu/../csu/libc-start.c:310

A Pernosco session is available here: https://pernos.co/debug/pd_8cCzi666lKXE7m7_UAA/index.html

The regressed by bug 1504638 is tentatively as it just introduced the crashing statement. The cause is probably that we have a raw WorkerPrivate* here whose lifetime is not controlled well. Could this better become a WeakPtr ?

Flags: needinfo?(ytausky)
Regressed by: 1504638
Flags: needinfo?(sgiesecke)
See Also: → 1684331
Assignee: nobody → sgiesecke
Status: NEW → ASSIGNED
Flags: needinfo?(sgiesecke)

I think this was regressed by Bug 1594572, which introduced failureCleanup.

Regressed by: 1594572
No longer regressed by: 1504638
Has Regression Range: --- → yes
Keywords: regression
Keywords: sec-high
Crash Signature: [@ RtlAcquireSRWLockExclusive | mozilla::detail::MutexImpl::lock | mozilla::dom::WorkerPrivate::ResetWorkerPrivateInWorkerThread ] [@ RtlAcquireSRWLockExclusive | mozilla::dom::WorkerPrivate::ResetWorkerPrivateInWorkerThread ] [@ pthread_mutex_lock | mo…
Flags: needinfo?(ytausky)

Based on the regressing bug, this also affects ESR78, and we have also occasional crash reports matching this, e.g. https://crash-stats.mozilla.org/report/index/38436ccb-7baf-4873-ba35-006a50210107

Comment on attachment 9198196 [details]
Bug 1687597 - Ensure worker thread is reset before scheduling for deletion. r=#dom-workers-and-storage

Security Approval Request

  • How easily could an exploit be constructed based on the patch?: It's necessarily obvious from the patch that it changes the order of cleanups to avoid a UAF. It's unclear though how the situation can be provoked.

The comment could be moved to a separate patch if this is considered to attract additional attention.

  • Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?: Yes
  • Which older supported branches are affected by this flaw?: all
  • If not all supported branches, which bug introduced the flaw?: None
  • Do you have backports for the affected branches?: No
  • If not, how different, hard to create, and risky will they be?: Should backporting require manual adaptation, this should be fairly easy. At least the general structure of WorkerThreadPrimaryRunnable::Run hasn't changed since.
  • How likely is this patch to cause regressions; how much testing does it need?: Unlikely, this only changes the behaviour for rather rare error cases.
Attachment #9198196 - Flags: sec-approval?

There are more signatures related to this, in particular on Linux, but they are very scattered, see Bug 1687907.

See Also: → 1687907
Severity: -- → S2
Priority: -- → P1

Comment on attachment 9198196 [details]
Bug 1687597 - Ensure worker thread is reset before scheduling for deletion. r=#dom-workers-and-storage

Approved to land and uplift.

Attachment #9198196 - Flags: sec-approval?
Attachment #9198196 - Flags: sec-approval+
Attachment #9198196 - Flags: approval-mozilla-esr78+
Attachment #9198196 - Flags: approval-mozilla-beta+
Group: dom-core-security → core-security-release
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → 87 Branch

As part of a security bug pattern analysis, we are requesting your help with a high level analysis of this bug. It is our hope to develop static analysis (or potentially runtime/dynamic analysis) in the future to identify classes of bugs.

Please visit this google form to reply.

Flags: needinfo?(sgiesecke)
Whiteboard: [sec-survey]
QA Whiteboard: [post-critsmash-triage]
Flags: qe-verify-

This needs a rebased patch for ESR78 uplift.

(In reply to Release mgmt bot [:sylvestre / :calixte / :marco for bugbug] from comment #10)

As part of a security bug pattern analysis, we are requesting your help with a high level analysis of this bug. It is our hope to develop static analysis (or potentially runtime/dynamic analysis) in the future to identify classes of bugs.

Please visit this google form to reply.

Responded.

Flags: needinfo?(sgiesecke)

(In reply to Ryan VanderMeulen [:RyanVM] from comment #12)

This needs a rebased patch for ESR78 uplift.

A rebased patch is now attached and reviewed.

(In reply to Simon Giesecke [:sg] [he/him] from comment #15)

A rebased patch is now attached and reviewed.

TY!
https://hg.mozilla.org/releases/mozilla-esr78/rev/9c662ba11011

Whiteboard: [sec-survey] → [sec-survey][adv-main86+r]
Whiteboard: [sec-survey][adv-main86+r] → [sec-survey][adv-main86+r][adv-esr78.8+r]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: