1493689 - SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:772:48 in nr_ice_media_stream_component_failed

Status: RESOLVED FIXED

(Core :: WebRTC: Networking, defect, P1)

Description

[Jason Kratzer [:jkratzer]]

Found while fuzzing mozilla-central rev 095ec59a8800.

I'm currently reducing the testcase and will update once complete.

==11993==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b7c5c at pc 0x7fb0f0916ab2 bp 0x7fb0979fd120 sp 0x7fb0979fd118
READ of size 8 at 0x6110000b7c5c thread T8 (Socket Thread)
    #0 0x7fb0f0916ab1 in nr_ice_media_stream_component_failed /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:772:48
    #1 0x7fb0f09380ce in nr_ice_peer_ctx_trickle_wait_cb /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:354:9
    #2 0x7fb0e4ca0a35 in Notify /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp:132:3
    #3 0x7fb0e4ca0a35 in non-virtual thunk to mozilla::nrappkitTimerCallback::Notify(nsITimer*) /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp
    #4 0x7fb0e252f733 in nsTimerImpl::Fire(int) /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp:704:40
    #5 0x7fb0e24eae6d in nsTimerEvent::Run() /builds/worker/workspace/build/src/xpcom/threads/TimerThread.cpp:297:11
    #6 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #7 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #8 0x7fb0e283160f in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #9 0x7fb0e28345a4 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #10 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #11 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #12 0x7fb0e371e3a1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #13 0x7fb0e361f73c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #14 0x7fb0e361f73c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #15 0x7fb0e361f73c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #16 0x7fb0e24fa55f in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:465:11
    #17 0x7fb10593e008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #18 0x7fb1055846da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #19 0x7fb10455d88e in clone /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x6110000b7c5c is located 28 bytes inside of 200-byte region [0x6110000b7c40,0x6110000b7d08)
freed by thread T8 (Socket Thread) here:
    #0 0x562d397a5372 in __interceptor_free /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:68:3
    #1 0x7fb0f091dd94 in nr_ice_media_stream_destroy /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:130:5
    #2 0x7fb0f091d267 in nr_ice_peer_ctx_remove_pstream /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:259:10
    #3 0x7fb0f091d267 in nr_ice_remove_media_stream /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:829
    #4 0x7fb0e4cd1e35 in mozilla::NrIceMediaStream::CloseStream(nr_ice_media_stream_**) /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:690:13
    #5 0x7fb0e4ce5afc in mozilla::NrIceMediaStream::Failed() /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:673:5
    #6 0x7fb0e4cb00d4 in mozilla::NrIceCtx::stream_failed(void*, nr_ice_media_stream_*) /builds/worker/workspace/build/src/media/mtransport/nricectx.cpp:383:8
    #7 0x7fb0f0916a9b in nr_ice_media_stream_component_failed /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:768:7
    #8 0x7fb0f09380ce in nr_ice_peer_ctx_trickle_wait_cb /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:354:9
    #9 0x7fb0e4ca0a35 in Notify /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp:132:3
    #10 0x7fb0e4ca0a35 in non-virtual thunk to mozilla::nrappkitTimerCallback::Notify(nsITimer*) /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp
    #11 0x7fb0e252f733 in nsTimerImpl::Fire(int) /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp:704:40
    #12 0x7fb0e24eae6d in nsTimerEvent::Run() /builds/worker/workspace/build/src/xpcom/threads/TimerThread.cpp:297:11
    #13 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #14 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #15 0x7fb0e283160f in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #16 0x7fb0e28345a4 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #17 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #18 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #19 0x7fb0e371e3a1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #20 0x7fb0e361f73c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #21 0x7fb0e361f73c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #22 0x7fb0e361f73c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #23 0x7fb0e24fa55f in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:465:11
    #24 0x7fb10593e008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #25 0x7fb1055846da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)

previously allocated by thread T8 (Socket Thread) here:
    #0 0x562d397a56b3 in malloc /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:88:3
    #1 0x7fb0f09708b6 in r_malloc /builds/worker/workspace/build/src/media/mtransport/third_party/nrappkit/src/util/libekr/r_memory.c:73:16
    #2 0x7fb0f09708b6 in r_calloc /builds/worker/workspace/build/src/media/mtransport/third_party/nrappkit/src/util/libekr/r_memory.c:98
    #3 0x7fb0f091c845 in nr_ice_media_stream_create /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:54:17
    #4 0x7fb0f0929d04 in nr_ice_peer_ctx_parse_stream_attributes /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:102:10
    #5 0x7fb0e4cd0cd3 in mozilla::NrIceMediaStream::ConnectToPeer(std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&) /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:241:13
    #6 0x7fb0e4aa8697 in mozilla::PeerConnectionMedia::ActivateTransport_s(std::string const&, std::string const&, std::string const&, unsigned long, std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&) /builds/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionMedia.cpp:454:25
    #7 0x7fb0e4afa97e in apply<RefPtr<mozilla::PeerConnectionMedia>, void (mozilla::PeerConnectionMedia::*)(const std::basic_string<char> &, const std::basic_string<char> &, const std::basic_string<char> &, unsigned long, const std::basic_string<char> &, const std::basic_string<char> &, const std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > &), std::basic_string<char>, std::basic_string<char>, std::basic_string<char>, unsigned long, std::basic_string<char>, std::basic_string<char>, std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > , 0, 1, 2, 3, 4, 5, 6> /builds/worker/workspace/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:86:5
    #8 0x7fb0e4afa97e in mozilla::runnable_args_memfn<RefPtr<mozilla::PeerConnectionMedia>, void (mozilla::PeerConnectionMedia::*)(std::string const&, std::string const&, std::string const&, unsigned long, std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&), std::string, std::string, std::string, unsigned long, std::string, std::string, std::vector<std::string, std::allocator<std::string> > >::Run() /builds/worker/workspace/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:156
    #9 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #10 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #11 0x7fb0e283160f in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #12 0x7fb0e28345a4 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #13 0x7fb0e2502897 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1166:14
    #14 0x7fb0e250b415 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #15 0x7fb0e371e3a1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #16 0x7fb0e361f73c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #17 0x7fb0e361f73c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #18 0x7fb0e361f73c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #19 0x7fb0e24fa55f in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:465:11
    #20 0x7fb10593e008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #21 0x7fb1055846da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)

Thread T8 (Socket Thread) created by T0 (file:// Content) here:
    #0 0x562d3978e73d in __interceptor_pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:204:3
    #1 0x7fb10593ad45 in _PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:433:14
    #2 0x7fb10593a92e in PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:518:12
    #3 0x7fb0e24fdad8 in nsThread::Init(nsTSubstring<char> const&) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:660:8
    #4 0x7fb0e2509bee in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp:518:22
    #5 0x7fb0e250ee0e in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, nsIRunnable*, unsigned int) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:143:45
    #6 0x7fb0e282e7b1 in NS_NewNamedThread<14> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:75:10
    #7 0x7fb0e282e7b1 in mozilla::net::nsSocketTransportService::Init() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:640
    #8 0x7fb0e35e06b4 in nsSocketTransportServiceConstructor(nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/netwerk/build/nsNetModule.cpp:75:1
    #9 0x7fb0e2498bce in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1161:19
    #10 0x7fb0e248fbf3 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1521:10
    #11 0x7fb0e249efc5 in CallGetService /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:67:43
    #12 0x7fb0e249efc5 in nsGetServiceByContractIDWithError::operator()(nsID const&, void**) const /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:292
    #13 0x7fb0e22e4ce3 in nsCOMPtr_base::assign_from_gs_contractid_with_error(nsGetServiceByContractIDWithError const&, nsID const&) /builds/worker/workspace/build/src/xpcom/base/nsCOMPtr.cpp:106:7
    #14 0x7fb0e2752158 in operator= /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:704:5
    #15 0x7fb0e2752158 in InitializeSocketTransportService /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:299
    #16 0x7fb0e2752158 in mozilla::net::nsIOService::SetOffline(bool) /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:1074
    #17 0x7fb0e27505a7 in mozilla::net::nsIOService::Init() /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:263:5
    #18 0x7fb0e2754963 in mozilla::net::nsIOService::GetInstance() /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:360:13
    #19 0x7fb0e35e0396 in nsIOServiceConstructor(nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/netwerk/build/nsNetModule.cpp:57:1
    #20 0x7fb0e2498bce in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1161:19
    #21 0x7fb0e248fbf3 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1521:10
    #22 0x7fb0e4e9f12a in CallGetService<nsIIOService> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsServiceManagerUtils.h:90:10
    #23 0x7fb0e4e9f12a in nsScriptSecurityManager::Init() /builds/worker/workspace/build/src/caps/nsScriptSecurityManager.cpp:1389
    #24 0x7fb0e4ea020c in nsScriptSecurityManager::InitStatics() /builds/worker/workspace/build/src/caps/nsScriptSecurityManager.cpp:1458:30
    #25 0x7fb0e47e7acf in nsXPConnect::InitStatics() /builds/worker/workspace/build/src/js/xpconnect/src/nsXPConnect.cpp:140:5
    #26 0x7fb0e476aa70 in xpcModuleCtor() /builds/worker/workspace/build/src/js/xpconnect/src/XPCModule.cpp:13:5
    #27 0x7fb0ed7db838 in Initialize() /builds/worker/workspace/build/src/layout/build/nsLayoutModule.cpp:235:8
    #28 0x7fb0e24965d7 in nsComponentManagerImpl::KnownModule::Load() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:838:21
    #29 0x7fb0e2497e64 in nsFactoryEntry::GetFactory() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1859:19
    #30 0x7fb0e2498b86 in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1158:41
    #31 0x7fb0e249e9fb in CallCreateInstance /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:149:38
    #32 0x7fb0e249e9fb in nsCreateInstanceByContractID::operator()(nsID const&, void**) const /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:197
    #33 0x7fb0e22e5160 in nsCOMPtr_base::assign_from_helper(nsCOMPtr_helper const&, nsID const&) /builds/worker/workspace/build/src/xpcom/base/nsCOMPtr.cpp:128:7
    #34 0x7fb0e247f151 in nsCOMPtr /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:605:5
    #35 0x7fb0e247f151 in LogMessageWithContext(mozilla::FileLocation&, unsigned int, char const*, ...) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:151
    #36 0x7fb0e2485b12 in nsComponentManagerImpl::ManifestContract(nsComponentManagerImpl::ManifestProcessingContext&, int, char* const*) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:769:5
    #37 0x7fb0e24829f7 in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:695:7
    #38 0x7fb0e2494096 in DoRegisterManifest /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:662:5
    #39 0x7fb0e2494096 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:675
    #40 0x7fb0e24944ad in nsComponentManagerImpl::ManifestManifest(nsComponentManagerImpl::ManifestProcessingContext&, int, char* const*) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:684:3
    #41 0x7fb0e24829f7 in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:695:7
    #42 0x7fb0e2494096 in DoRegisterManifest /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:662:5
    #43 0x7fb0e2494096 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:675
    #44 0x7fb0e2492d52 in nsComponentManagerImpl::RereadChromeManifests(bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:796:5
    #45 0x7fb0e24910ff in nsComponentManagerImpl::Init() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:414:5
    #46 0x7fb0e257434d in NS_InitXPCOM2 /builds/worker/workspace/build/src/xpcom/build/XPCOMInit.cpp:684:51
    #47 0x7fb0f07cf48c in XRE_InitEmbedding2(nsIFile*, nsIFile*, nsIDirectoryServiceProvider*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:195:8
    #48 0x7fb0e372bf2e in mozilla::ipc::ScopedXREEmbed::Start() /builds/worker/workspace/build/src/ipc/glue/ScopedXREEmbed.cpp
    #49 0x7fb0eb9889d7 in mozilla::dom::ContentProcess::Init(int, char**) /builds/worker/workspace/build/src/dom/ipc/ContentProcess.cpp:296:13
    #50 0x7fb0f07d03f0 in XRE_InitChildProcess(int, char**, XREChildData const*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:739:21
    #51 0x562d397d5b91 in content_process_main /builds/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:50:30
    #52 0x562d397d5b91 in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:287
    #53 0x7fb10445db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:772:48 in nr_ice_media_stream_component_failed
Shadow bytes around the buggy address:
  0x0c228000ef30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000ef40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000ef50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000ef60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000ef70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c228000ef80: fa fa fa fa fa fa fa fa fd fd fd[fd]fd fd fd fd
  0x0c228000ef90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c228000efa0: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000efb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000efc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c228000efd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==11993==ABORTING

Comment 1

[Nils Ohlmeier [:drno]]

Byron, can you please have a look at this once Jason can provide a reduced test case?

Comment 2

[Byron Campen [:bwc]]

I see the problem here.

Comment 3

[Byron Campen [:bwc]]

Try push here, might save you the trouble of reducing the test-case:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=bb7c8519ab7d816a64e805e8e8bdb48a44e3726b

Comment 4

[Jason Kratzer [:jkratzer]]

(In reply to Byron Campen [:bwc] from comment #3)
> Try push here, might save you the trouble of reducing the test-case:
> 
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=bb7c8519ab7d816a64e805e8e8bdb48a44e3726b

Unfortunately, it appears that the issue still exists using that try.  I'm nearly done reducing this testcase.  I should have it ready later today.

Comment 5

[Jason Kratzer [:jkratzer]]

Attachment: testcase.html

The attached testcase is pretty trick to reproduce.  I've added a spray routine to cause memory pressure so you may have to adjust that depending on your setup.

Steps to reproduce:
1.  Start local webserver:
    python -m SimpleHTTPServer
2.  Use ffpuppet and the attached prefs via:
    python -m ffpuppet -p prefs --xvfb -d -l log ~/builds/mc-asan/firefox http://localhost:8000/testcase.html

FFpuppet can be found at:
https://github.com/MozillaSecurity/ffpuppet

Comment 6

[Jason Kratzer [:jkratzer]]

Attachment: prefs.js

Comment 7

[Byron Campen [:bwc]]

I've tried using ffpuppet, but it does not seem to work. The closest I could get was by creating a shell script that called "mozilla-central/objdir-ff-asan/dist/bin/firefox http://localhost:8000/testcase.html" (because ffpuppet doesn't seem to handle arguments to the binary you hand it), and then calling that shell script like this:

python -m ffpuppet -p prefs.js --xvfb -d -l log run_testcase.sh

I get the following output, which seems to indicate that the browser never successfully launched:

[2018-10-02 10:47:11] Shutting down...
[2018-10-02 10:47:11] Firefox process closed
[2018-10-02 10:47:11] Dumping browser log...

===
=== Dumping 'log_ffp_asan_31372.log.31452.txt' (0.29KB)
===
=================================================================
==31452==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x14ba273951da bp 0x14ba213d3280 sp 0x14ba213d3270 T2)
==31452==The signal is caused by a WRITE memory access.
==31452==Hint: address points to the zero page.

===
=== Dumping 'log_ffp_asan_31372.log.31527.txt' (0.29KB)
===
=================================================================
==31527==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x14e04c7951da bp 0x14e0467f4280 sp 0x14e0467f4270 T2)
==31527==The signal is caused by a WRITE memory access.
==31527==Hint: address points to the zero page.

===
=== Dumping 'log_ffp_asan_31372.log.31571.txt' (0.29KB)
===
=================================================================
==31571==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x14a10bd301da bp 0x14a105d69280 sp 0x14a105d69270 T2)
==31571==The signal is caused by a WRITE memory access.
==31571==Hint: address points to the zero page.

===
=== Dumping 'log_stdout.txt' (0.86KB)
===
Couldn't convert chrome URL: chrome://branding/locale/brand.properties
Couldn't convert chrome URL: chrome://branding/locale/brand.properties
Couldn't convert chrome URL: chrome://branding/locale/brand.properties
Couldn't convert chrome URL: chrome://branding/locale/brand.properties
nsStringStats
 => mAllocCount:          72077
 => mReallocCount:            0
 => mFreeCount:           72077
 => mShareCount:          88710
 => mAdoptCount:           2797
 => mAdoptFreeCount:       2819
 => Process ID: 31494, Thread ID: 22820382873536
Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Receive IPC close with reason=AbnormalShutdown (t=293.213) Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Receive IPC close with reason=AbnormalShutdown (t=297.81) Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Receive IPC close with reason=AbnormalShutdown (t=294.567) 
===
=== Dumping 'log_stderr.txt' (20.73KB)
===
[ffpuppet] Launch command: /home/bcampen/checkouts/run_testcase.sh -no-remote -profile /tmp/ffprof_YjYDGa http://127.0.0.1:28586

++DOCSHELL 0x619000292c80 == 1 [pid = 31386] [id = {19bc9245-c3cf-4bab-a507-106f36672b5a}]
++DOMWINDOW == 1 (0x6150001b4980) [pid = 31386] [serial = 1] [outer = (nil)]
++DOMWINDOW == 2 (0x619000293b80) [pid = 31386] [serial = 2] [outer = 0x6150001b4980]
[31386, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, NS_ERROR_UNEXPECTED) failed with result 0x80004005: file /home/bcampen/checkouts/mozilla-central/extensions/cookie/nsPermissionManager.cpp, line 1035
[31386, Main Thread] WARNING: Last startup was detected as a crash.: file /home/bcampen/checkouts/mozilla-central/toolkit/components/startup/nsAppStartup.cpp, line 906
[31386, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040111: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/res/SubstitutingProtocolHandler.cpp, line 342
++DOCSHELL 0x619000421380 == 2 [pid = 31386] [id = {3b0a1fa1-e68d-4fe9-b9a9-1514a6d96130}]
++DOMWINDOW == 3 (0x615000328c80) [pid = 31386] [serial = 3] [outer = (nil)]
++DOMWINDOW == 4 (0x619000421d80) [pid = 31386] [serial = 4] [outer = 0x615000328c80]
++DOMWINDOW == 5 (0x61900047e080) [pid = 31386] [serial = 5] [outer = 0x615000328c80]
[31386, Main Thread] WARNING: Attempting to get a displayport from a content with no primary frame!: file /home/bcampen/checkouts/mozilla-central/layout/base/nsLayoutUtils.cpp, line 805
++DOCSHELL 0x61900050c380 == 3 [pid = 31386] [id = {375129af-55be-4fdc-8916-82e088fb26ca}]
++DOMWINDOW == 6 (0x61500042f980) [pid = 31386] [serial = 6] [outer = (nil)]
++DOCSHELL 0x61900073af80 == 4 [pid = 31386] [id = {38c1dcdf-c9d4-451a-848f-aefe3b25b1f1}]
++DOMWINDOW == 7 (0x61500065ed00) [pid = 31386] [serial = 7] [outer = (nil)]
++DOMWINDOW == 8 (0x61900077d380) [pid = 31386] [serial = 8] [outer = 0x61500065ed00]
++DOMWINDOW == 9 (0x6190007d6480) [pid = 31386] [serial = 9] [outer = 0x61500042f980]
++DOCSHELL 0x619000206c80 == 1 [pid = 31452] [id = {4f9fc255-06f6-41ee-97b4-4d6b85bd2f0c}]
++DOMWINDOW == 1 (0x6150001b8a80) [pid = 31452] [serial = 1] [outer = (nil)]
++DOMWINDOW == 2 (0x619000299580) [pid = 31452] [serial = 2] [outer = 0x6150001b8a80]
[Child 31452, Main Thread] WARNING: site security information will not be persisted: file /home/bcampen/checkouts/mozilla-central/security/manager/ssl/nsSiteSecurityService.cpp, line 553
[Parent 31386, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040111: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/res/SubstitutingProtocolHandler.cpp, line 342
++DOMWINDOW == 3 (0x6190002be880) [pid = 31452] [serial = 3] [outer = 0x6150001b8a80]
[Parent 31386, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040111: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/res/SubstitutingProtocolHandler.cpp, line 342
++DOCSHELL 0x619000352a80 == 2 [pid = 31452] [id = {cb12b8ee-23d7-4069-a336-c49cc06af856}]
++DOMWINDOW == 4 (0x61500021bb80) [pid = 31452] [serial = 4] [outer = (nil)]
++DOMWINDOW == 5 (0x619000356680) [pid = 31452] [serial = 5] [outer = 0x61500021bb80]
++DOMWINDOW == 6 (0x61900035de80) [pid = 31452] [serial = 6] [outer = 0x61500021bb80]
++DOCSHELL 0x619000e71380 == 5 [pid = 31386] [id = {be8e6431-7e95-4c30-a1b0-7389eea0aa18}]
++DOMWINDOW == 10 (0x615000c65c80) [pid = 31386] [serial = 10] [outer = (nil)]
++DOMWINDOW == 11 (0x619000e71d80) [pid = 31386] [serial = 11] [outer = 0x615000c65c80]
++DOMWINDOW == 12 (0x619000e73680) [pid = 31386] [serial = 12] [outer = 0x615000c65c80]
++DOMWINDOW == 13 (0x619000e7c280) [pid = 31386] [serial = 13] [outer = 0x615000c65c80]
[Parent 31386, Main Thread] WARNING: Need TabChild to get the nativeWindow from!: file /home/bcampen/checkouts/mozilla-central/widget/PuppetWidget.cpp, line 1187
++DOCSHELL 0x619000ecf980 == 6 [pid = 31386] [id = {a8beba3d-ac57-43d4-9366-5df0285be581}]
++DOMWINDOW == 14 (0x615000cb6e00) [pid = 31386] [serial = 14] [outer = (nil)]
++DOMWINDOW == 15 (0x619000ee0280) [pid = 31386] [serial = 15] [outer = 0x615000cb6e00]
++DOMWINDOW == 16 (0x619000ee1b80) [pid = 31386] [serial = 16] [outer = 0x615000cb6e00]
++DOMWINDOW == 17 (0x619000eec580) [pid = 31386] [serial = 17] [outer = 0x615000cb6e00]
[Parent 31386, Main Thread] WARNING: Need TabChild to get the nativeWindow from!: file /home/bcampen/checkouts/mozilla-central/widget/PuppetWidget.cpp, line 1187
++DOCSHELL 0x6190001e9680 == 1 [pid = 31527] [id = {e9dca5f5-446e-4c55-8255-5d12e8f7d7af}]
++DOMWINDOW == 1 (0x6150001b9c00) [pid = 31527] [serial = 1] [outer = (nil)]
[Child 31527, Main Thread] WARNING: Fallback to BasicLayerManager: file /home/bcampen/checkouts/mozilla-central/dom/ipc/TabChild.cpp, line 2858
++DOMWINDOW == 2 (0x61900029d680) [pid = 31527] [serial = 2] [outer = 0x6150001b9c00]
++DOCSHELL 0x6190002c9280 == 2 [pid = 31527] [id = {e21be36a-fd38-4747-a797-2d5227215e0f}]
++DOMWINDOW == 3 (0x6150001fbb00) [pid = 31527] [serial = 3] [outer = (nil)]
[Child 31527, Main Thread] WARNING: Fallback to BasicLayerManager: file /home/bcampen/checkouts/mozilla-central/dom/ipc/TabChild.cpp, line 2858
++DOMWINDOW == 4 (0x6190002cba80) [pid = 31527] [serial = 4] [outer = 0x6150001fbb00]
++DOMWINDOW == 5 (0x6190002d4b80) [pid = 31527] [serial = 5] [outer = 0x6150001b9c00]
++DOMWINDOW == 6 (0x6190002d5f80) [pid = 31527] [serial = 6] [outer = 0x6150001fbb00]
++DOCSHELL 0x619001033d80 == 7 [pid = 31386] [id = {5c3a878d-453f-444b-90d3-87748e8ae128}]
++DOMWINDOW == 18 (0x615000e43c00) [pid = 31386] [serial = 18] [outer = (nil)]
++DOMWINDOW == 19 (0x619001034780) [pid = 31386] [serial = 19] [outer = 0x615000e43c00]
++DOMWINDOW == 20 (0x619001036a80) [pid = 31386] [serial = 20] [outer = 0x615000e43c00]
[Parent 31386, Main Thread] WARNING: Need TabChild to get the nativeWindow from!: file /home/bcampen/checkouts/mozilla-central/widget/PuppetWidget.cpp, line 1187
++DOCSHELL 0x61900020c180 == 1 [pid = 31494] [id = {6d037dbb-fbb6-412a-bfb5-87f1876d779c}]
++DOMWINDOW == 1 (0x6150001cf000) [pid = 31494] [serial = 1] [outer = (nil)]
[Child 31494, Main Thread] WARNING: Fallback to BasicLayerManager: file /home/bcampen/checkouts/mozilla-central/dom/ipc/TabChild.cpp, line 2858
++DOMWINDOW == 2 (0x6190002b1b80) [pid = 31494] [serial = 2] [outer = 0x6150001cf000]
[Child 31494, Main Thread] WARNING: site security information will not be persisted: file /home/bcampen/checkouts/mozilla-central/security/manager/ssl/nsSiteSecurityService.cpp, line 553
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
++DOCSHELL 0x6190004c4580 == 8 [pid = 31386] [id = {8443348d-8bb0-4e99-98f9-ac47bd97b5d1}]
++DOMWINDOW == 21 (0x615001dfce80) [pid = 31386] [serial = 21] [outer = (nil)]
++DOMWINDOW == 22 (0x6190004c3b80) [pid = 31386] [serial = 22] [outer = 0x615001dfce80]
++DOMWINDOW == 23 (0x6190004c2c80) [pid = 31386] [serial = 23] [outer = 0x615001dfce80]
++DOMWINDOW == 24 (0x6190004b9180) [pid = 31386] [serial = 24] [outer = 0x615001dfce80]
++DOMWINDOW == 3 (0x6190002cf180) [pid = 31494] [serial = 3] [outer = 0x6150001cf000]
[Parent 31386, Main Thread] WARNING: Need TabChild to get the nativeWindow from!: file /home/bcampen/checkouts/mozilla-central/widget/PuppetWidget.cpp, line 1187
++DOCSHELL 0x619000425980 == 3 [pid = 31527] [id = {37030fa0-b54d-4018-a1dc-512e858af06c}]
++DOMWINDOW == 7 (0x615000352a80) [pid = 31527] [serial = 7] [outer = (nil)]
[Child 31527, Main Thread] WARNING: Fallback to BasicLayerManager: file /home/bcampen/checkouts/mozilla-central/dom/ipc/TabChild.cpp, line 2858
++DOMWINDOW == 8 (0x619000427780) [pid = 31527] [serial = 8] [outer = 0x615000352a80]
++DOMWINDOW == 9 (0x61900042cc80) [pid = 31527] [serial = 9] [outer = 0x615000352a80]
++DOCSHELL 0x619000ed1c80 == 2 [pid = 31494] [id = {387da1f7-9cc5-46ec-8069-2b4c43b1cbb3}]
++DOMWINDOW == 4 (0x61500062af00) [pid = 31494] [serial = 4] [outer = (nil)]
[Child 31494, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/layout/style/StyleSheet.cpp, line 768
[Child 31494, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/layout/style/StyleSheet.cpp, line 768
++DOMWINDOW == 5 (0x619000f25880) [pid = 31494] [serial = 5] [outer = 0x61500062af00]
--DOCSHELL 0x61900073af80 == 7 [pid = 31386] [id = {38c1dcdf-c9d4-451a-848f-aefe3b25b1f1}]
[Parent 31386, QuotaManager IO] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001: file ../../storage/mozStorageConnection.cpp, line 754
[Parent 31386, QuotaManager IO] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001: file /home/bcampen/checkouts/mozilla-central/storage/mozStorageService.cpp, line 713
[Parent 31386, QuotaManager IO] WARNING: Received NS_ERROR_STORAGE_BUSY when attempting to open database '1231742593tbwecw-.sqlite', retrying for up to 10 seconds: file ../../../dom/indexedDB/ActorsParent.cpp, line 4447
[Child 31494, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, false) failed with result 0x80004005: file /home/bcampen/checkouts/mozilla-central/docshell/shistory/nsSHistory.cpp, line 1291
[Child 31494, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, false) failed with result 0x80004005: file /home/bcampen/checkouts/mozilla-central/docshell/shistory/nsSHistory.cpp, line 1291
[Parent 31386, QuotaManager IO] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001: file ../../storage/mozStorageConnection.cpp, line 754
[Parent 31386, QuotaManager IO] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001: file /home/bcampen/checkouts/mozilla-central/storage/mozStorageService.cpp, line 713
[Parent 31386, QuotaManager IO] WARNING: Received NS_ERROR_STORAGE_BUSY when attempting to open database '2293386553%25B430%25tcw.sqlite', retrying for up to 10 seconds: file ../../../dom/indexedDB/ActorsParent.cpp, line 4447
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
[Child 31494, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/layout/style/StyleSheet.cpp, line 768
[Child 31494, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/layout/style/StyleSheet.cpp, line 768
--DOMWINDOW == 5 (0x619000299580) [pid = 31452] [serial = 2] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 4 (0x619000356680) [pid = 31452] [serial = 5] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 23 (0x619000421d80) [pid = 31386] [serial = 4] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 22 (0x6190004c2c80) [pid = 31386] [serial = 23] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 21 (0x6190004c3b80) [pid = 31386] [serial = 22] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 20 (0x619000e71d80) [pid = 31386] [serial = 11] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 19 (0x619000e73680) [pid = 31386] [serial = 12] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 18 (0x619000ee0280) [pid = 31386] [serial = 15] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 17 (0x619000ee1b80) [pid = 31386] [serial = 16] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 16 (0x619001034780) [pid = 31386] [serial = 19] [outer = (nil)] [url = about:blank]
--DOCSHELL 0x619000ed1c80 == 1 [pid = 31494] [id = {387da1f7-9cc5-46ec-8069-2b4c43b1cbb3}]
--DOMWINDOW == 15 (0x61500065ed00) [pid = 31386] [serial = 7] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 14 (0x61900077d380) [pid = 31386] [serial = 8] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 8 (0x61900029d680) [pid = 31527] [serial = 2] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 7 (0x619000427780) [pid = 31527] [serial = 8] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 6 (0x6190002cba80) [pid = 31527] [serial = 4] [outer = (nil)] [url = about:blank]
[Child 31494, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Child 31494, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
++DOMWINDOW == 6 (0x6190000f8780) [pid = 31494] [serial = 6] [outer = 0x6150001cf000]
[Child 31494, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Parent 31386, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Child 31527, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Child 31571, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Child 31494, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Child 31494, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file /home/bcampen/checkouts/mozilla-central/dom/workers/WorkerPrivate.cpp, line 1585
[Child 31494, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file /home/bcampen/checkouts/mozilla-central/dom/workers/WorkerPrivate.cpp, line 1585
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
[Child 31452, Main Thread] WARNING: '!window', file /home/bcampen/checkouts/mozilla-central/dom/cache/CacheStorage.cpp, line 596
[Parent 31386, Main Thread] WARNING: 'NS_FAILED(rv)', file /home/bcampen/checkouts/mozilla-central/netwerk/base/nsChannelClassifier.cpp, line 483
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
--DOMWINDOW == 5 (0x6190002b1b80) [pid = 31494] [serial = 2] [outer = (nil)] [url = about:blank]
++DOMWINDOW == 6 (0x619001146d80) [pid = 31494] [serial = 7] [outer = 0x6150001cf000]
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
[Parent 31386, Main Thread] WARNING: NS_ENSURE_TRUE(mCacheEntry) failed: file /home/bcampen/checkouts/mozilla-central/netwerk/protocol/http/nsHttpChannel.cpp, line 5287
[Parent 31386, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/dom/indexedDB/IDBDatabase.cpp, line 586
[Parent 31386, Main Thread] WARNING: 'aRv.Failed()', file /home/bcampen/checkouts/mozilla-central/dom/indexedDB/IDBDatabase.cpp, line 586
++DOMWINDOW == 7 (0x619001349880) [pid = 31494] [serial = 8] [outer = 0x6150001cf000]
--DOMWINDOW == 6 (0x61500062af00) [pid = 31494] [serial = 4] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 5 (0x6190002cf180) [pid = 31494] [serial = 3] [outer = (nil)] [url = https://weather.com/]
--DOMWINDOW == 4 (0x6190000f8780) [pid = 31494] [serial = 6] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 3 (0x619000f25880) [pid = 31494] [serial = 5] [outer = (nil)] [url = about:blank]
--DOMWINDOW == 2 (0x619001146d80) [pid = 31494] [serial = 7] [outer = (nil)] [url = https://www.lego.com/en-us/themes/city/games/mycity2-c7722ac2c54b4a1a837ae33e20ec90bc]

###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

[Child 31494, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThread.cpp, line 929
[Child 31494, Main Thread] WARNING: '!gThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsTimerImpl.cpp, line 399
--DOCSHELL 0x61900020c180 == 0 [pid = 31494] [id = {6d037dbb-fbb6-412a-bfb5-87f1876d779c}]
--DOMWINDOW == 1 (0x6150001cf000) [pid = 31494] [serial = 1] [outer = (nil)] [url = about:blank]
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
[Child 31494, Main Thread] WARNING: '!mMainThread', file /home/bcampen/checkouts/mozilla-central/xpcom/threads/nsThreadManager.cpp, line 510
--DOMWINDOW == 0 (0x619001349880) [pid = 31494] [serial = 8] [outer = (nil)] [url = about:blank]
Hit MOZ_CRASH(Aborting on channel error.) at /home/bcampen/checkouts/mozilla-central/ipc/glue/MessageChannel.cpp:2662
Hit MOZ_CRASH(Aborting on channel error.) at /home/bcampen/checkouts/mozilla-central/ipc/glue/MessageChannel.cpp:2662
Hit MOZ_CRASH(Aborting on channel error.) at /home/bcampen/checkouts/mozilla-central/ipc/glue/MessageChannel.cpp:2662
#01: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4dd083e]
#01: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4dd083e]
#01: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4dd083e]
#02: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d63973]
#02: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d63973]
#02: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d63973]
#03: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d42cf4]
#03: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d42cf4]
#03: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4d42cf4]
#04: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cca98f]
#04: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cca98f]
#04: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cca98f]
#05: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc45eb]
#05: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc45eb]
#05: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc45eb]
#06: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc4377]
#06: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc4377]
#06: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cc4377]
#07: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cee509]
#07: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cee509]
#07: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cee509]
#08: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cd5176]
#09: ???[/lib64/libpthread.so.0 +0x750b]
#10: clone[/lib64/libc.so.6 +0xf516f]
#11: ??? (???:???)
#08: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cd5176]
#09: ???[/lib64/libpthread.so.0 +0x750b]
ASAN:DEADLYSIGNAL
#10: clone[/lib64/libc.so.6 +0xf516f]
#11: ??? (???:???)
ASAN:DEADLYSIGNAL
#08: ???[/home/bcampen/checkouts/mozilla-central/objdir-ff-asan/dist/bin/libxul.so +0x4cd5176]
#09: ???[/lib64/libpthread.so.0 +0x750b]
#10: clone[/lib64/libc.so.6 +0xf516f]
#11: ??? (???:???)
ASAN:DEADLYSIGNAL
[ffpuppet] Reason code: CLOSED

Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/bcampen/checkouts/ffpuppet/ffpuppet/__main__.py", line 9, in <module>
    main()
  File "/home/bcampen/checkouts/ffpuppet/ffpuppet/main.py", line 167, in main
    extension=args.extension)
  File "/home/bcampen/checkouts/ffpuppet/ffpuppet/core.py", line 598, in launch
    self._bootstrap_finish(init_soc, timeout=launch_timeout, url=location)
  File "/home/bcampen/checkouts/ffpuppet/ffpuppet/core.py", line 677, in _bootstrap_finish
    raise BrowserTimeoutError("Launching browser timed out (%ds)" % timeout)
ffpuppet.core.BrowserTimeoutError: Launching browser timed out (300s)

Comment 8

[Byron Campen [:bwc]]

Can you give me the stack that the binary in comment 3 gives you?

Comment 9

[Jason Kratzer [:jkratzer]]

==767==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000460fc at pc 0x7f06fcba472c bp 0x7f06a3cfd1a0 sp 0x7f06a3cfd198
READ of size 8 at 0x6110000460fc thread T8 (Socket Thread)
    #0 0x7f06fcba472b in nr_ice_peer_ctx_trickle_wait_cb /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c
    #1 0x7f06f0f096b5 in Notify /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp:132:3
    #2 0x7f06f0f096b5 in non-virtual thunk to mozilla::nrappkitTimerCallback::Notify(nsITimer*) /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp
    #3 0x7f06ee787f03 in nsTimerImpl::Fire(int) /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp:704:40
    #4 0x7f06ee744a1d in nsTimerEvent::Run() /builds/worker/workspace/build/src/xpcom/threads/TimerThread.cpp:297:11
    #5 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #6 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #7 0x7f06eea882cf in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #8 0x7f06eea8b264 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #9 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #10 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #11 0x7f06ef96b991 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #12 0x7f06ef86ca6c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #13 0x7f06ef86ca6c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #14 0x7f06ef86ca6c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #15 0x7f06ee753ef3 in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:502:11
    #16 0x7f0711b67008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #17 0x7f07117ac6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #18 0x7f071078a88e in clone /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x6110000460fc is located 188 bytes inside of 200-byte region [0x611000046040,0x611000046108)
freed by thread T8 (Socket Thread) here:
    #0 0x559a4f20b372 in __interceptor_free /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:68:3
    #1 0x7f06fcb7661a in nr_ice_component_destroy /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_component.c:177:5
    #2 0x7f06fcb8a0a7 in nr_ice_media_stream_destroy /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:108:7
    #3 0x7f06fcb89867 in nr_ice_peer_ctx_remove_pstream /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:259:10
    #4 0x7f06fcb89867 in nr_ice_remove_media_stream /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:829
    #5 0x7f06f0f3bf75 in mozilla::NrIceMediaStream::CloseStream(nr_ice_media_stream_**) /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:690:13
    #6 0x7f06f0f4fc3c in mozilla::NrIceMediaStream::Failed() /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:673:5
    #7 0x7f06f0f18d54 in mozilla::NrIceCtx::stream_failed(void*, nr_ice_media_stream_*) /builds/worker/workspace/build/src/media/mtransport/nricectx.cpp:383:8
    #8 0x7f06fcb82e8c in nr_ice_media_stream_component_failed /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:770:7
    #9 0x7f06fcba46ce in nr_ice_peer_ctx_trickle_wait_cb /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:354:9
    #10 0x7f06f0f096b5 in Notify /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp:132:3
    #11 0x7f06f0f096b5 in non-virtual thunk to mozilla::nrappkitTimerCallback::Notify(nsITimer*) /builds/worker/workspace/build/src/media/mtransport/nr_timer.cpp
    #12 0x7f06ee787f03 in nsTimerImpl::Fire(int) /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp:704:40
    #13 0x7f06ee744a1d in nsTimerEvent::Run() /builds/worker/workspace/build/src/xpcom/threads/TimerThread.cpp:297:11
    #14 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #15 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #16 0x7f06eea882cf in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #17 0x7f06eea8b264 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #18 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #19 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #20 0x7f06ef96b991 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #21 0x7f06ef86ca6c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #22 0x7f06ef86ca6c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #23 0x7f06ef86ca6c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #24 0x7f06ee753ef3 in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:502:11
    #25 0x7f0711b67008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #26 0x7f07117ac6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)

previously allocated by thread T8 (Socket Thread) here:
    #0 0x559a4f20b6b3 in malloc /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:88:3
    #1 0x7f06fcbdceb6 in r_malloc /builds/worker/workspace/build/src/media/mtransport/third_party/nrappkit/src/util/libekr/r_memory.c:73:16
    #2 0x7f06fcbdceb6 in r_calloc /builds/worker/workspace/build/src/media/mtransport/third_party/nrappkit/src/util/libekr/r_memory.c:98
    #3 0x7f06fcb88fde in nr_ice_component_create /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_component.c:117:15
    #4 0x7f06fcb88fde in nr_ice_media_stream_create /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:71
    #5 0x7f06fcb96304 in nr_ice_peer_ctx_parse_stream_attributes /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c:102:10
    #6 0x7f06f0f3ae13 in mozilla::NrIceMediaStream::ConnectToPeer(std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&) /builds/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:241:13
    #7 0x7f06f0d0cb57 in mozilla::PeerConnectionMedia::ActivateTransport_s(std::string const&, std::string const&, std::string const&, unsigned long, std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&) /builds/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionMedia.cpp:591:25
    #8 0x7f06f0d625be in apply<RefPtr<mozilla::PeerConnectionMedia>, void (mozilla::PeerConnectionMedia::*)(const std::basic_string<char> &, const std::basic_string<char> &, const std::basic_string<char> &, unsigned long, const std::basic_string<char> &, const std::basic_string<char> &, const std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > &), std::basic_string<char>, std::basic_string<char>, std::basic_string<char>, unsigned long, std::basic_string<char>, std::basic_string<char>, std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > , 0, 1, 2, 3, 4, 5, 6> /builds/worker/workspace/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:86:5
    #9 0x7f06f0d625be in mozilla::runnable_args_memfn<RefPtr<mozilla::PeerConnectionMedia>, void (mozilla::PeerConnectionMedia::*)(std::string const&, std::string const&, std::string const&, unsigned long, std::string const&, std::string const&, std::vector<std::string, std::allocator<std::string> > const&), std::string, std::string, std::string, unsigned long, std::string, std::string, std::vector<std::string, std::allocator<std::string> > >::Run() /builds/worker/workspace/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:156
    #10 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #11 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #12 0x7f06eea882cf in mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:1071:21
    #13 0x7f06eea8b264 in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp
    #14 0x7f06ee75b87f in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1231:14
    #15 0x7f06ee763e7d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:519:10
    #16 0x7f06ef96b991 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:334:20
    #17 0x7f06ef86ca6c in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:325:10
    #18 0x7f06ef86ca6c in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:318
    #19 0x7f06ef86ca6c in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:298
    #20 0x7f06ee753ef3 in nsThread::ThreadFunc(void*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:502:11
    #21 0x7f0711b67008 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #22 0x7f07117ac6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)

Thread T8 (Socket Thread) created by T0 (file:// Content) here:
    #0 0x559a4f1f473d in __interceptor_pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:204:3
    #1 0x7f0711b63d45 in _PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:433:14
    #2 0x7f0711b6392e in PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:518:12
    #3 0x7f06ee756e78 in nsThread::Init(nsTSubstring<char> const&) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:712:8
    #4 0x7f06ee762b9e in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp:485:22
    #5 0x7f06ee767457 in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, nsIRunnable*, unsigned int) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:143:45
    #6 0x7f06eea85471 in NS_NewNamedThread<14> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:75:10
    #7 0x7f06eea85471 in mozilla::net::nsSocketTransportService::Init() /builds/worker/workspace/build/src/netwerk/base/nsSocketTransportService2.cpp:640
    #8 0x7f06ef82dcd4 in nsSocketTransportServiceConstructor(nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/netwerk/build/nsNetModule.cpp:76:1
    #9 0x7f06ee6f2dde in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1161:19
    #10 0x7f06ee6e9de3 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1521:10
    #11 0x7f06ee6f91d5 in CallGetService /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:67:43
    #12 0x7f06ee6f91d5 in nsGetServiceByContractIDWithError::operator()(nsID const&, void**) const /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:292
    #13 0x7f06ee540153 in nsCOMPtr_base::assign_from_gs_contractid_with_error(nsGetServiceByContractIDWithError const&, nsID const&) /builds/worker/workspace/build/src/xpcom/base/nsCOMPtr.cpp:106:7
    #14 0x7f06ee9a9208 in operator= /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:704:5
    #15 0x7f06ee9a9208 in InitializeSocketTransportService /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:299
    #16 0x7f06ee9a9208 in mozilla::net::nsIOService::SetOffline(bool) /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:1074
    #17 0x7f06ee9a7657 in mozilla::net::nsIOService::Init() /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:263:5
    #18 0x7f06ee9aba13 in mozilla::net::nsIOService::GetInstance() /builds/worker/workspace/build/src/netwerk/base/nsIOService.cpp:360:13
    #19 0x7f06ef82d9b6 in nsIOServiceConstructor(nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/netwerk/build/nsNetModule.cpp:58:1
    #20 0x7f06ee6f2dde in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1161:19
    #21 0x7f06ee6e9de3 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1521:10
    #22 0x7f06f110858a in CallGetService<nsIIOService> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsServiceManagerUtils.h:90:10
    #23 0x7f06f110858a in nsScriptSecurityManager::Init() /builds/worker/workspace/build/src/caps/nsScriptSecurityManager.cpp:1394
    #24 0x7f06f110966c in nsScriptSecurityManager::InitStatics() /builds/worker/workspace/build/src/caps/nsScriptSecurityManager.cpp:1463:30
    #25 0x7f06f0a5093f in nsXPConnect::InitStatics() /builds/worker/workspace/build/src/js/xpconnect/src/nsXPConnect.cpp:140:5
    #26 0x7f06f09d3c80 in xpcModuleCtor() /builds/worker/workspace/build/src/js/xpconnect/src/XPCModule.cpp:13:5
    #27 0x7f06f9a55408 in Initialize() /builds/worker/workspace/build/src/layout/build/nsLayoutModule.cpp:235:8
    #28 0x7f06ee6f07e7 in nsComponentManagerImpl::KnownModule::Load() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:838:21
    #29 0x7f06ee6f2074 in nsFactoryEntry::GetFactory() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1859:19
    #30 0x7f06ee6f2d96 in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:1158:41
    #31 0x7f06ee6f8c0b in CallCreateInstance /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:149:38
    #32 0x7f06ee6f8c0b in nsCreateInstanceByContractID::operator()(nsID const&, void**) const /builds/worker/workspace/build/src/xpcom/components/nsComponentManagerUtils.cpp:197
    #33 0x7f06ee5405d0 in nsCOMPtr_base::assign_from_helper(nsCOMPtr_helper const&, nsID const&) /builds/worker/workspace/build/src/xpcom/base/nsCOMPtr.cpp:128:7
    #34 0x7f06ee6d9301 in nsCOMPtr /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:605:5
    #35 0x7f06ee6d9301 in LogMessageWithContext(mozilla::FileLocation&, unsigned int, char const*, ...) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:151
    #36 0x7f06ee6dfcc2 in nsComponentManagerImpl::ManifestContract(nsComponentManagerImpl::ManifestProcessingContext&, int, char* const*) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:769:5
    #37 0x7f06ee6dcba7 in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:695:7
    #38 0x7f06ee6ee296 in DoRegisterManifest /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:662:5
    #39 0x7f06ee6ee296 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:675
    #40 0x7f06ee6ee6ad in nsComponentManagerImpl::ManifestManifest(nsComponentManagerImpl::ManifestProcessingContext&, int, char* const*) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:684:3
    #41 0x7f06ee6dcba7 in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool) /builds/worker/workspace/build/src/xpcom/components/ManifestParser.cpp:695:7
    #42 0x7f06ee6ee296 in DoRegisterManifest /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:662:5
    #43 0x7f06ee6ee296 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:675
    #44 0x7f06ee6ecf42 in nsComponentManagerImpl::RereadChromeManifests(bool) /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:796:5
    #45 0x7f06ee6eb2ef in nsComponentManagerImpl::Init() /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp:414:5
    #46 0x7f06ee7cc25d in NS_InitXPCOM2 /builds/worker/workspace/build/src/xpcom/build/XPCOMInit.cpp:696:51
    #47 0x7f06fca3c2ac in XRE_InitEmbedding2(nsIFile*, nsIFile*, nsIDirectoryServiceProvider*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:195:8
    #48 0x7f06ef9794be in mozilla::ipc::ScopedXREEmbed::Start() /builds/worker/workspace/build/src/ipc/glue/ScopedXREEmbed.cpp
    #49 0x7f06f7c04b17 in mozilla::dom::ContentProcess::Init(int, char**) /builds/worker/workspace/build/src/dom/ipc/ContentProcess.cpp:296:13
    #50 0x7f06fca3d210 in XRE_InitChildProcess(int, char**, XREChildData const*) /builds/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:739:21
    #51 0x559a4f23bb91 in content_process_main /builds/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:50:30
    #52 0x559a4f23bb91 in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:287
    #53 0x7f071068ab96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_peer_ctx.c in nr_ice_peer_ctx_trickle_wait_cb
Shadow bytes around the buggy address:
  0x0c2280000bc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2280000bd0: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2280000be0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2280000bf0: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
  0x0c2280000c00: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c2280000c10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]
  0x0c2280000c20: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2280000c30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2280000c40: fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa
  0x0c2280000c50: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c2280000c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==767==ABORTING

Comment 10

[Jason Kratzer [:jkratzer]]

Also, I'm not sure exactly what would cause your issue with ffpuppet.  I use fuzzfetch to pull the ASAN build from taskcluster:
https://github.com/MozillaSecurity/fuzzfetch

`python -m fuzzfetch -n mc-asan -a`

Then start SimpleHTTPServer:
`python -m SimpleHTTPServer &`

Then ffpuppet:
`python -m ffpuppet -p prefs.js --xvfb -d -l log ./mc-asan/firefox -u http://localhost:8000/testcase.html`

Comment 11

[Jason Kratzer [:jkratzer]]

(In reply to Byron Campen [:bwc] from comment #7)
> I've tried using ffpuppet, but it does not seem to work. The closest I could
> get was by creating a shell script that called
> "mozilla-central/objdir-ff-asan/dist/bin/firefox
> http://localhost:8000/testcase.html" (because ffpuppet doesn't seem to
> handle arguments to the binary you hand it), and then calling that shell
> script like this:

Ah, I see.  You need to pass it the -u option in order to specify a URL.

Comment 12

[Byron Campen [:bwc]]

Here's another try build:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2757c7f9d5a3f87da2c420d377da9d925f649102

I will make another attempt to get ffpuppet to work, but I think my environment might be too different...

Comment 13

[Byron Campen [:bwc]]

After a little more fiddling, I've gotten the testcase to work on m-c, but the latest patch (the one pushed to try in comment 12) seems to fix the problem. I'll get a final patch up next.

Comment 14

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b7aa13ae14ad98dd626d75d72bbbc930c00b07c2

Comment 15

[Byron Campen [:bwc]]

Attachment: Bug 1493689: Defer close of old stream after ICE restart.

Comment 16

[Byron Campen [:bwc]]

Comment on attachment 9014084 [details]
Bug 1493689: Defer close of old stream after ICE restart.

[Security Approval Request]

How easily could an exploit be constructed based on the patch?: Probably not very hard, if you knew a-priori that there was a sec-bug being fixed.

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?: Dunno

Which older supported branches are affected by this flaw?: 

If not all supported branches, which bug introduced the flaw?: Bug 1493765

Do you have backports for the affected branches?: Yes

If not, how different, hard to create, and risky will they be?: 

How likely is this patch to cause regressions; how much testing does it need?: It is possible, but it is not a huge change.

Comment 17

[[Back July 11] Ryan VanderMeulen]

Comment on attachment 9014084 [details]
Bug 1493689: Defer close of old stream after ICE restart.

This doesn't need sec-approval if it only affects trunk. Go ahead and land when ready :)

Comment 18

[Jason Kratzer [:jkratzer]]

(In reply to Byron Campen [:bwc] from comment #12)
> Here's another try build:
> 
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=2757c7f9d5a3f87da2c420d377da9d925f649102
> 
> I will make another attempt to get ffpuppet to work, but I think my
> environment might be too different...

I can confirm that this no longer triggers using the try build in comment #12.

Comment 19

[[Back July 11] Ryan VanderMeulen]

https://hg.mozilla.org/integration/autoland/rev/6e67424a39d745df5adee264bd6b084cf1042adc

Comment 20

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/6e67424a39d7