Closed Bug 1418854 Opened 7 years ago Closed 7 years ago

Intermittent SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:263:7 in CloseWithStatusLocked

Categories

(Core :: Networking: Cache, defect, P1)

defect

Tracking

()

RESOLVED FIXED
mozilla59
Tracking Status
firefox-esr52 58+ fixed
firefox57 --- wontfix
firefox58 + fixed
firefox59 + fixed

People

(Reporter: aryx, Assigned: michal)

References

(Blocks 1 open bug)

Details

(Keywords: csectype-uaf, intermittent-failure, sec-high, Whiteboard: [OA][necko-triaged][adv-main58+][adv-esr52.6+][post-critsmash-triage])

Attachments

(2 files, 2 obsolete files)

FTR, we enabled RCWN yesterday in bug 1392841.

https://treeherder.mozilla.org/logviewer.html#?job_id=145974827&repo=mozilla-inbound

[task 2017-11-19T23:19:41.022Z] 23:19:41     INFO - TEST-START | accessible/tests/browser/e10s/browser_treeupdate_ariadialog.js
[task 2017-11-19T23:19:42.101Z] 23:19:42     INFO - GECKO(1400) | =================================================================
[task 2017-11-19T23:19:42.102Z] 23:19:42    ERROR - GECKO(1400) | ==1400==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b00040fa94 at pc 0x7fb3e6489961 bp 0x7ffce449a0d0 sp 0x7ffce449a0c8
[task 2017-11-19T23:19:42.103Z] 23:19:42     INFO - GECKO(1400) | READ of size 1 at 0x60b00040fa94 thread T0
[task 2017-11-19T23:19:42.616Z] 23:19:42     INFO - GECKO(1400) |     #0 0x7fb3e6489960 in CloseWithStatusLocked /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:263:7
[task 2017-11-19T23:19:42.619Z] 23:19:42     INFO - GECKO(1400) |     #1 0x7fb3e6489960 in mozilla::net::CacheFileInputStream::CloseWithStatus(nsresult) /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:254
[task 2017-11-19T23:19:42.636Z] 23:19:42     INFO - GECKO(1400) |     #2 0x7fb3e66944e9 in Close /builds/worker/workspace/build/src/netwerk/base/AutoClose.h:63:13
[task 2017-11-19T23:19:42.637Z] 23:19:42     INFO - GECKO(1400) |     #3 0x7fb3e66944e9 in CloseAndRelease /builds/worker/workspace/build/src/netwerk/base/AutoClose.h:50
[task 2017-11-19T23:19:42.637Z] 23:19:42     INFO - GECKO(1400) |     #4 0x7fb3e66944e9 in mozilla::net::nsHttpChannel::CloseCacheEntry(bool) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:5062
[task 2017-11-19T23:19:42.638Z] 23:19:42     INFO - GECKO(1400) |     #5 0x7fb3e66d5505 in mozilla::net::nsHttpChannel::OnStopRequest(nsIRequest*, nsISupports*, nsresult) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:7443:5
[task 2017-11-19T23:19:42.639Z] 23:19:42     INFO - GECKO(1400) |     #6 0x7fb3e5de4946 in nsInputStreamPump::OnStateStop() /builds/worker/workspace/build/src/netwerk/base/nsInputStreamPump.cpp:704:20
[task 2017-11-19T23:19:42.641Z] 23:19:42     INFO - GECKO(1400) |     #7 0x7fb3e5de2d16 in nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) /builds/worker/workspace/build/src/netwerk/base/nsInputStreamPump.cpp:428:25
[task 2017-11-19T23:19:42.641Z] 23:19:42     INFO - GECKO(1400) |     #8 0x7fb3e5bd8b7d in nsInputStreamReadyEvent::Run() /builds/worker/workspace/build/src/xpcom/io/nsStreamUtils.cpp:97:20
[task 2017-11-19T23:19:42.642Z] 23:19:42     INFO - GECKO(1400) |     #9 0x7fb3e5c411c6 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1037:14
[task 2017-11-19T23:19:42.643Z] 23:19:42     INFO - GECKO(1400) |     #10 0x7fb3e5c5bb48 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:513:10
[task 2017-11-19T23:19:42.659Z] 23:19:42     INFO - GECKO(1400) |     #11 0x7fb3e6a36f51 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:97:21
[task 2017-11-19T23:19:42.660Z] 23:19:42     INFO - GECKO(1400) |     #12 0x7fb3e69976cb in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:326:10
[task 2017-11-19T23:19:42.661Z] 23:19:42     INFO - GECKO(1400) |     #13 0x7fb3e69976cb in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:319
[task 2017-11-19T23:19:42.661Z] 23:19:42     INFO - GECKO(1400) |     #14 0x7fb3e69976cb in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:299
[task 2017-11-19T23:19:42.663Z] 23:19:42     INFO - GECKO(1400) |     #15 0x7fb3ec49d6ef in nsBaseAppShell::Run() /builds/worker/workspace/build/src/widget/nsBaseAppShell.cpp:159:27
[task 2017-11-19T23:19:42.663Z] 23:19:42     INFO - GECKO(1400) |     #16 0x7fb3f05d4f51 in nsAppStartup::Run() /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:288:30
[task 2017-11-19T23:19:42.664Z] 23:19:42     INFO - GECKO(1400) |     #17 0x7fb3f07cd220 in XREMain::XRE_mainRun() /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4685:22
[task 2017-11-19T23:19:42.665Z] 23:19:42     INFO - GECKO(1400) |     #18 0x7fb3f07cedf5 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4847:8
[task 2017-11-19T23:19:42.666Z] 23:19:42     INFO - GECKO(1400) |     #19 0x7fb3f07d01a6 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4942:21
[task 2017-11-19T23:19:42.666Z] 23:19:42     INFO - GECKO(1400) |     #20 0x4ebd2c in do_main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:231:22
[task 2017-11-19T23:19:42.667Z] 23:19:42     INFO - GECKO(1400) |     #21 0x4ebd2c in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:304
[task 2017-11-19T23:19:42.729Z] 23:19:42     INFO - GECKO(1400) |     #22 0x7fb4041cd82f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291
[task 2017-11-19T23:19:42.731Z] 23:19:42     INFO - GECKO(1400) |     #23 0x41d408 in _start (/builds/worker/workspace/build/application/firefox/firefox+0x41d408)
[task 2017-11-19T23:19:42.731Z] 23:19:42     INFO - GECKO(1400) | 0x60b00040fa94 is located 68 bytes inside of 112-byte region [0x60b00040fa50,0x60b00040fac0)
[task 2017-11-19T23:19:42.732Z] 23:19:42     INFO - GECKO(1400) | freed by thread T17 (Cache2 I/O) here:
[task 2017-11-19T23:19:42.735Z] 23:19:42     INFO - GECKO(1400) |     #0 0x4bb93b in __interceptor_free /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:47:3
[task 2017-11-19T23:19:42.736Z] 23:19:42     INFO - GECKO(1400) |     #1 0x7fb3e64866e0 in mozilla::net::CacheFileInputStream::Release() /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:26:5
[task 2017-11-19T23:19:42.739Z] 23:19:42     INFO - GECKO(1400) |     #2 0x7fb3e64b8d6c in Release /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:41:11
[task 2017-11-19T23:19:42.741Z] 23:19:42     INFO - GECKO(1400) |     #3 0x7fb3e64b8d6c in Release /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:398
[task 2017-11-19T23:19:42.742Z] 23:19:42     INFO - GECKO(1400) |     #4 0x7fb3e64b8d6c in ~RefPtr /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:79
[task 2017-11-19T23:19:42.743Z] 23:19:42     INFO - GECKO(1400) |     #5 0x7fb3e64b8d6c in Destruct /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:562
[task 2017-11-19T23:19:42.744Z] 23:19:42     INFO - GECKO(1400) |     #6 0x7fb3e64b8d6c in DestructRange /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:2026
[task 2017-11-19T23:19:42.744Z] 23:19:42     INFO - GECKO(1400) |     #7 0x7fb3e64b8d6c in nsTArray_Impl<RefPtr<nsISupports>, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned long, unsigned long) /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:2079
[task 2017-11-19T23:19:42.745Z] 23:19:42     INFO - GECKO(1400) |     #8 0x7fb3e6446746 in Clear /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:1752:18
[task 2017-11-19T23:19:42.746Z] 23:19:42     INFO - GECKO(1400) |     #9 0x7fb3e6446746 in ~nsTArray_Impl /builds/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:885
[task 2017-11-19T23:19:42.747Z] 23:19:42     INFO - GECKO(1400) |     #10 0x7fb3e6446746 in mozilla::net::CacheFile::Unlock() /builds/worker/workspace/build/src/netwerk/cache2/CacheFile.cpp:1414
[task 2017-11-19T23:19:42.753Z] 23:19:42     INFO - GECKO(1400) |     #11 0x7fb3e644c4da in ~CacheFileAutoLock /builds/worker/workspace/build/src/netwerk/cache2/CacheFile.h:253:14
[task 2017-11-19T23:19:42.756Z] 23:19:42     INFO - GECKO(1400) |     #12 0x7fb3e644c4da in mozilla::net::CacheFile::RemoveInput(mozilla::net::CacheFileInputStream*, nsresult) /builds/worker/workspace/build/src/netwerk/cache2/CacheFile.cpp:2142
[task 2017-11-19T23:19:42.757Z] 23:19:42     INFO - GECKO(1400) |     #13 0x7fb3e648671a in mozilla::net::CacheFileInputStream::Release() /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:31:12
[task 2017-11-19T23:19:42.758Z] 23:19:42     INFO - GECKO(1400) |     #14 0x7fb3e66bf3a4 in ~nsCOMPtr_base /builds/worker/workspace/build/src/obj-firefox/dist/include/nsCOMPtr.h:313:7
[task 2017-11-19T23:19:42.759Z] 23:19:42     INFO - GECKO(1400) |     #15 0x7fb3e66bf3a4 in mozilla::net::nsHttpChannel::OpenCacheInputStream(nsICacheEntry*, bool, bool) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:4922
[task 2017-11-19T23:19:42.760Z] 23:19:42     INFO - GECKO(1400) |     #16 0x7fb3e66bd835 in mozilla::net::nsHttpChannel::OnCacheEntryCheck(nsICacheEntry*, nsIApplicationCache*, unsigned int*) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:4312:14
[task 2017-11-19T23:19:42.760Z] 23:19:42     INFO - GECKO(1400) |     #17 0x7fb3e642e7b5 in mozilla::net::CacheEntry::InvokeCallback(mozilla::net::CacheEntry::Callback&) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:758:46
[task 2017-11-19T23:19:42.761Z] 23:19:42     INFO - GECKO(1400) |     #18 0x7fb3e642ec74 in mozilla::net::CacheEntry::InvokeCallback(mozilla::net::CacheEntry::Callback&) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:817:14
[task 2017-11-19T23:19:42.762Z] 23:19:42     INFO - GECKO(1400) |     #19 0x7fb3e642da59 in mozilla::net::CacheEntry::InvokeCallbacks(bool) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:687:30
[task 2017-11-19T23:19:42.763Z] 23:19:42     INFO - GECKO(1400) |     #20 0x7fb3e642976f in mozilla::net::CacheEntry::InvokeCallbacks() /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:627:7
[task 2017-11-19T23:19:42.764Z] 23:19:42     INFO - GECKO(1400) |     #21 0x7fb3e642a4bf in mozilla::net::CacheEntry::BackgroundOp(unsigned int, bool) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:1900:5
[task 2017-11-19T23:19:42.765Z] 23:19:42     INFO - GECKO(1400) |     #22 0x7fb3e643895b in Run /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:1614:3
[task 2017-11-19T23:19:42.766Z] 23:19:42     INFO - GECKO(1400) |     #23 0x7fb3e643895b in non-virtual thunk to mozilla::net::CacheEntry::Run() /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:1608
[task 2017-11-19T23:19:42.767Z] 23:19:42     INFO - GECKO(1400) |     #24 0x7fb3e6497451 in mozilla::net::CacheIOThread::LoopOneLevel(unsigned int) /builds/worker/workspace/build/src/netwerk/cache2/CacheIOThread.cpp:570:22
[task 2017-11-19T23:19:42.767Z] 23:19:42     INFO - GECKO(1400) |     #25 0x7fb3e6496cf8 in mozilla::net::CacheIOThread::ThreadFunc() /builds/worker/workspace/build/src/netwerk/cache2/CacheIOThread.cpp:508:9
[task 2017-11-19T23:19:42.769Z] 23:19:42     INFO - GECKO(1400) |     #26 0x7fb3e6495e73 in mozilla::net::CacheIOThread::ThreadFunc(void*) /builds/worker/workspace/build/src/netwerk/cache2/CacheIOThread.cpp:450:11
[task 2017-11-19T23:19:42.770Z] 23:19:42     INFO - GECKO(1400) |     #27 0x7fb400ed7513 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:216:5
[task 2017-11-19T23:19:42.771Z] 23:19:42     INFO - GECKO(1400) |     #28 0x7fb40522b6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
[task 2017-11-19T23:19:42.772Z] 23:19:42     INFO - GECKO(1400) | previously allocated by thread T0 here:
[task 2017-11-19T23:19:42.774Z] 23:19:42     INFO - GECKO(1400) |     #0 0x4bbc8c in malloc /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64:3
[task 2017-11-19T23:19:42.776Z] 23:19:42     INFO - GECKO(1400) |     #1 0x4ed09d in moz_xmalloc /builds/worker/workspace/build/src/memory/mozalloc/mozalloc.cpp:84:17
[task 2017-11-19T23:19:42.779Z] 23:19:42     INFO - GECKO(1400) |     #2 0x7fb3e643343d in operator new /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/mozalloc.h:206:12
[task 2017-11-19T23:19:42.780Z] 23:19:42     INFO - GECKO(1400) |     #3 0x7fb3e643343d in mozilla::net::CacheFile::OpenInputStream(nsICacheEntry*, nsIInputStream**) /builds/worker/workspace/build/src/netwerk/cache2/CacheFile.cpp:784
[task 2017-11-19T23:19:42.782Z] 23:19:42     INFO - GECKO(1400) |     #4 0x7fb3e643211a in mozilla::net::CacheEntry::OpenInputStreamInternal(long, char const*, nsIInputStream**) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:1197:17
[task 2017-11-19T23:19:42.783Z] 23:19:42     INFO - GECKO(1400) |     #5 0x7fb3e66beddd in mozilla::net::nsHttpChannel::OpenCacheInputStream(nsICacheEntry*, bool, bool) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:4858:26
[task 2017-11-19T23:19:42.790Z] 23:19:42     INFO - GECKO(1400) |     #6 0x7fb3e66bd835 in mozilla::net::nsHttpChannel::OnCacheEntryCheck(nsICacheEntry*, nsIApplicationCache*, unsigned int*) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:4312:14
[task 2017-11-19T23:19:42.792Z] 23:19:42     INFO - GECKO(1400) |     #7 0x7fb3e642e7b5 in mozilla::net::CacheEntry::InvokeCallback(mozilla::net::CacheEntry::Callback&) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:758:46
[task 2017-11-19T23:19:42.793Z] 23:19:42     INFO - GECKO(1400) |     #8 0x7fb3e642da59 in mozilla::net::CacheEntry::InvokeCallbacks(bool) /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:687:30
[task 2017-11-19T23:19:42.796Z] 23:19:42     INFO - GECKO(1400) |     #9 0x7fb3e642976f in mozilla::net::CacheEntry::InvokeCallbacks() /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:627:7
[task 2017-11-19T23:19:42.799Z] 23:19:42     INFO - GECKO(1400) |     #10 0x7fb3e64377db in mozilla::net::CacheEntry::MetaDataReady() /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.cpp:1475:3
[task 2017-11-19T23:19:42.799Z] 23:19:42     INFO - GECKO(1400) |     #11 0x7fb3e64b778f in mozilla::net::CacheEntryHandle::MetaDataReady() /builds/worker/workspace/build/src/netwerk/cache2/CacheEntry.h:400:3
[task 2017-11-19T23:19:42.801Z] 23:19:42     INFO - GECKO(1400) |     #12 0x7fb3e66c2db7 in mozilla::net::DoAddCacheEntryHeaders(mozilla::net::nsHttpChannel*, nsICacheEntry*, mozilla::net::nsHttpRequestHead*, mozilla::net::nsHttpResponseHead*, nsISupports*) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:5341:17
[task 2017-11-19T23:19:42.802Z] 23:19:42     INFO - GECKO(1400) |     #13 0x7fb3e66b27f6 in AddCacheEntryHeaders /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:5349:12
[task 2017-11-19T23:19:42.802Z] 23:19:42     INFO - GECKO(1400) |     #14 0x7fb3e66b27f6 in mozilla::net::nsHttpChannel::InitCacheEntry() /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:5181
[task 2017-11-19T23:19:42.803Z] 23:19:42     INFO - GECKO(1400) |     #15 0x7fb3e66b2d71 in mozilla::net::nsHttpChannel::ContinueProcessNormal(nsresult) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:2787:14
[task 2017-11-19T23:19:42.804Z] 23:19:42     INFO - GECKO(1400) |     #16 0x7fb3e66af2ef in mozilla::net::nsHttpChannel::ProcessNormal() /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:2752:12
[task 2017-11-19T23:19:42.805Z] 23:19:42     INFO - GECKO(1400) |     #17 0x7fb3e66adbf5 in mozilla::net::nsHttpChannel::ContinueProcessResponse2(nsresult) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:2631:14
[task 2017-11-19T23:19:42.806Z] 23:19:42     INFO - GECKO(1400) |     #18 0x7fb3e66ad28c in mozilla::net::nsHttpChannel::ContinueProcessResponse1() /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:2460:12
[task 2017-11-19T23:19:42.807Z] 23:19:42     INFO - GECKO(1400) |     #19 0x7fb3e66ac646 in mozilla::net::nsHttpChannel::ProcessResponse() /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:2367:12
[task 2017-11-19T23:19:42.808Z] 23:19:42     INFO - GECKO(1400) |     #20 0x7fb3e66d1ea6 in mozilla::net::nsHttpChannel::OnStartRequest(nsIRequest*, nsISupports*) /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp:6988:20
[task 2017-11-19T23:19:42.809Z] 23:19:42     INFO - GECKO(1400) |     #21 0x7fb3e5de35f2 in nsInputStreamPump::OnStateStart() /builds/worker/workspace/build/src/netwerk/base/nsInputStreamPump.cpp:518:25
[task 2017-11-19T23:19:42.810Z] 23:19:42     INFO - GECKO(1400) |     #22 0x7fb3e5de2c7e in nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) /builds/worker/workspace/build/src/netwerk/base/nsInputStreamPump.cpp:421:25
[task 2017-11-19T23:19:42.811Z] 23:19:42     INFO - GECKO(1400) |     #23 0x7fb3e5bd8b7d in nsInputStreamReadyEvent::Run() /builds/worker/workspace/build/src/xpcom/io/nsStreamUtils.cpp:97:20
[task 2017-11-19T23:19:42.814Z] 23:19:42     INFO - GECKO(1400) |     #24 0x7fb3e5c411c6 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1037:14
[task 2017-11-19T23:19:42.815Z] 23:19:42     INFO - GECKO(1400) |     #25 0x7fb3e5c5bb48 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:513:10
[task 2017-11-19T23:19:42.816Z] 23:19:42     INFO - GECKO(1400) |     #26 0x7fb3e6a36f51 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:97:21
[task 2017-11-19T23:19:42.817Z] 23:19:42     INFO - GECKO(1400) |     #27 0x7fb3e69976cb in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:326:10
[task 2017-11-19T23:19:42.818Z] 23:19:42     INFO - GECKO(1400) |     #28 0x7fb3e69976cb in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:319
[task 2017-11-19T23:19:42.819Z] 23:19:42     INFO - GECKO(1400) |     #29 0x7fb3e69976cb in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:299
[task 2017-11-19T23:19:42.820Z] 23:19:42     INFO - GECKO(1400) |     #30 0x7fb3ec49d6ef in nsBaseAppShell::Run() /builds/worker/workspace/build/src/widget/nsBaseAppShell.cpp:159:27
[task 2017-11-19T23:19:42.821Z] 23:19:42     INFO - GECKO(1400) |     #31 0x7fb3f05d4f51 in nsAppStartup::Run() /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:288:30
[task 2017-11-19T23:19:42.821Z] 23:19:42     INFO - GECKO(1400) |     #32 0x7fb3f07cd220 in XREMain::XRE_mainRun() /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4685:22
[task 2017-11-19T23:19:42.823Z] 23:19:42     INFO - GECKO(1400) |     #33 0x7fb3f07cedf5 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4847:8
[task 2017-11-19T23:19:42.837Z] 23:19:42     INFO - GECKO(1400) | Thread T17 (Cache2 I/O) created by T0 here:
[task 2017-11-19T23:19:42.840Z] 23:19:42     INFO - GECKO(1400) |     #0 0x4a4066 in __interceptor_pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:245:3
[task 2017-11-19T23:19:42.841Z] 23:19:42     INFO - GECKO(1400) |     #1 0x7fb400ed42b9 in _PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:457:14
[task 2017-11-19T23:19:42.844Z] 23:19:42     INFO - GECKO(1400) |     #2 0x7fb400ed3ece in PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:548:12
[task 2017-11-19T23:19:42.845Z] 23:19:42     INFO - GECKO(1400) |     #3 0x7fb3e6467e52 in mozilla::net::CacheIOThread::Init() /builds/worker/workspace/build/src/netwerk/cache2/CacheIOThread.cpp:273:13
[task 2017-11-19T23:19:42.846Z] 23:19:42     INFO - GECKO(1400) |     #4 0x7fb3e6467bf3 in mozilla::net::CacheFileIOManager::InitInternal() /builds/worker/workspace/build/src/netwerk/cache2/CacheFileIOManager.cpp:1242:19
[task 2017-11-19T23:19:42.847Z] 23:19:42     INFO - GECKO(1400) |     #5 0x7fb3e64676d9 in mozilla::net::CacheFileIOManager::Init() /builds/worker/workspace/build/src/netwerk/cache2/CacheFileIOManager.cpp:1228:24
[task 2017-11-19T23:19:42.848Z] 23:19:42     INFO - GECKO(1400) |     #6 0x7fb3e64b6132 in mozilla::net::CacheObserver::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/netwerk/cache2/CacheObserver.cpp:491:5
[task 2017-11-19T23:19:42.849Z] 23:19:42     INFO - GECKO(1400) |     #7 0x7fb3e5b4702c in nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverList.cpp:112:19
[task 2017-11-19T23:19:42.852Z] 23:19:42     INFO - GECKO(1400) |     #8 0x7fb3e5b4a9f8 in nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverService.cpp:296:19
[task 2017-11-19T23:19:42.852Z] 23:19:42     INFO - GECKO(1400) |     #9 0x7fb3f07ed03e in nsXREDirProvider::DoStartup() /builds/worker/workspace/build/src/toolkit/xre/nsXREDirProvider.cpp:1010:13
[task 2017-11-19T23:19:42.853Z] 23:19:42     INFO - GECKO(1400) |     #10 0x7fb3f07cc6f8 in XREMain::XRE_mainRun() /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4514:16
[task 2017-11-19T23:19:42.853Z] 23:19:42     INFO - GECKO(1400) |     #11 0x7fb3f07cedf5 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4847:8
[task 2017-11-19T23:19:42.854Z] 23:19:42     INFO - GECKO(1400) |     #12 0x7fb3f07d01a6 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4942:21
[task 2017-11-19T23:19:42.854Z] 23:19:42     INFO - GECKO(1400) |     #13 0x4ebd2c in do_main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:231:22
[task 2017-11-19T23:19:42.854Z] 23:19:42     INFO - GECKO(1400) |     #14 0x4ebd2c in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:304
[task 2017-11-19T23:19:42.854Z] 23:19:42     INFO - GECKO(1400) |     #15 0x7fb4041cd82f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291
[task 2017-11-19T23:19:42.855Z] 23:19:42     INFO - GECKO(1400) | SUMMARY: AddressSanitizer: heap-use-after-free /builds/worker/workspace/build/src/netwerk/cache2/CacheFileInputStream.cpp:263:7 in CloseWithStatusLocked
[task 2017-11-19T23:19:42.855Z] 23:19:42     INFO - GECKO(1400) | Shadow bytes around the buggy address:
[task 2017-11-19T23:19:42.856Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f00: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
[task 2017-11-19T23:19:42.857Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f10: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fd fd
[task 2017-11-19T23:19:42.858Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f20: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa
[task 2017-11-19T23:19:42.858Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f30: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
[task 2017-11-19T23:19:42.859Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f40: fd fd fa fa fa fa fa fa fa fa fd fd fd fd fd fd
[task 2017-11-19T23:19:42.860Z] 23:19:42     INFO - GECKO(1400) | =>0x0c1680079f50: fd fd[fd]fd fd fd fd fd fa fa fa fa fa fa fa fa
[task 2017-11-19T23:19:42.860Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa
[task 2017-11-19T23:19:42.861Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f70: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
[task 2017-11-19T23:19:42.861Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f80: fd fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd
[task 2017-11-19T23:19:42.862Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079f90: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
[task 2017-11-19T23:19:42.862Z] 23:19:42     INFO - GECKO(1400) |   0x0c1680079fa0: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa
[task 2017-11-19T23:19:42.863Z] 23:19:42     INFO - GECKO(1400) | Shadow byte legend (one shadow byte represents 8 application bytes):
[task 2017-11-19T23:19:42.863Z] 23:19:42     INFO - GECKO(1400) |   Addressable:           00
[task 2017-11-19T23:19:42.864Z] 23:19:42     INFO - GECKO(1400) |   Partially addressable: 01 02 03 04 05 06 07
[task 2017-11-19T23:19:42.869Z] 23:19:42     INFO - GECKO(1400) |   Heap left redzone:       fa
[task 2017-11-19T23:19:42.869Z] 23:19:42     INFO - GECKO(1400) |   Heap right redzone:      fb
[task 2017-11-19T23:19:42.869Z] 23:19:42     INFO - GECKO(1400) |   Freed heap region:       fd
[task 2017-11-19T23:19:42.869Z] 23:19:42     INFO - GECKO(1400) |   Stack left redzone:      f1
[task 2017-11-19T23:19:42.869Z] 23:19:42     INFO - GECKO(1400) |   Stack mid redzone:       f2
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Stack right redzone:     f3
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Stack partial redzone:   f4
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Stack after return:      f5
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Stack use after scope:   f8
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Global redzone:          f9
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Global init order:       f6
[task 2017-11-19T23:19:42.870Z] 23:19:42     INFO - GECKO(1400) |   Poisoned by user:        f7
[task 2017-11-19T23:19:42.871Z] 23:19:42     INFO - GECKO(1400) |   Container overflow:      fc
[task 2017-11-19T23:19:42.871Z] 23:19:42     INFO - GECKO(1400) |   Array cookie:            ac
[task 2017-11-19T23:19:42.871Z] 23:19:42     INFO - GECKO(1400) |   Intra object redzone:    bb
[task 2017-11-19T23:19:42.872Z] 23:19:42     INFO - GECKO(1400) |   ASan internal:           fe
[task 2017-11-19T23:19:42.873Z] 23:19:42     INFO - GECKO(1400) |   Left alloca redzone:     ca
[task 2017-11-19T23:19:42.873Z] 23:19:42     INFO - GECKO(1400) |   Right alloca redzone:    cb
[task 2017-11-19T23:19:42.874Z] 23:19:42     INFO - GECKO(1400) | ==1400==ABORTING
Michal, this is your code, can you take a look please?
Assignee: nobody → michal.novotny
Severity: normal → critical
Priority: -- → P1
Whiteboard: [necko-triaged]
It seems to be the same problem as in bug 1313934. I'll try to reproduce it with refcount logging and some sanity checks.
Attached patch fix (obsolete) — Splinter Review
I was able to reproduce it on try and the problem is in AutoClose helper class. On the main thread imgRequest::Cancel() cancels the http channel and mCacheInputStream.CloseAndRelease() is called. At the same time, mCacheInputStream.takeOver() closes the same input stream in nsHttpChannel::OpenCacheInputStream() on the cache IO thread.

2017-11-22 13:21:33.400927 UTC - [Main Thread]: D/nsHttp nsHttpChannel::Cancel [this=0xc0e43800 status=804b0002]
2017-11-22 13:21:33.424896 UTC - [Cache2 I/O]: D/cache2 CacheEntry CALLBACKS (invoke) [this=0xda117c90]
2017-11-22 13:21:33.424900 UTC - [Cache2 I/O]: D/cache2 CacheEntry::InvokeCallbacks BEGIN [this=0xda117c90]
2017-11-22 13:21:33.424909 UTC - [Cache2 I/O]: D/cache2 CacheEntry::InvokeCallback [this=0xda117c90, state=READY, cb=0xc0e43c34]
2017-11-22 13:21:33.424929 UTC - [Cache2 I/O]: D/cache2 CacheEntry::InvokeCallback [this=0xda117c90, state=READY, cb=0xc0e43c34]
2017-11-22 13:21:33.424935 UTC - [Cache2 I/O]: D/nsHttp nsHttpChannel::OnCacheEntryCheck enter [channel=0xc0e43800 entry=0xda117c90]
2017-11-22 13:21:33.435915 UTC - [Cache2 I/O]: D/cache2 CacheEntry::GetDataSize [this=0xda117c90]
2017-11-22 13:21:33.435930 UTC - [Cache2 I/O]: D/cache2 CacheEntry::GetIsForcedValid [this=0xda117c90, IsForcedValid=0]
2017-11-22 13:21:33.435954 UTC - [Cache2 I/O]: D/cache2 CacheEntry::OpenInputStream [this=0xda117c90]
2017-11-22 13:21:33.435957 UTC - [Cache2 I/O]: D/cache2 CacheEntry::OpenInputStreamInternal [this=0xda117c90]
2017-11-22 13:21:33.435961 UTC - [Cache2 I/O]: D/cache2 New CacheEntryHandle 0xc0e5fd80 for entry 0xda117c90
2017-11-22 13:21:33.436021 UTC - [Cache2 I/O]: D/nsHttp Opened cache input stream without buffering [channel=0xc0e43800, mCacheEntry=0xda117c90, stream=0xc0ef24c0]
2017-11-22 13:21:33.436025 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::Close() [this=0xc0e5a150]
2017-11-22 13:21:33.436028 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::CloseWithStatus() [this=0xc0e5a150, aStatus=0x00000000]
2017-11-22 13:21:33.436032 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::CloseWithStatusLocked() [this=0xc0e5a150, aStatus=0x00000000]
2017-11-22 13:21:33.436036 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::MaybeNotifyListener() [this=0xc0e5a150, mCallback=(nil), mClosed=1, mStatus=0x80470002, mChunk=(nil), mListeningForChunk=-1, mWaitingForUpdate=0]
2017-11-22 13:21:33.451552 UTC - [Cache2 I/O]: D/cache2 CacheEntry::OnHandleClosed [this=0xda117c90, state=READY, handle=0xc0e5fc90]
2017-11-22 13:21:33.451565 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::Release() [this=0xc0e5a150]
2017-11-22 13:21:33.451570 UTC - [Cache2 I/O]: D/cache2 CacheFile::RemoveInput() [this=0xc4761400, input=0xc0e5a150, status=0x80470002]
2017-11-22 13:21:33.451580 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::Release() [this=0xc0e5a150]
2017-11-22 13:21:33.451584 UTC - [Cache2 I/O]: D/cache2 CacheFileInputStream::~CacheFileInputStream() [this=0xc0e5a150]
2017-11-22 13:21:33.451591 UTC - [Cache2 I/O]: D/nsHttp nsHTTPChannel::OnCacheEntryCheck exit [this=0xc0e43800 doValidation=0 result=0]
2017-11-22 13:21:33.451600 UTC - [Cache2 I/O]: D/cache2 CacheEntry::InvokeAvailableCallback [this=0xda117c90, state=READY, cb=0xc0e43c34, r/o=0, n/w=0]
2017-11-22 13:21:33.451614 UTC - [Cache2 I/O]: D/cache2 CacheEntry::InvokeCallbacks END [this=0xda117c90]
2017-11-22 13:21:33.425009 UTC - [Main Thread]: D/cache2 CacheFileInputStream::Close() [this=0xc0e5a150]
Attachment #8931541 - Flags: review?(honzab.moz)
Comment on attachment 8931541 [details] [diff] [review]
fix

Review of attachment 8931541 [details] [diff] [review]:
-----------------------------------------------------------------

::: netwerk/base/AutoClose.h
@@ +50,5 @@
> +    nsCOMPtr<T> ptr;
> +    ptr.swap(mPtr);
> +    if (ptr) {
> +      ptr->Close();
> +    }

This (swap) is not atomic at all, unless I'm missing something.  I am afraid you need to lock over this method, either from outside or inside this class or at least manipulate with mPtr member fully atomically (locked).

To be honest, I'd love to get rid of this helper class.  It has been introduced as an optimization when we were still using the old cache to simplify some manipulation with the input stream.  But I think these days it's kinda obsolete.
Attachment #8931541 - Flags: review?(honzab.moz) → review-
Attached patch patch v2 (obsolete) — Splinter Review
(In reply to Honza Bambas (:mayhemer) from comment #5)
> To be honest, I'd love to get rid of this helper class.  It has been
> introduced as an optimization when we were still using the old cache to
> simplify some manipulation with the input stream.  But I think these days
> it's kinda obsolete.

IMO the functionality is still needed. I kept the helper class and used the lock. I also removed unused functions.
Attachment #8931541 - Attachment is obsolete: true
Attachment #8932140 - Flags: review?(honzab.moz)
Assuming this is sec-high since it's UAF, we'll want sec-approval on this before it lands.
Track 58+/59+ as sec-high.
Comment on attachment 8932140 [details] [diff] [review]
patch v2

Review of attachment 8932140 [details] [diff] [review]:
-----------------------------------------------------------------

::: netwerk/base/AutoClose.h
@@ +18,5 @@
>  template <typename T>
>  class AutoClose
>  {
>  public:
> +  AutoClose() : mRecursiveMutex("AutoClose.mRecursiveMutex") { }

could it be prefixed with "net::" ?

@@ +25,5 @@
>    }
>  
>    explicit operator bool() const
>    {
>      return mPtr;

not sure how this is used, but the lock might be here for consistency (not that it would sync any logic based on the result of this operator, tho)

@@ +53,5 @@
>    {
> +    nsCOMPtr<T> ptr;
> +    ptr.swap(mPtr);
> +    if (ptr) {
> +      ptr->Close();

I'm not very happy that Close() is called under the lock, regardless it's recursive.  you never know what all that Close() is going to do, now or in two years.  unless there is a strong reason, it would be good move this out of the lock (forget mPtr to be closed outside the lock).. that would mostly remove CloseAndReleaseLocked method.
Attachment #8932140 - Flags: review?(honzab.moz) → review-
Attached patch patch v3Splinter Review
Attachment #8932140 - Attachment is obsolete: true
Attachment #8934326 - Flags: review?(honzab.moz)
Comment on attachment 8934326 [details] [diff] [review]
patch v3

Review of attachment 8934326 [details] [diff] [review]:
-----------------------------------------------------------------

thanks!
Attachment #8934326 - Flags: review?(honzab.moz) → review+
Comment on attachment 8934326 [details] [diff] [review]
patch v3

[Security approval request comment]
How easily could an exploit be constructed based on the patch?
I guess it's very hard or impossible, but I'm not sure.

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?
Comment mentions race condition.

Which older supported branches are affected by this flaw?
The bug is present IMO since we enabled cache2, i.e. since release 32.

If not all supported branches, which bug introduced the flaw?

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be?
Should be easy, I guess the patch should apply cleanly on beta.

How likely is this patch to cause regressions; how much testing does it need?
Existing automated test should reveal regression.
Attachment #8934326 - Flags: sec-approval?
sec-approval+ for trunk.
Please nominate patches for beta (58) and ERSR52 once it has landed and passed tests on trunk.
Attachment #8934326 - Flags: sec-approval? → sec-approval+
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/3e425d2e71df
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
Please request Beta approval on this patch when you get a chance, I've verified that it grafts cleanly. It'll need a rebased patch for ESR52 before that uplift request, however.
Flags: needinfo?(michal.novotny)
Attached patch patch for esr52Splinter Review
Flags: needinfo?(michal.novotny)
Comment on attachment 8934326 [details] [diff] [review]
patch v3

Approval Request Comment
[Feature/Bug causing the regression]: probably 764171
[User impact if declined]: use after free
[Is this code covered by automated tests?]: yes, this code is triggered by any http activity
[Has the fix been verified in Nightly?]: it landed on m-c on 2017-12-07
[Needs manual test from QE? If yes, steps to reproduce]: no, it's not easily reproducible
[List of other uplifts needed for the feature/fix]: none
[Is the change risky?]: probably not
[Why is the change risky/not risky?]: relatively simple change
[String changes made/needed]: none
Attachment #8934326 - Flags: approval-mozilla-beta?
Comment on attachment 8935884 [details] [diff] [review]
patch for esr52

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: possible crash
Fix Landed on Version: 59, will be uplifted to 58
Risk to taking this patch (and alternatives if risky): low
String or UUID changes made by this patch: none

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Attachment #8935884 - Flags: approval-mozilla-esr52?
Comment on attachment 8934326 [details] [diff] [review]
patch v3

uaf fix, beta58+
Attachment #8934326 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Group: network-core-security → core-security-release
Comment on attachment 8935884 [details] [diff] [review]
patch for esr52

Sec-high, ESR52+
Attachment #8935884 - Flags: approval-mozilla-esr52? → approval-mozilla-esr52+
Whiteboard: [necko-triaged] → [necko-triaged][adv-main58+][adv-esr52.6+]
Flags: qe-verify-
Whiteboard: [necko-triaged][adv-main58+][adv-esr52.6+] → [OA][necko-triaged][adv-main58+][adv-esr52.6+][post-critsmash-triage]
Group: core-security-release
Depends on: 1492743
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: