Closed Bug 1270752 Opened 9 years ago Closed 9 years ago

AddressSanitizer: heap-use-after-free /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:509 __interceptor_strlen

Categories

(Core :: IPC, defect)

Product:
Core
Component:
IPC
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla49
Tracking Flags:
Tracking Status
firefox49 --- fixed

People

(Reporter: jya, Assigned: mayhemer)

References

(Blocks 1 open bug)

Details

(Keywords: csectype-uaf, Whiteboard: btpp-active)

Attachments

(1 file, 1 obsolete file)

v1.0.1 (keep buffer passed to PR_SetEnv() during process launch)
4.71 KB, patch
mayhemer
: review+
Details | Diff | Splinter Review
https://treeherder.mozilla.org/logviewer.html#?job_id=20451930&repo=try#L2572 STR: * Modify testing/mochitest/runtests.py so that some verbose mochitest are set: +MOZ_LOG_MODULES = "MediaFormatReader:5" * Run try with mochitest-media-e10s for linux64-asan You get crashes: 23:46:11 INFO - ==10299==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d000020508 at pc 0x4609ce bp 0x7f0e167b6940 sp 0x7f0e167b6920 23:46:11 INFO - READ of size 123 at 0x60d000020508 thread T4 (Gecko_IOThread) 23:46:11 INFO - #0 0x4609cd in __interceptor_strlen /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:509 23:46:11 INFO - #1 0x7f0e3874b98c in PR_DuplicateEnvironment /builds/slave/try-l64-asan-00000000000000000/build/src/nsprpub/pr/src/misc/prenv.c:135 23:46:12 INFO - #2 0x7f0e1e0fc1d0 in EnvironmentEnvp /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/process_util_linux.cc:69 23:46:12 INFO - #3 0x7f0e1e0fc1d0 in Environment /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/process_util_linux.cc:127 23:46:12 INFO - #4 0x7f0e1e0fc1d0 in base::LaunchApp(std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::pair<int, int>, std::allocator<std::pair<int, int> > > const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, base::ChildPrivileges, bool, int*, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/process_util_linux.cc:234 23:46:12 INFO - #5 0x7f0e1e16d69d in mozilla::ipc::GeckoChildProcessHost::PerformAsyncLaunchInternal(std::vector<std::string, std::allocator<std::string> >&, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:873 23:46:12 INFO - #6 0x7f0e1e16be36 in mozilla::ipc::GeckoChildProcessHost::PerformAsyncLaunch(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:545 23:46:12 INFO - #7 0x7f0e1e16a3d3 in mozilla::ipc::GeckoChildProcessHost::RunPerformAsyncLaunch(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:563 23:46:12 INFO - #8 0x7f0e1e1712aa in _ZN25nsRunnableMethodArgumentsIJSt6vectorISsSaISsEEN4base19ProcessArchitectureEEE9applyImplIN7mozilla3ipc21GeckoChildProcessHostEMS9_FbS2_S4_EJ20StoreCopyPassByValueIS2_ESC_IS4_EEJLm0ELm1EEEEDTcldsdefp_fp0_spcldtclsr7mozillaE3GetIXT2_EEfp1_E15PassAsParameterEEEPT_T0_RNS7_5TupleIJDpT1_EEENS7_13IndexSequenceIJXspT2_EEEE /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:707 23:46:12 INFO - #9 0x7f0e1e170bbb in apply<mozilla::ipc::GeckoChildProcessHost, bool (mozilla::ipc::GeckoChildProcessHost::*)(std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > >, base::ProcessArchitecture)> /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:713 23:46:12 INFO - #10 0x7f0e1e170bbb in nsRunnableMethodImpl<bool (mozilla::ipc::GeckoChildProcessHost::*)(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture), false, false, std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture>::Run() /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:741 23:46:12 INFO - #11 0x7f0e1e117cfc in forget /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:335 23:46:12 INFO - #12 0x7f0e1e117cfc in DeferOrRunPendingTask /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:343 23:46:12 INFO - #13 0x7f0e1e117cfc in MessageLoop::DoWork() /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:418 23:46:12 INFO - #14 0x7f0e1e11addc in base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_pump_libevent.cc:341 23:46:12 INFO - #15 0x7f0e1e1153dc in RunInternal /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:233 23:46:12 INFO - #16 0x7f0e1e1153dc in RunHandler /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:226 23:46:12 INFO - #17 0x7f0e1e1153dc in MessageLoop::Run() /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:206 23:46:12 INFO - #18 0x7f0e1e12d0b5 in base::Thread::ThreadMain() /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/thread.cc:178 23:46:12 INFO - #19 0x7f0e1e12e85c in ThreadFunc(void*) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/platform_thread_posix.cc:36 23:46:12 INFO - #20 0x7f0e3bc92e99 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7e99) 23:46:12 INFO - #21 0x7f0e3ada238c (/lib/x86_64-linux-gnu/libc.so.6+0xf338c) 23:46:12 INFO - 0x60d000020508 is located 8 bytes inside of 135-byte region [0x60d000020500,0x60d000020587) 23:46:12 INFO - freed by thread T4 (Gecko_IOThread) here: 23:46:12 INFO - #0 0x471fe1 in __interceptor_free /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64 23:46:12 INFO - #1 0x7f0e1e16be23 in ~nsACString_internal /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsTSubstring.h:95 23:46:12 INFO - #2 0x7f0e1e16be23 in SetChildLogName /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:510 23:46:12 INFO - #3 0x7f0e1e16be23 in mozilla::ipc::GeckoChildProcessHost::PerformAsyncLaunch(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:542 23:46:12 INFO - #4 0x7f0e1e16a3d3 in mozilla::ipc::GeckoChildProcessHost::RunPerformAsyncLaunch(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/glue/GeckoChildProcessHost.cpp:563 23:46:12 INFO - #5 0x7f0e1e1712aa in _ZN25nsRunnableMethodArgumentsIJSt6vectorISsSaISsEEN4base19ProcessArchitectureEEE9applyImplIN7mozilla3ipc21GeckoChildProcessHostEMS9_FbS2_S4_EJ20StoreCopyPassByValueIS2_ESC_IS4_EEJLm0ELm1EEEEDTcldsdefp_fp0_spcldtclsr7mozillaE3GetIXT2_EEfp1_E15PassAsParameterEEEPT_T0_RNS7_5TupleIJDpT1_EEENS7_13IndexSequenceIJXspT2_EEEE /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:707 23:46:12 INFO - #6 0x7f0e1e170bbb in apply<mozilla::ipc::GeckoChildProcessHost, bool (mozilla::ipc::GeckoChildProcessHost::*)(std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > >, base::ProcessArchitecture)> /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:713 23:46:12 INFO - #7 0x7f0e1e170bbb in nsRunnableMethodImpl<bool (mozilla::ipc::GeckoChildProcessHost::*)(std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture), false, false, std::vector<std::string, std::allocator<std::string> >, base::ProcessArchitecture>::Run() /builds/slave/try-l64-asan-00000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:741 23:46:12 INFO - previously allocated by thread T4 (Gecko_IOThread) here: 23:46:12 INFO - #0 0x4721e1 in malloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74 23:46:12 INFO - #1 0x7f0e1d2d0724 in Alloc /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/string/nsSubstring.cpp:217 23:46:12 INFO - #2 0x7f0e1d2d0724 in nsACString_internal::MutatePrep(unsigned int, char**, unsigned int*) /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/string/nsTSubstring.cpp:133 23:46:12 INFO - #3 0x7f0e1d2d99ee in nsACString_internal::ReplacePrepInternal(unsigned int, unsigned int, unsigned int, unsigned int) /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/string/nsTSubstring.cpp:195 23:46:12 INFO - #4 0x7f0e1d2d98d2 in nsACString_internal::ReplacePrep(unsigned int, unsigned int, unsigned int) /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/string/nsTSubstring.cpp:185 23:46:13 INFO - #5 0x7f0e1d2dc3b7 in nsACString_internal::Replace(unsigned int, unsigned int, char const*, unsigned int, mozilla::fallible_t const&) /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/string/nsTSubstring.cpp:562 23:46:13 INFO - Thread T4 (Gecko_IOThread) created by T0 here: 23:46:13 INFO - #0 0x45ea55 in __interceptor_pthread_create /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:175 23:46:13 INFO - #1 0x7f0e1e12cc74 in CreateThread /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/platform_thread_posix.cc:135 23:46:13 INFO - #2 0x7f0e1e12cc74 in Create /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/platform_thread_posix.cc:146 23:46:13 INFO - #3 0x7f0e1e12cc74 in base::Thread::StartWithOptions(base::Thread::Options const&) /builds/slave/try-l64-asan-00000000000000000/build/src/ipc/chromium/src/base/thread.cc:96 23:46:13 INFO - #4 0x7f0e1d475078 in NS_InitXPCOM2 /builds/slave/try-l64-asan-00000000000000000/build/src/xpcom/build/XPCOMInit.cpp:546 23:46:13 INFO - #5 0x7f0e256d2598 in Initialize /builds/slave/try-l64-asan-00000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:1532 23:46:13 INFO - #6 0x7f0e256d2598 in XREMain::XRE_main(int, char**, nsXREAppData const*) /builds/slave/try-l64-asan-00000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4447 23:46:13 INFO - #7 0x7f0e256d351e in XRE_main /builds/slave/try-l64-asan-00000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4559 23:46:13 INFO - #8 0x48a793 in do_main /builds/slave/try-l64-asan-00000000000000000/build/src/browser/app/nsBrowserApp.cpp:220 23:46:13 INFO - #9 0x48a793 in main /builds/slave/try-l64-asan-00000000000000000/build/src/browser/app/nsBrowserApp.cpp:360 23:46:13 INFO - #10 0x7f0e3acd076c (/lib/x86_64-linux-gnu/libc.so.6+0x2176c) 23:46:13 INFO - SUMMARY: AddressSanitizer: heap-use-after-free /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:509 __interceptor_strlen 23:46:13 INFO - Shadow bytes around the buggy address: 23:46:13 INFO - 0x0c1a7fffc050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 23:46:13 INFO - 0x0c1a7fffc060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 23:46:13 INFO - 0x0c1a7fffc070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 23:46:13 INFO - 0x0c1a7fffc080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 23:46:13 INFO - 0x0c1a7fffc090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 23:46:13 INFO - =>0x0c1a7fffc0a0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd 23:46:13 INFO - 0x0c1a7fffc0b0: fd fa fa fa fa fa fa fa fa fa 00 00 00 00 00 00 23:46:13 INFO - 0x0c1a7fffc0c0: 00 00 00 00 00 00 00 00 00 00 00 fa fa fa fa fa 23:46:13 INFO - 0x0c1a7fffc0d0: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00 23:46:13 INFO - 0x0c1a7fffc0e0: 00 00 00 00 00 fa fa fa fa fa fa fa fa fa 00 00 23:46:13 INFO - 0x0c1a7fffc0f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa 23:46:13 INFO - Shadow byte legend (one shadow byte represents 8 application bytes): 23:46:13 INFO - Addressable: 00 23:46:13 INFO - Partially addressable: 01 02 03 04 05 06 07 23:46:13 INFO - Heap left redzone: fa 23:46:13 INFO - Heap right redzone: fb 23:46:13 INFO - Freed heap region: fd 23:46:13 INFO - Stack left redzone: f1 23:46:13 INFO - Stack mid redzone: f2 23:46:13 INFO - Stack right redzone: f3 23:46:13 INFO - Stack partial redzone: f4 23:46:13 INFO - Stack after return: f5 23:46:13 INFO - Stack use after scope: f8 23:46:13 INFO - Global redzone: f9 23:46:13 INFO - Global init order: f6 23:46:13 INFO - Poisoned by user: f7 23:46:13 INFO - Contiguous container OOB:fc 23:46:13 INFO - ASan internal: fe 23:46:13 INFO - ==10299==ABORTING
Eric, could this be related to logging doing something incorrect? The stacks look like Chromium IPC stuff, but the STR seems to be just enabling logging. Thanks.
Flags: needinfo?(erahm)
The only think I can think of here is if something is essentially doing free(PR_GetEnv("foo")), I don't see anything obviously doing that in the logging code.
Flags: needinfo?(erahm)
(In reply to Eric Rahm [:erahm] from comment #2) > The only think I can think of here is if something is essentially doing > free(PR_GetEnv("foo")), I don't see anything obviously doing that in the > logging code. If you look at the link to the full log, you can see who is freeing it. Specifically, it is happening inside GeckoChildProcessHost::SetChildLogName(). I'm guessing the comment "Passing temporary to PR_SetEnv is ok here because env gets copied by exec, etc., to permanent storage in child when process launched." is not right...
Maybe this is a regression from bug 1248565, which last touched that code.
FWIW, here is an example of try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7428ec1bea4b83a60804d763d7954c4aaba61df8 where NSPR logging is enabled ; it makes troubleshooting e10s on asan extremely difficult for me at this stage (we have intermittent failures occurring) example: https://treeherder.mozilla.org/logviewer.html#?job_id=21282469&repo=try
See Also: → 1275117
Seems like I screwed this up in bug 1248565. Before my patch, the string passed to PR_SetEnv() was still alive (on stack) while PerformAsyncLaunchInternal was executed. I thought that PR_SetEnv() does the copy immediately (badly translated the comment to myself). But PR_DuplicateEnvironment() does that later.
Assignee: nobody → honzab.moz
Status: NEW → ASSIGNED
Blocks: 1248565
Attachment #8755866 - Flags: review?(jduell.mcbugs)
Comment on attachment 8755866 [details] [diff] [review] v1 (keep buffer passed to PR_SetEnv() during process launch) https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7610f1a85f5
Whiteboard: btpp-active
Attachment #8755866 - Flags: review?(jduell.mcbugs) → review+
Comment on attachment 8755866 [details] [diff] [review] v1 (keep buffer passed to PR_SetEnv() during process launch) Review of attachment 8755866 [details] [diff] [review]: ----------------------------------------------------------------- There's a small typo in the comment : // Muest keep
(Fixed the typo)
Attachment #8755866 - Attachment is obsolete: true
Attachment #8756790 - Flags: review+
Keywords: checkin-needed
Blocks: 1275860
https://hg.mozilla.org/mozilla-central/rev/062491c62640
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
status-firefox49: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla49
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: