Closed Bug 1395092 Opened 7 years ago Closed 7 years ago

Intermittent test_profile_management.py TestProfileManagement.test_clean_profile | application crashed [@ google_breakpad::ExceptionHandler::WriteMinidump()]

Categories

(Toolkit :: Crash Reporting, defect, P5)

Unspecified
Windows
defect

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: crash, intermittent-failure)

Crash Data

21:51:10     INFO -  Crash reason:  EXCEPTION_NONCONTINUABLE_EXCEPTION
21:51:10     INFO -  Crash address: 0x0
21:51:10     INFO -  Process uptime: 9 seconds
21:51:10     INFO -  Thread 0 (crashed)
21:51:10     INFO -   0  xul.dll!google_breakpad::ExceptionHandler::WriteMinidump() [exception_handler.cc:70052a839bc1 : 740 + 0x0]
21:51:10     INFO -      rax = 0x000000a9af9fdf20   rdx = 0x000000a9af9fe850
21:51:10     INFO -      rcx = 0x000000a9af9fdf20   rbx = 0x000000a9af9fe460
21:51:10     INFO -      rsi = 0x0000000000000001   rdi = 0x000000a9af9fe850
21:51:10     INFO -      rbp = 0x000000a9af9fe6e0   rsp = 0x000000a9af9fde50
21:51:10     INFO -       r8 = 0x00000000000004d0    r9 = 0x0000000000000000
21:51:10     INFO -      r10 = 0x0000000020000000   r11 = 0x000000a9af9fdf20
21:51:10     INFO -      r12 = 0x0000000000000898   r13 = 0x0000000000000000
21:51:10     INFO -      r14 = 0x000000a9af9fe948   r15 = 0x0000000000001e70
21:51:10     INFO -      rip = 0x00007ffd806bf096
21:51:10     INFO -      Found by: given as instruction pointer in context
21:51:10     INFO -   1  xul.dll!google_breakpad::ExceptionHandler::WriteMinidump(std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > const &,bool (*)(wchar_t const *,wchar_t const *,void *,_EXCEPTION_POINTERS *,MDRawAssertionInfo *,bool),void *,_MINIDUMP_TYPE) [exception_handler.cc:70052a839bc1 : 764 + 0xa]
21:51:10     INFO -      rbx = 0x000000a9af9fe460   rbp = 0x000000a9af9fe6e0
21:51:10     INFO -      rsp = 0x000000a9af9fe410   r12 = 0x0000000000000898
21:51:10     INFO -      r13 = 0x0000000000000000   r14 = 0x000000a9af9fe948
21:51:10     INFO -      r15 = 0x0000000000001e70   rip = 0x00007ffd806bf116
21:51:10     INFO -      Found by: call frame info
21:51:10     INFO -   2  xul.dll!CrashReporter::CreateMinidumpsAndPair(void *,unsigned long,nsACString const &,nsIFile *,nsIFile * *,std::function<void > &&,bool) [nsExceptionHandler.cpp:70052a839bc1 : 4172 + 0x1d]
21:51:10     INFO -      rbx = 0x000000a9af9fe460   rbp = 0x000000a9af9fe6e0
21:51:10     INFO -      rsp = 0x000000a9af9fe5e0   r12 = 0x0000000000000898
21:51:10     INFO -      r13 = 0x0000000000000000   r14 = 0x000000a9af9fe948
21:51:10     INFO -      r15 = 0x0000000000001e70   rip = 0x00007ffd806b4e0e
21:51:10     INFO -      Found by: call frame info
21:51:10     INFO -   3  xul.dll!mozilla::ipc::CrashReporterHost::GenerateMinidumpAndPair(mozilla::ipc::GeckoChildProcessHost *,nsIFile *,nsACString const &,std::function<void > &&,bool) [CrashReporterHost.cpp:70052a839bc1 : 214 + 0x3d]
21:51:10     INFO -      rbx = 0x000000a9af9fe460   rbp = 0x000000a9af9fe6e0
21:51:10     INFO -      rsp = 0x000000a9af9fe7e0   r12 = 0x0000000000000898
21:51:10     INFO -      r13 = 0x0000000000000000   r14 = 0x000000a9af9fe948
21:51:10     INFO -      r15 = 0x0000000000001e70   rip = 0x00007ffd7e5db846
21:51:10     INFO -      Found by: call frame info
21:51:10     INFO -   4  xul.dll!mozilla::dom::ContentParent::KillHard(char const *) [ContentParent.cpp:70052a839bc1 : 3080 + 0x45]
21:51:10     INFO -      rbx = 0x000000a9af9fe460   rbp = 0x000000a9af9fe6e0
21:51:10     INFO -      rsp = 0x000000a9af9fe8e0   r12 = 0x0000000000000898
21:51:10     INFO -      r13 = 0x0000000000000000   r14 = 0x000000a9af9fe948
21:51:10     INFO -      r15 = 0x0000000000001e70   rip = 0x00007ffd7f692906
21:51:10     INFO -      Found by: call frame info
21:51:10     INFO -   5  xul.dll!mozilla::dom::ContentParent::ForceKillTimerCallback(nsITimer *,void *) [ContentParent.cpp:70052a839bc1 : 3035 + 0xf]
21:51:10     INFO -      rbx = 0x000000a9af9fe460   rbp = 0x000000a9af9fe6e0
21:51:10     INFO -      rsp = 0x000000a9af9fea40   r12 = 0x0000000000000898
21:51:10     INFO -      r13 = 0x0000000000000000   r14 = 0x000000a9af9fe948
21:51:10     INFO -      r15 = 0x0000000000001e70   rip = 0x00007ffd7f68da22
21:51:10     INFO -      Found by: call frame info

The null pointer crash happens here:

https://dxr.mozilla.org/mozilla-central/rev/db7f19e26e571ae1dd309f5d2f387b06ba670c30/toolkit/crashreporter/breakpad-client/windows/handler/exception_handler.cc#740
OS: Unspecified → Windows
Please note that this is not a null deref crash:

ExceptionHandler::WriteMinidump() @ https://dxr.mozilla.org/mozilla-central/rev/db7f19e26e571ae1dd309f5d2f387b06ba670c30/toolkit/crashreporter/breakpad-client/windows/handler/exception_handler.cc#740

the ExceptionHandler instance *this* is created on the stack: https://dxr.mozilla.org/mozilla-central/rev/db7f19e26e571ae1dd309f5d2f387b06ba670c30/toolkit/crashreporter/breakpad-client/windows/handler/exception_handler.cc#764

And its only argument &exception_ptrs also points to a struct that is created on the stack.

From the stack, ContentParent::KillHard() is called from ContentParent::ForceKillTimerCallback(), which is triggered when it timed out in shutting down the child process. It looks like the child process refuses to die when the parent process tried to shut it down.

Looking at the child stacks might provide some insight: https://treeherder.mozilla.org/logviewer.html#?job_id=126865830&repo=autoland&lineNumber=35312 but in this case the child process is idling:

21:51:15     INFO -  GPU: UNKNOWN
21:51:15     INFO -  Crash reason:  EXCEPTION_BREAKPOINT
21:51:15     INFO -  Crash address: 0x7ffdb1038be4
21:51:15     INFO -  Process uptime: 8 seconds
21:51:15     INFO -  Thread 0 (crashed)
21:51:15     INFO -   0  ntdll.dll!NtWaitForAlertByThreadId + 0x14
21:51:15     INFO -      rax = 0x00007ffd80f669b0   rdx = 0x0000000000000001
21:51:15     INFO -      rcx = 0x000002241aaf8180   rbx = 0x0000000000000000
21:51:15     INFO -      rsi = 0x0000000000000000   rdi = 0x0000000000000000
21:51:15     INFO -      rbp = 0x00000035d13ff080   rsp = 0x00000035d13ff028
21:51:15     INFO -       r8 = 0x00007ffd80f81ee8    r9 = 0x0000022415342a60
21:51:15     INFO -      r10 = 0x0000000000000000   r11 = 0x0000000000000008
21:51:15     INFO -      r12 = 0x0000000000000000   r13 = 0x00000224182c6160
21:51:15     INFO -      r14 = 0x000002241533d410   r15 = 0x000002241533d448
21:51:15     INFO -      rip = 0x00007ffdb1038be4
21:51:15     INFO -      Found by: given as instruction pointer in context
21:51:15     INFO -   1  ntdll.dll!RtlSleepConditionVariableCS + 0xc9
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff030   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffdb10001b9
21:51:15     INFO -      Found by: call frame info
21:51:15     INFO -   2  KERNELBASE.dll!Normalization__LoadTables + 0xfc
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff0a0   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffdae1ee988
21:51:15     INFO -      Found by: call frame info
21:51:15     INFO -   3  mozglue.dll!mozilla::detail::ConditionVariableImpl::wait(mozilla::detail::MutexImpl &) [ConditionVariable_windows.cpp:70052a839bc1 : 58 + 0x10]
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff0d0   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffda3f4a011
21:51:15     INFO -      Found by: call frame info
21:51:15     INFO -   4  xul.dll!mozilla::ThreadEventQueue<mozilla::PrioritizedEventQueue<mozilla::EventQueue> >::GetEvent(bool,mozilla::EventPriority *) [ThreadEventQueue.cpp:70052a839bc1 : 138 + 0x13]
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff100   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffd7e2b536b
21:51:15     INFO -      Found by: call frame info
21:51:15     INFO -   5  xul.dll!nsThread::ProcessNextEvent(bool,bool *) [nsThread.cpp:70052a839bc1 : 967 + 0x19]
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff140   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffd7e2b8816
21:51:15     INFO -      Found by: call frame info
21:51:15     INFO -   6  xul.dll!NS_ProcessNextEvent(nsIThread *,bool) [nsThreadUtils.cpp:70052a839bc1 : 521 + 0xd]
21:51:15     INFO -      rbx = 0x0000000000000000   rbp = 0x00000035d13ff080
21:51:15     INFO -      rsp = 0x00000035d13ff7f0   r12 = 0x0000000000000000
21:51:15     INFO -      r13 = 0x00000224182c6160   r14 = 0x000002241533d410
21:51:15     INFO -      r15 = 0x000002241533d448   rip = 0x00007ffd7e2b7ceb
21:51:15     INFO -      Found by: call frame info
We should add some better support in mozcrash for displaying minidump pairs like this. The current state of affairs is confusing.
There is bug 1293270 open for improvements.
https://wiki.mozilla.org/Bugmasters#Intermittent_Test_Failure_Cleanup
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.