1589966 - Intermittent <test-name> application crashed [@ google_breakpad::ExceptionHandler::WriteMinidump(std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > const &,bool (*)(wchar_t const *,

Status: RESOLVED DUPLICATE of bug 1538088

(Toolkit :: Crash Reporting, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=272084681&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/bQQ8n69aTr-6XZVwhdxOHg/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-10-20T17:12:29.708Z] 17:12:29 INFO - TEST-START | awsy\test_memory_usage.py TestMemoryUsage.test_open_tabs
[task 2019-10-20T17:23:05.472Z] 17:23:05 INFO - starting checkpoint TabsClosedSettled...
[task 2019-10-20T17:23:05.473Z] 17:23:05 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/C8j1TmTsSyqS4SUqVByw6Q/artifacts/public/build/target.crashreporter-symbols.zip
[task 2019-10-20T17:23:10.374Z] 17:23:10 INFO - mozcrash Copy/paste: Z:\task_1571591087\build\win32-minidump_stackwalk.exe c:\users\task_1571591087\appdata\local\temp\tmppx2glp.profile\minidumps\aa7a8e31-b0b9-4d7a-9b09-f78893ac18c0-browser.dmp c:\users\task_1571591087\appdata\local\temp\tmphwhaph
[task 2019-10-20T17:23:16.888Z] 17:23:16 INFO - mozcrash Saved minidump as Z:\task_1571591087\build\blobber_upload_dir\aa7a8e31-b0b9-4d7a-9b09-f78893ac18c0-browser.dmp
[task 2019-10-20T17:23:16.935Z] 17:23:16 INFO - PROCESS-CRASH | awsy\test_memory_usage.py TestMemoryUsage.test_open_tabs | application crashed [@ google_breakpad::ExceptionHandler::WriteMinidump(std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > const &,bool (*)(wchar_t const *,wchar_t const *,void *,_EXCEPTION_POINTERS *,MDRawAssertionInfo *,mozilla::phc::AddrInfo const *,bool),void ,_MINIDUMP_TYPE)]
[task 2019-10-20T17:23:16.935Z] 17:23:16 INFO - Crash dump filename: c:\users\task_1571591087\appdata\local\temp\tmppx2glp.profile\minidumps\aa7a8e31-b0b9-4d7a-9b09-f78893ac18c0-browser.dmp
[task 2019-10-20T17:23:16.935Z] 17:23:16 INFO - Operating system: Windows NT
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO - 10.0.17134
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO - CPU: amd64
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO - family 6 model 79 stepping 1
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO - 4 CPUs
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO -
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO - GPU: UNKNOWN
[task 2019-10-20T17:23:16.936Z] 17:23:16 INFO -
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - Crash reason: EXCEPTION_NONCONTINUABLE_EXCEPTION
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - Crash address: 0x0
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - Process uptime: 568 seconds
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO -
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - Thread 0 (crashed)
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - 0 xul.dll!google_breakpad::ExceptionHandler::WriteMinidump(std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > const &,bool ()(wchar_t const *,wchar_t const *,void *,_EXCEPTION_POINTERS *,MDRawAssertionInfo *,mozilla::phc::AddrInfo const *,bool),void *,_MINIDUMP_TYPE) [exception_handler.cc:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 775 + 0x87]
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - rax = 0x00000008397fea30 rdx = 0x00000008397feb60
[task 2019-10-20T17:23:16.937Z] 17:23:16 INFO - rcx = 0x00000008397fe560 rbx = 0x0000000000000960
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - rsi = 0x00000008397fe560 rdi = 0x00000008397feb60
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - rbp = 0x0000000000000047 rsp = 0x00000008397fe390
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - r8 = 0x00000000000004d0 r9 = 0x0000000000000000
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - r10 = 0x0000000000000000 r11 = 0x00000008397fe560
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - r12 = 0x0000000000002978 r13 = 0x0000000000001d24
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - r14 = 0x00007fff877d2da0 r15 = 0x0000000000000000
[task 2019-10-20T17:23:16.938Z] 17:23:16 INFO - rip = 0x00007fff877d9759
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - Found by: given as instruction pointer in context
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - 1 xul.dll!CrashReporter::CreateMinidumpsAndPair(void *,unsigned long,nsTSubstring<char> const &,nsIFile *,mozilla::EnumeratedArray<CrashReporter::Annotation,151,nsTString<char> > &,nsIFile * *) [nsExceptionHandler.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 3543 + 0x1e]
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - rsp = 0x00000008397feb10 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff877d33f5
[task 2019-10-20T17:23:16.939Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - 2 xul.dll!mozilla::ipc::CrashReporterHost::GenerateMinidumpAndPair<mozilla::dom::ContentParent>(mozilla::dom::ContentParent *,nsIFile *,nsTSubstring<char> const &) [CrashReporterHost.h:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 74 + 0x26]
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - rsp = 0x00000008397febd0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff862e1761
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.940Z] 17:23:16 INFO - 3 xul.dll!mozilla::dom::ContentParent::GeneratePairedMinidump(char const *) [ContentParent.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 3455 + 0xb]
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - rsp = 0x00000008397fec40 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff863c703a
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - 4 xul.dll!mozilla::dom::ContentParent::KillHard(char const *) [ContentParent.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 3476 + 0xb]
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.941Z] 17:23:16 INFO - rsp = 0x00000008397fed10 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff863bfd80
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - 5 xul.dll!nsTimerImpl::Fire(int) [nsTimerImpl.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 561 + 0x7]
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - rsp = 0x00000008397fed90 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.942Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8428da67
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - 6 xul.dll!nsTimerEvent::Run() [TimerThread.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 260 + 0x8]
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - rsp = 0x00000008397feee0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.943Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8428d714
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - 7 xul.dll!nsThread::ProcessNextEvent(bool,bool *) [nsThread.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 1225 + 0xa]
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - rsp = 0x00000008397fef60 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.944Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff842942ba
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - 8 xul.dll!NS_ProcessNextEvent(nsIThread *,bool) [nsThreadUtils.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 486 + 0xd]
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - rsp = 0x00000008397ff4e0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff842968ae
[task 2019-10-20T17:23:16.945Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - 9 xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *) [MessagePump.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 110 + 0xa]
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - rsp = 0x00000008397ff530 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8475e818
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - 10 xul.dll!MessageLoop::RunHandler() [message_loop.cc:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 308 + 0xf]
[task 2019-10-20T17:23:16.946Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - rsp = 0x00000008397ff5a0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8472baf9
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - 11 xul.dll!MessageLoop::Run() [message_loop.cc:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 290 + 0x5]
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - rsp = 0x00000008397ff5f0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8472ba78
[task 2019-10-20T17:23:16.947Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - 12 xul.dll!nsBaseAppShell::Run() [nsBaseAppShell.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 137 + 0xd]
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - rsp = 0x00000008397ff640 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff86633768
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.948Z] 17:23:16 INFO - 13 xul.dll!nsAppShell::Run() [nsAppShell.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 406 + 0x8]
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - rsp = 0x00000008397ff680 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff8669c054
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - 14 xul.dll!nsAppStartup::Run() [nsAppStartup.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 276 + 0x6]
[task 2019-10-20T17:23:16.949Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - rsp = 0x00000008397ff6d0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff87738b26
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - 15 xul.dll!XREMain::XRE_mainRun() [nsAppRunner.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 4599 + 0x6]
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.950Z] 17:23:16 INFO - rsp = 0x00000008397ff700 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff877f4af7
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - 16 xul.dll!XREMain::XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [nsAppRunner.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 4734 + 0x8]
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.951Z] 17:23:16 INFO - rsp = 0x00000008397ff920 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff877f5944
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - 17 xul.dll!XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [nsAppRunner.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 4815 + 0x10]
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - rsp = 0x00000008397ff9e0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.952Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fff877f603c
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - 18 firefox.exe!NS_internal_main(int,char * *,char * *) [nsBrowserApp.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 300 + 0x99]
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - rsp = 0x00000008397ffb40 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.953Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007ff70ad2180d
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - 19 firefox.exe!wmain [nsWindowsWMain.cpp:a55fbe37780387823c0013f11a5e90ea4ee7ed76 : 131 + 0x15]
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - rsp = 0x00000008397ffd10 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.954Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007ff70ad21232
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - 20 firefox.exe!static int __scrt_common_main_seh() [exe_common.inl : 288 + 0x22]
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - rsp = 0x00000008397ffdd0 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007ff70ad74098
[task 2019-10-20T17:23:16.955Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - 21 kernel32.dll!BaseThreadInitThunk + 0x14
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - rsp = 0x00000008397ffe10 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fffcc8e3034
[task 2019-10-20T17:23:16.956Z] 17:23:16 INFO - Found by: call frame info
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - 22 ntdll.dll!SdbpCheckMatchingFiles + 0x81
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - rbx = 0x0000000000000960 rbp = 0x0000000000000047
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - rsp = 0x00000008397ffe40 r12 = 0x0000000000002978
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - r13 = 0x0000000000001d24 r14 = 0x00007fff877d2da0
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - r15 = 0x0000000000000000 rip = 0x00007fffcf061461
[task 2019-10-20T17:23:16.957Z] 17:23:16 INFO - Found by: call frame info

Comment 1

[Intermittent Failures Robot]

2 failures in 3964 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• windows10-64-qr: 1
• windows10-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-10-14&endday=2019-10-20&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:glandium, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 3

[Eric Rahm [:erahm]]

This appears to be a crash while crashing, redirecting to gsvelto.

Comment 4

[Gabriele Svelto [:gsvelto]]

This is a content process hang, not a crash (see the crash reason: EXCEPTION_NONCONTINUABLE_EXCEPTION). As with all hangs we've grabbed two minidumps, one for the main process (which is what you see in the trace, the filename is aa7a8e31-b0b9-4d7a-9b09-f78893ac18c0-browser.dmp) and one for the hung content process which you can find below the first one (filename aa7a8e31-b0b9-4d7a-9b09-f78893ac18c0.dmp, note the missing -browser suffix).

You can see the content process minidump in these lines in the log. The content process was GCing/CCing and must have been stuck doing that for a long while to trigger the hang detector.

We should probably modify the scripts we use in automation to first analyze the minidump with the unprefixed filename (<uuid>.dmp) and only afterwards the ones with the prefixed ones (<uuid>-browser.dmp). It should also be possible to flag these as hangs and not crashes.

Comment 5

[Eric Rahm [:erahm]]

I guess I'm confused here, why is the parent process crashing? I don't think there was a hang in the child process, it looks like it triggered EXCEPTION_BREAKPOINT during gc/cc by poking some random memory.

Comment 6

[Gabriele Svelto [:gsvelto]]

(In reply to Eric Rahm [:erahm] from comment #5)

I guess I'm confused here, why is the parent process crashing? I don't think there was a hang in the child process, it looks like it triggered EXCEPTION_BREAKPOINT during gc/cc by poking some random memory.

The main process isn't crashing, if you look up the stack you'll find it's here:

https://searchfox.org/mozilla-central/rev/d061ba55ac76f41129618d638f4ef674303ec103/dom/ipc/ContentParent.cpp#3478

When the main process takes a minidump of itself without crashing it will clone() itself at the point of writing the minidump, trigger an exception in the cloned process (hence the EXCEPTION_BREAKPOINT cause) and then let the minidump writing machinery do its job.

BTW you can be sure that the main process hasn't crashed because the crash is finalized (the .extra file has been written out and is complete) and that happens past the point captured in the minidump:

https://searchfox.org/mozilla-central/rev/d061ba55ac76f41129618d638f4ef674303ec103/dom/ipc/ContentParent.cpp#3459

Comment 7

[Gabriele Svelto [:gsvelto]]

I forgot an important detail: this is a shutdown content hang to be precise. I.e. the browser wasn't shutting down but a content process was. After a content process is signaled to shut down we wait a minute and then kill it after taking a minidump.

Comment 8

[Intermittent Failures Robot]

1 failures in 4331 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-11-25&endday=2019-12-01&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 5144 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-12-02&endday=2019-12-08&tree=all

Comment 17

[Intermittent Failures Robot]

12 failures in 4582 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 11

Platform breakdown:

• windows10-64-shippable-qr: 5
• windows10-64-ref-hw-2017: 4
• linux64-shippable-qr: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-12-16&endday=2019-12-22&tree=all

Comment 18

[Intermittent Failures Robot]

1 failures in 1365 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows10-64-ref-hw-2017: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-12-23&endday=2019-12-29&tree=all

Comment 19

[Intermittent Failures Robot]

2 failures in 1862 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• windows10-64-shippable-qr: 1
• windows10-64-ref-hw-2017: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2019-12-30&endday=2020-01-05&tree=all

Comment 20

[Alexandru Michis [:malexandru]]

Grabiele,
This looks to be quite frequent on central, could you please take another look?
Recent log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=284109854&repo=mozilla-central&lineNumber=5950

Comment 21

[Gabriele Svelto [:gsvelto]]

Same as with the others this is a hang. I filed bug 1605328 to make it possible to distinguish them instead of all being lumped under this signature. Once that's fixed we'll have better visibility in what's going on. That being said there seems to be something wrong with the Windows libraries' symbols in that trace which I will investigate.

Comment 22

[Gabriele Svelto [:gsvelto]]

Henrik, do you know which stackwalker is used to parse minidumps in test crashes? It seems like it's misinterpreting the Windows symbols when walking the stack in this failures. For example see the stack here:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=284109854&repo=mozilla-central&lineNumber=10687-10705

0 ntdll.dll!AslHashFree + 0x35028
1 ntdll.dll!SdbResolveDatabase + 0x8d
2 mozglue.dll!mozilla::detail::ConditionVariableImpl::wait(mozilla::detail::MutexImpl&) [ConditionVariable_windows.cpp:374a48ce8690a53e44ef3f4134b8ecd8099bea14 : 50 + 0xf]

If I process the minidump using the regular breakpad stackwalker I get this instead:

0  ntdll.dll!NtWaitForAlertByThreadId + 0x14
1  ntdll.dll!RtlSleepConditionVariableSRW + 0xf9
2  KERNELBASE.dll!SleepConditionVariableSRW + 0x28
3  mozglue.dll!mozilla::detail::ConditionVariableImpl::wait(mozilla::detail::MutexImpl&) [ConditionVariable_windows.cpp:374a48ce8690a53e44ef3f4134b8ecd8099bea14 : 50 + 0xf]

This is obviously the correct stack. I wonder if the stackwalker used to parse those minidumps is out-of-date compared to what we have in mozilla-central. We recently replaced the dump_syms command we use to dump Windows libraries and it's spitting out richer information than the old one so it's possible that an old tool might misinterpret it and yield a broken stack trace.

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Isn't it https://searchfox.org/mozilla-central/source/tools/crashreporter/minidump_stackwalk what we use?

Comment 24

[Gabriele Svelto [:gsvelto]]

I'm not sure. That tool doesn't get build in a local build and looking at the log there's a reference to a tool with a similar name coming from tooltool:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=284109854&repo=mozilla-central&lineNumber=74-75

I'm not familiar with tooltool so I'll have to ask around.

Comment 25

[Gabriele Svelto [:gsvelto]]

OK, I found it, it's a tool that Ted wrote and which we're pulling down from tooltool, see bug 1487410 comment 0.

Comment 26

[Intermittent Failures Robot]

21 failures in 4472 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 21

Platform breakdown:

• windows10-64-ref-hw-2017: 7
• windows10-64-shippable-qr: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 27

[Gabriele Svelto [:gsvelto]]

Added bug 1605328 as a dependency because until that's fixed we won't have actionable signatures here.

Comment 28

[Intermittent Failures Robot]

11 failures in 3773 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 10
• autoland: 1

Platform breakdown:

• windows10-64-ref-hw-2017: 6
• windows10-64-shippable-qr: 4
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-01-13&endday=2020-01-19&tree=all

Comment 29

[Intermittent Failures Robot]

18 failures in 4004 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 18

Platform breakdown:

• windows10-64-ref-hw-2017: 11
• windows10-64-shippable-qr: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-01-20&endday=2020-01-26&tree=all

Comment 30

[Intermittent Failures Robot]

10 failures in 2190 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 10

Platform breakdown:

• windows10-64-ref-hw-2017: 8
• windows10-64-shippable-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-01-27&endday=2020-02-02&tree=all

Comment 31

[Intermittent Failures Robot]

2 failures in 3364 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• windows10-64-ref-hw-2017: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-02-03&endday=2020-02-09&tree=all

Comment 32

[Intermittent Failures Robot]

1 failures in 4690 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows10-64-ref-hw-2017: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-02-10&endday=2020-02-16&tree=all

Comment 33

[Intermittent Failures Robot]

3 failures in 4045 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 1

Platform breakdown:

• windows10-64-ref-hw-2017: 2
• linux1804-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 34

[Intermittent Failures Robot]

2 failures in 5233 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• linux1804-64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 5582 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux1804-64-shippable-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-04-20&endday=2020-04-26&tree=all

Comment 36

[Intermittent Failures Robot]

3 failures in 5423 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform breakdown:

• linux1804-64-qr: 2
• linux1804-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1589966&startday=2020-04-27&endday=2020-05-03&tree=all