Open Bug 1700687 Opened 3 years ago Updated 1 year ago

Intermittent PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::ipc::SharedMemory::SystemProtect]

Categories

(Core :: IPC, defect)

defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: crash, intermittent-failure)

Crash Data

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=334297085&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ezeFjWxARUmYoIb7X03xeg/runs/0/artifacts/public/logs/live_backing.log


[task 2021-03-24T16:51:03.640Z] 16:51:03     INFO - PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::ipc::SharedMemory::SystemProtect(char*, unsigned long, int)]
[task 2021-03-24T16:51:03.640Z] 16:51:03     INFO - Mozilla crash reason: MOZ_CRASH(can't mprotect())
[task 2021-03-24T16:51:03.641Z] 16:51:03     INFO - Crash dump filename: /tmp/tmpdMJTBQ.mozrunner/minidumps/058c9150-3a20-4c42-3f11-40761add1668.dmp
[task 2021-03-24T16:51:03.641Z] 16:51:03     INFO - Operating system: Linux
[task 2021-03-24T16:51:03.642Z] 16:51:03     INFO -                   0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2021-03-24T16:51:03.642Z] 16:51:03     INFO - CPU: amd64
[task 2021-03-24T16:51:03.642Z] 16:51:03     INFO -      family 6 model 85 stepping 7
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO -      2 CPUs
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - 
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - GPU: UNKNOWN
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - 
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - Crash reason:  SIGSEGV /SEGV_MAPERR
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - Crash address: 0x0
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - Process uptime: not available
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - 
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO - Thread 12 (crashed)
[task 2021-03-24T16:51:03.643Z] 16:51:03     INFO -  0  libxul.so!mozilla::ipc::SharedMemory::SystemProtect(char*, unsigned long, int) [SharedMemory_posix.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 25 + 0x29]
[task 2021-03-24T16:51:03.644Z] 16:51:03     INFO -     rax = 0x00007efe8e660c23   rdx = 0x0000000000000000
[task 2021-03-24T16:51:03.644Z] 16:51:03     INFO -     rcx = 0x000055efc03f90b8   rbx = 0x0000000000001000
[task 2021-03-24T16:51:03.645Z] 16:51:03     INFO -     rsi = 0x00007efea245e8b0   rdi = 0x00007efea245d680
[task 2021-03-24T16:51:03.645Z] 16:51:03     INFO -     rbp = 0x00007efe84ecd7d0   rsp = 0x00007efe84ecd7d0
[task 2021-03-24T16:51:03.646Z] 16:51:03     INFO -      r8 = 0x00007efea245e8b0    r9 = 0x00007efe84ece700
[task 2021-03-24T16:51:03.646Z] 16:51:03     INFO -     r10 = 0x0000000000000000   r11 = 0x0000000000000000
[task 2021-03-24T16:51:03.647Z] 16:51:03     INFO -     r12 = 0x00007efe75a36150   r13 = 0x00007efe84ecd890
[task 2021-03-24T16:51:03.647Z] 16:51:03     INFO -     r14 = 0x00000000000170a4   r15 = 0x00007efe84ecd890
[task 2021-03-24T16:51:03.648Z] 16:51:03     INFO -     rip = 0x00007efe91a03151
[task 2021-03-24T16:51:03.648Z] 16:51:03     INFO -     Found by: given as instruction pointer in context
[task 2021-03-24T16:51:03.648Z] 16:51:03     INFO -  1  libxul.so!mozilla::ipc::Shmem::Shmem(mozilla::ipc::Shmem::PrivateIPDLCaller, mozilla::ipc::SharedMemory*, int) [Shmem.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 246 + 0x19]
[task 2021-03-24T16:51:03.649Z] 16:51:03     INFO -     rbx = 0x0000000000001000   rbp = 0x00007efe84ecd830
[task 2021-03-24T16:51:03.649Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd7e0   r12 = 0x00007efe75a36150
[task 2021-03-24T16:51:03.649Z] 16:51:03     INFO -     r13 = 0x00007efe84ecd890   r14 = 0x00000000000170a4
[task 2021-03-24T16:51:03.650Z] 16:51:03     INFO -     r15 = 0x00007efe84ecd890   rip = 0x00007efe91a03298
[task 2021-03-24T16:51:03.650Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.650Z] 16:51:03     INFO -  2  libxul.so!mozilla::ipc::IToplevelProtocol::CreateSharedMemory(unsigned long, mozilla::ipc::SharedMemory::SharedMemoryType, bool, int*) [ProtocolUtils.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 752 + 0x22]
[task 2021-03-24T16:51:03.651Z] 16:51:03     INFO -     rbx = 0x00007efe84ecd890   rbp = 0x00007efe84ecd8e0
[task 2021-03-24T16:51:03.651Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd840   r12 = 0x00007efe8b534800
[task 2021-03-24T16:51:03.651Z] 16:51:03     INFO -     r13 = 0x00007efe84ecd890   r14 = 0x00007efe75a36150
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     r15 = 0x00007efe84ecd8f4   rip = 0x00007efe919f74d5
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -  3  libxul.so!mozilla::ipc::IProtocol::AllocUnsafeShmem(unsigned long, mozilla::ipc::SharedMemory::SharedMemoryType, mozilla::ipc::Shmem*) [ProtocolUtils.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 480 + 0xe]
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     rbx = 0x00007efe844433c0   rbp = 0x00007efe84ecd930
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd8f0   r12 = 0x000000000001c200
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     r13 = 0x00007efe8b534800   r14 = 0x0000000000000001
[task 2021-03-24T16:51:03.652Z] 16:51:03     INFO -     r15 = 0x0000000000000000   rip = 0x00007efe919f805c
[task 2021-03-24T16:51:03.653Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.653Z] 16:51:03     INFO -  4  libxul.so!mozilla::layers::ImageBridgeChild::ProxyAllocShmemNow(mozilla::layers::SynchronousTask*, unsigned long, mozilla::ipc::SharedMemory::SharedMemoryType, mozilla::ipc::Shmem*, bool, bool*) [ImageBridgeChild.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 735 + 0x5]
[task 2021-03-24T16:51:03.653Z] 16:51:03     INFO -     rbx = 0x00007efe84443310   rbp = 0x00007efe84ecd980
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd940   r12 = 0x000000000001c200
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     r13 = 0x00007efe8b534800   r14 = 0x0000000000000001
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     r15 = 0x0000000000000000   rip = 0x00007efe92206d23
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -  5  libxul.so!mozilla::runnable_args_memfn<RefPtr<mozilla::layers::ImageBridgeChild>, void (mozilla::layers::ImageBridgeChild::*)(mozilla::layers::SynchronousTask*, unsigned long, mozilla::ipc::SharedMemory::SharedMemoryType, mozilla::ipc::Shmem*, bool, bool*), mozilla::layers::SynchronousTask*, unsigned long, mozilla::ipc::SharedMemory::SharedMemoryType, mozilla::ipc::Shmem*, bool, bool*>::RunInternal() [runnable_utils.h:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 121 + 0x36]
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     rbx = 0x00007efe92206ccc   rbp = 0x00007efe84ecd9e0
[task 2021-03-24T16:51:03.654Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd990   r12 = 0x000000000001c200
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     r13 = 0x00007efe84443310   r14 = 0x0000000000000000
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     r15 = 0x00007efe8b482801   rip = 0x00007efe9221c33b
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -  6  libxul.so!mozilla::detail::runnable_args_base<(mozilla::detail::RunnableResult)0>::Run() [runnable_utils.h:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 41 + 0x6]
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     rbx = 0x00007efe75a39080   rbp = 0x00007efe84ecda00
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     rsp = 0x00007efe84ecd9f0   r12 = 0x00007efe84ecda40
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     r13 = 0x0000000000000001   r14 = 0x00007efe8b482760
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     r15 = 0x00007efe8b482830   rip = 0x00007efe9198bf28
[task 2021-03-24T16:51:03.655Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -  7  libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 1149 + 0x11]
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     rbx = 0x00007efe84ecdab0   rbp = 0x00007efe84ecdb40
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     rsp = 0x00007efe84ecda10   r12 = 0x00007efe84ecda40
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     r13 = 0x0000000000000001   r14 = 0x00007efe8b482760
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     r15 = 0x00007efe8b482830   rip = 0x00007efe9138e872
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -  8  libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 548 + 0xc]
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     rbx = 0x0000000000000001   rbp = 0x00007efe84ecdb70
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdb50   r12 = 0x0000000000000000
[task 2021-03-24T16:51:03.656Z] 16:51:03     INFO -     r13 = 0x00007efe8bfa4ca0   r14 = 0x00007efe84ecdb57
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     r15 = 0x00007efe8b482760   rip = 0x00007efe91392f52
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -  9  libxul.so!mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) [MessagePump.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 332 + 0xd]
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     rbx = 0x00007efe84ecdc68   rbp = 0x00007efe84ecdbc0
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdb80   r12 = 0x0000000000000000
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     r13 = 0x00007efe8bfa4ca0   r14 = 0x00007efe8bfa4c80
[task 2021-03-24T16:51:03.657Z] 16:51:03     INFO -     r15 = 0x00007efe8b482760   rip = 0x00007efe919f4d93
[task 2021-03-24T16:51:03.658Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.658Z] 16:51:03     INFO - 10  libxul.so!MessageLoop::RunInternal() [message_loop.cc:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 335 + 0x17]
[task 2021-03-24T16:51:03.658Z] 16:51:03     INFO -     rbx = 0x00007efe84ecdc68   rbp = 0x00007efe84ecdc00
[task 2021-03-24T16:51:03.658Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdbd0   r12 = 0x00007efe84ecdc68
[task 2021-03-24T16:51:03.659Z] 16:51:03     INFO -     r13 = 0x00007efe8b482760   r14 = 0x00007efe84ecdc10
[task 2021-03-24T16:51:03.659Z] 16:51:03     INFO -     r15 = 0x00007efe8b4827a8   rip = 0x00007efe9199d267
[task 2021-03-24T16:51:03.659Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO - 11  libxul.so!MessageLoop::Run() [message_loop.cc:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 310 + 0x8]
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     rbx = 0x00007efe84ecdc68   rbp = 0x00007efe84ecdc40
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdc10   r12 = 0x00007efe84ecdc68
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     r13 = 0x00007efe8b482760   r14 = 0x00007efe84ecdc10
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     r15 = 0x00007efe8b4827a8   rip = 0x00007efe9199d1c1
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO - 12  libxul.so!nsThread::ThreadFunc(void*) [nsThread.cpp:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 391 + 0x8]
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     rbx = 0x000000000000000f   rbp = 0x00007efe84ecde60
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdc50   r12 = 0x00007efe84ecdc68
[task 2021-03-24T16:51:03.660Z] 16:51:03     INFO -     r13 = 0x00007efe8b482760   r14 = 0x00007efe8b4827b0
[task 2021-03-24T16:51:03.661Z] 16:51:03     INFO -     r15 = 0x00007efe8b4827a8   rip = 0x00007efe9138beb9
[task 2021-03-24T16:51:03.661Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.661Z] 16:51:03     INFO - 13  libnspr4.so!_pt_root [ptthread.c:3afbfb073c46dd4597a862cc683d5e0fbcfdd90d : 201 + 0x7]
[task 2021-03-24T16:51:03.661Z] 16:51:03     INFO -     rbx = 0x00007efe8b4745e0   rbp = 0x00007efe84ecdeb0
[task 2021-03-24T16:51:03.662Z] 16:51:03     INFO -     rsp = 0x00007efe84ecde70   r12 = 0x00007efe84ece630
[task 2021-03-24T16:51:03.662Z] 16:51:03     INFO -     r13 = 0x0000000000000002   r14 = 0x0000000000000e2b
[task 2021-03-24T16:51:03.662Z] 16:51:03     INFO -     r15 = 0x00007efe84ece700   rip = 0x00007efea339b622
[task 2021-03-24T16:51:03.663Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.663Z] 16:51:03     INFO - 14  libpthread.so.0 + 0x76db
[task 2021-03-24T16:51:03.663Z] 16:51:03     INFO -     rbx = 0x0000000000000000   rbp = 0x0000000000000000
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdec0   r12 = 0x00007efe84ecdf80
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO -     r13 = 0x0000000000000000   r14 = 0x00007efe8b4745e0
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO -     r15 = 0x00007ffea5cd9a80   rip = 0x00007efea2fac6db
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO -     Found by: call frame info
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO - 15  [stack] + 0x1ca80
[task 2021-03-24T16:51:03.664Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdf00   rip = 0x00007ffea5cd9a80
[task 2021-03-24T16:51:03.665Z] 16:51:03     INFO -     Found by: stack scanning
[task 2021-03-24T16:51:03.665Z] 16:51:03     INFO - 16  libc.so.6 + 0x121a3f
[task 2021-03-24T16:51:03.665Z] 16:51:03     INFO -     rsp = 0x00007efe84ecdf80   rip = 0x00007efea2192a3f
[task 2021-03-24T16:51:03.665Z] 16:51:03     INFO -     Found by: stack scanning
[task 2021-03-24T16:51:03.666Z] 16:51:03     INFO - ```

During:

[task 2021-03-24T16:39:20.761Z] 16:39:20 INFO - TEST-START | dom/media/webrtc/tests/mochitests/test_fingerprinting_resistance.html

MOZ_CRASH("can't mprotect()"); while trying to set one of the “sentinel” guard pages before/after the shmem contents to PROT_NONE, which suggests that it exceeded the limit of 64k virtual memory areas. This is confirmed by the minidump: 65532 lines in the MD_LINUX_MAPS stream. We'd need to get rid of those guard pages, which we don't use on non-debug builds if I recall correctly (and I don't know if they've ever actually caught any bugs); I'd been thinking of doing this anyway to reduce the code complexity.

It's also a little worrying that we're only a 3x safety factor from having a crash that we can't do anything about; if shared memory areas were tagged with strings identifying the users we could at least see what subsystem may need to economize.

Crash Signature: [@ mozilla::ipc::SharedMemory::SystemProtect(char*, unsigned long, int)] → [@ mozilla::ipc::SharedMemory::SystemProtect]
Summary: Intermittent PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::ipc::SharedMemory::SystemProtect(char*, unsigned long, int)] → Intermittent PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::ipc::SharedMemory::SystemProtect]
You need to log in before you can comment on or make changes to this bug.