Closed Bug 1357301 Opened 7 years ago Closed 7 years ago

Investigate how Jemalloc trunk performs

Categories

(Core :: Memory Allocator, defect)

Product:
Core
Component:
Memory Allocator
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: glandium, Unassigned)

Details

To have an idea where things are going forward with jemalloc, let's already take a look at how the current trunk does, compared to mozjemalloc and jemalloc4. I pushed it to try, we can see from there.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c189a36749a57ce45fba30a653540e7c5bf8e9a9
Can either of you check out how this looks compared to mozjemalloc and jemalloc4 while I'm away? If it's (even) worse than jemalloc4, that would be a strong signal that upgrading jemalloc is going to be a dead end, and we might as well focus back on doing the necessary changes to mozjemalloc to support what we would like from jemalloc4.
Flags: needinfo?(erahm)
Flags: needinfo?(agaynor)
Couple more failures there:

On windows we set a commit and decommit hooks, but it looks like the signatures have changed:
> bool (__cdecl *)(void *,std::size_t,std::size_t,std::size_t,unsigned int)' to 'extent_commit_t (__cdecl *)'
> bool (__cdecl *)(void *,std::size_t,std::size_t,std::size_t,unsigned int)' to 'extent_decommit_t (__cdecl *)'

On mac memory/build/zone.c can't find "jemalloc/internal/jemalloc_internal.h" and the purge hook seems to have gone away:

> memory/build/jemalloc_config.cpp:102:13: error: no member named 'purge' in 'extent_hooks_s'
>        hooks.purge = PurgeHook;

And then a whole lotta crashes:

> Crash dump filename: /tmp/xpc-other-ftWQeP/0fdc5856-feed-a69d-640c-280a05316b75.dmp
> Operating system: Linux
>                   0.0.0 Linux 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016 x86_64
> CPU: x86
>      GenuineIntel family 6 model 62 stepping 4
>      1 CPU
> GPU: UNKNOWN
> Crash reason:  SIGSYS
> Crash address: 0x13e
> Process uptime: not available
> Thread 17 (crashed)
>  0  linux-gate.so + 0x440
>     eip = 0xf77ae440   esp = 0xe08e4350   ebp = 0xe08e43c8   ebx = 0xe08e436c
>     esi = 0xe08e517c   edi = 0x00000000   eax = 0x0000013e   ecx = 0x00000000
>     edx = 0x00000000   efl = 0x00000246
>     Found by: given as instruction pointer in context
>  1  plugin-container!je_arena_choose_impl [jemalloc_internal_inlines_b.h:2b28195c5697 : 25 + 0xa]
>     eip = 0x0805bb6c   esp = 0xe08e43d0   ebp = 0xe08e4408
>     Found by: previous frame's frame pointer
>  2  plugin-container!je_arena_malloc_hard [jemalloc_internal_inlines_b.h:2b28195c5697 : 65 + 0xa]
>     eip = 0x0805d047   esp = 0xe08e4410   ebp = 0xe08e4448   ebx = 0x08092198
>     esi = 0x00000000   edi = 0x0000014c
>     Found by: call frame info
>  3  plugin-container!je_malloc [arena_inlines_b.h:2b28195c5697 : 128 + 0x10]
>     eip = 0x08074325   esp = 0xe08e4450   ebp = 0xe08e44b8   ebx = 0x08092198
>     esi = 0xe08e517c   edi = 0x0000014c
>     Found by: call frame info
>  4  plugin-container!moz_xmalloc [mozalloc.cpp:2b28195c5697 : 83 + 0x9]
>     eip = 0x0804cbaf   esp = 0xe08e44c0   ebp = 0xe08e44d8   ebx = 0x08092198
>     esi = 0x0000014c   edi = 0xe08e458f
>     Found by: call frame info
>  5  libxul.so!locked_register_thread [mozalloc.h:2b28195c5697 : 194 + 0x5]
>     eip = 0xf58a4eee   esp = 0xe08e44e0   ebp = 0xe08e4528   ebx = 0xf7680804
>     esi = 0xeb5336a0   edi = 0xe08e458f
>     Found by: call frame info
>  6  libxul.so!profiler_register_thread [platform.cpp:2b28195c5697 : 2697 + 0xb]
>     eip = 0xf58a501f   esp = 0xe08e4530   ebp = 0xe08e4568   ebx = 0xf7680804
>     esi = 0xeb5336a0   edi = 0xf76bbdd8
>     Found by: call frame info
>  7  libxul.so!base::Thread::ThreadMain [GeckoProfiler.h:2b28195c5697 : 607 + 0x9]
>     eip = 0xf3df554d   esp = 0xe08e4570   ebp = 0xe08e4688   ebx = 0xf7680804
>     esi = 0xeb5336a0   edi = 0xe08e4590
>     Found by: call frame info
>  8  libxul.so!ThreadFunc [platform_thread_posix.cc:2b28195c5697 : 38 + 0x6]
>     eip = 0xf3deaff1   esp = 0xe08e4690   ebp = 0xe08e46a8   ebx = 0x00000000
>     esi = 0x00000000   edi = 0x003d0f00
>     Found by: call frame info
>  9  libpthread-2.23.so + 0x6295
>     eip = 0xf773b295   esp = 0xe08e46b0   ebp = 0xe08e4768   ebx = 0x00000000
>     esi = 0x00000000   edi = 0x003d0f00
>     Found by: call frame info
> 10  libc-2.23.so + 0xe6eee
>     eip = 0xf3444eee   esp = 0xe08e4770   ebp = 0x00000000
>     Found by: previous frame's frame pointer
Thanks for jumping on this Eric. Most of the build failures are innocuous conflicts with dmd or stylo. The macOS 10.7 one looks like it's caused by upstream dropping support for their C11 atomic polyfill.
Interestingly 64-bit linux seems happy, 32-bit is OOMing like there's no tomorrow. I wonder if they have some sort of 32-bit regression, my hunch was some sort of internal leak, but some of the OOMs seem to be the first test so that seems unlikely.
Flags: needinfo?(erahm)
RE: 32-bit oompocalypse

Looking at some of the tests we can see the vsize-max-contiguous is plummeting, my guess is jemalloc trunk has some serious fragmentation issues or is leaking.

> 17:57:54     INFO - GECKO(2696) | MEMORY STAT | vsize 945MB | vsizeMaxContiguous 127MB | residentFast 378MB | heapAllocated 115MB
> 17:57:54     INFO - TEST-OK | devtools/client/memory/test/chrome/test_CensusTreeItem_01.html | took 1480ms
> 17:57:54     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTreeItem_01.html
> 17:57:55     INFO - GECKO(2696) | MEMORY STAT | vsize 958MB | vsizeMaxContiguous 127MB | residentFast 389MB | heapAllocated 117MB
> 17:57:55     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTreeItem_01.html | took 175ms
> 17:57:55     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_01.html
> 17:57:55     INFO - GECKO(2696) | MEMORY STAT | vsize 978MB | vsizeMaxContiguous 121MB | residentFast 402MB | heapAllocated 120MB
> 17:57:55     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_01.html | took 197ms
> 17:57:55     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_02.html
> 17:57:55     INFO - GECKO(2696) | MEMORY STAT | vsize 992MB | vsizeMaxContiguous 70MB | residentFast 415MB | heapAllocated 124MB
> 17:57:55     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_02.html | took 214ms
> 17:57:55     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_03.html
> 17:57:55     INFO - GECKO(2696) | MEMORY STAT | vsize 1014MB | vsizeMaxContiguous 7MB | residentFast 435MB | heapAllocated 127MB
> 17:57:55     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_03.html | took 229ms
> 17:57:55     INFO - TEST-START | devtools/client/memory/test/chrome/test_Heap_01.html

Compare that to mozjemalloc:

> 09:01:22     INFO - TEST-START | devtools/client/memory/test/chrome/test_CensusTreeItem_01.html
> 09:01:23     INFO - GECKO(3508) | MEMORY STAT | vsize 762MB | vsizeMaxContiguous 582MB | residentFast 250MB | heapAllocated 115MB
> 09:01:23     INFO - TEST-OK | devtools/client/memory/test/chrome/test_CensusTreeItem_01.html | took 1445ms
> 09:01:23     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTreeItem_01.html
> 09:01:23     INFO - GECKO(3508) | MEMORY STAT | vsize 766MB | vsizeMaxContiguous 582MB | residentFast 254MB | heapAllocated 118MB
> 09:01:23     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTreeItem_01.html | took 155ms
> 09:01:23     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_01.html
> 09:01:24     INFO - GECKO(3508) | MEMORY STAT | vsize 779MB | vsizeMaxContiguous 582MB | residentFast 267MB | heapAllocated 121MB
> 09:01:24     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_01.html | took 165ms
> 09:01:24     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_02.html
> 09:01:24     INFO - GECKO(3508) | MEMORY STAT | vsize 784MB | vsizeMaxContiguous 582MB | residentFast 273MB | heapAllocated 125MB
> 09:01:24     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_02.html | took 166ms
> 09:01:24     INFO - TEST-START | devtools/client/memory/test/chrome/test_DominatorTree_03.html
> 09:01:24     INFO - GECKO(3508) | MEMORY STAT | vsize 789MB | vsizeMaxContiguous 582MB | residentFast 278MB | heapAllocated 127MB
> 09:01:24     INFO - TEST-OK | devtools/client/memory/test/chrome/test_DominatorTree_03.html | took 196ms
> 09:01:24     INFO - TEST-START | devtools/client/memory/test/chrome/test_Heap_01.html
Comparing the Win64 awsy run vs m-c in perfherder [1] is pretty crazy too:

> windows10-64-vm: Resident Memory opt 		Base 			New 		Delta
> RSS After tabs open [+30s, forced GC] opt 	948883456.00 	< 	15028736000.00 	1483.83%
> RSS After tabs open [+30s] opt 	 	1115672576.00 	< 	15003811840.00 	1244.82%
> RSS After tabs open opt 	 	 	1143963648.00 	< 	14787485696.00 	1192.65%
> RSS Fresh start [+30s] opt 	 	 	367435776.00 	< 	742334464.00 	102.03%
> RSS Fresh start opt 	 	 	 	493412352.00 	< 	740925440.00 	50.16%
> RSS Tabs closed [+30s, forced GC] opt 	518021120.00 	< 	14095405056.00 	2621.01%
> RSS Tabs closed [+30s] opt 	 	 	537153536.00 	< 	14034161664.00 	2512.69%
> RSS Tabs closed opt 	 	 	 	608604160.00 	< 	13915127808.00 	2186.40%

Explicit seemed to regress pretty bad as well in 25-30% range.

[1] https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-central&originalRevision=abe5868346c7&newProject=try&newRevision=af679f73e1752522b17f2fa244af30a848e16668&originalSignature=8fa341bcf132a6b47c5918245af0d74663e940d2&newSignature=8fa341bcf132a6b47c5918245af0d74663e940d2&framework=4
The dev branch may have fixed some issues since my try push. Like https://github.com/jemalloc/jemalloc/issues/766 was fixed after the try push, and it could explain some of the problems.
Per bug 1363992, jemalloc 4 related bugs are now irrelevant.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Flags: needinfo?(agaynor)
You need to log in before you can comment on or make changes to this bug.