1153683 - mozjemalloc crashes on FreeBSD in malloc_init()

Status: NEW

(Core :: Memory Allocator, defect)

Description

[Jan Beich]

Probably needs to use _pthread_mutex_init_calloc_cb like jemalloc3 to avoid infinite recursion during initialization when using pthread mutexes.

  * thread #1: tid = 100641, 0x0000000000416756 firefox`malloc_init [inlined] malloc_mutex_lock(mutex=0x0000000000424b78) + 7 at jemalloc.c:1669, stop reason = invalid address (fault address: 0x7fffdfffee98)
      frame #0: 0x0000000000416756 firefox`malloc_init [inlined] malloc_mutex_lock(mutex=0x0000000000424b78) + 7 at jemalloc.c:1669
     1666 #elif defined(MOZ_MEMORY_DARWIN)
     1667         OSSpinLockLock(&mutex->lock);
     1668 #elif defined(MOZ_MEMORY)
  -> 1669         pthread_mutex_lock(mutex);
     1670 #else
     1671         if (isthreaded)
     1672                 _SPINLOCK(&mutex->lock);
  (lldb) bt
  * thread #1: tid = 100641, 0x0000000000416756 firefox`malloc_init [inlined] malloc_mutex_lock(mutex=0x0000000000424b78) + 7 at jemalloc.c:1669, stop reason = invalid address (fault address: 0x7fffdfffee98)
    * frame #0: 0x0000000000416756 firefox`malloc_init [inlined] malloc_mutex_lock(mutex=0x0000000000424b78) + 7 at jemalloc.c:1669
      frame #1: 0x000000000041674f firefox`malloc_init [inlined] malloc_init_hard at jemalloc.c:5552
      frame #2: 0x000000000041674f firefox`malloc_init + 31 at jemalloc.c:5528
      frame #3: 0x0000000000417100 firefox`je_calloc(num=1, size=1016) + 16 at jemalloc.c:6330
      frame #4: 0x0000000800d5b4da
      frame #5: 0x0000000800d5c6e5
      frame #6: 0x0000000800d5978a
      frame #7: 0x000000000041675b firefox`malloc_init [inlined] malloc_mutex_lock(mutex=<unavailable>) + 43 at jemalloc.c:1669
      frame #8: 0x000000000041674f firefox`malloc_init [inlined] malloc_init_hard at jemalloc.c:5552
      frame #9: 0x000000000041674f firefox`malloc_init + 31 at jemalloc.c:5528
      ...

Comment 1

[Jan Beich]

Martin, does --enable-jemalloc=moz crash on NetBSD as well?

Comment 2

[Martin Husemann]

Indeed, FF51 crashes at startup (but it looked different at a quick glance, will investigate)

Comment 3

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: use spinlocks on FreeBSD to avoid recursion.

Review commit: https://reviewboard.mozilla.org/r/138900/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138900/

Comment 4

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: unbreak NO_TLS for other POSIX platforms.

Review commit: https://reviewboard.mozilla.org/r/138902/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138902/

Comment 5

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: track MADV_FREE'd pages on all BSDs.

Review commit: https://reviewboard.mozilla.org/r/138904/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138904/

Comment 6

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: FreeBSD can use its own alignment constants.

Review commit: https://reviewboard.mozilla.org/r/138906/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138906/

Comment 7

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: FreeBSD supports TLS on all archs.

Review commit: https://reviewboard.mozilla.org/r/138908/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138908/

Comment 8

[Jan Beich]

Attachment: Bug 1153683 - mozjemalloc: support aarch64/mips64/powerpc64/riscv on FreeBSD.

Review commit: https://reviewboard.mozilla.org/r/138910/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/138910/

Comment 9

[Jan Beich]

Comment on attachment 8867380 [details]
Bug 1153683 - mozjemalloc: use spinlocks on FreeBSD to avoid recursion.

https://reviewboard.mozilla.org/r/138900/#review142236

Comment 10

[Jan Beich]

Comment on attachment 8867380 [details]
Bug 1153683 - mozjemalloc: use spinlocks on FreeBSD to avoid recursion.

https://reviewboard.mozilla.org/r/138900/#review142236

Not enough yet as it works but very often crashes as

```
Fatal error 'thread 0x833f24800 exits with resources held!' at line 316 in file /usr/src/lib/libthr/thread/thr_exit.c (errno = 0)
Redirecting call to abort() to mozalloc_abort

Process 26790 stopped
* thread #78, stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
    frame #0: firefox`mozalloc_abort(msg=<unavailable>) at mozalloc_abort.cpp:33
   30   #ifdef MOZ_WIDGET_ANDROID
   31       abortThroughJava(msg);
   32   #endif
-> 33       MOZ_CRASH();
   34   }
   35
   36   #ifdef MOZ_WIDGET_ANDROID
(lldb) bt
* thread #78, stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
  * frame #0: firefox`mozalloc_abort(msg=<unavailable>) at mozalloc_abort.cpp:33
    frame #1: firefox`::abort() at mozalloc_abort.cpp:80
    frame #2: libthr.so.3`_thread_exitf(fname=<unavailable>, lineno=<unavailable>, fmt=<unavailable>) at thr_exit.c:194
    frame #3: libthr.so.3`exit_thread at thr_exit.c:316
    frame #4: libthr.so.3`thread_unwind_stop(version=<unavailable>, actions=<unavailable>, exc_class=<unavailable>, exc_obj=<unavailable>, context=<unavailable>, stop_parameter=<unavailable>) at thr_exit.c:160
    frame #5: libgcc_s.so.1`unwind_phase2_forced(uc=<unavailable>, cursor=<unavailable>, exception_object=<unavailable>, stop=<unavailable>, stop_parameter=<unavailable>) at UnwindLevel1.c:331
    frame #6: libgcc_s.so.1`_Unwind_ForcedUnwind(exception_object=0x0000000833f24a30, stop=(libthr.so.3`thread_unwind_stop at thr_exit.c:137), stop_parameter=0x0000000000000000) at UnwindLevel1.c:414
    frame #7: libthr.so.3`_pthread_exit_mask [inlined] _Unwind_ForcedUnwind(ex=<unavailable>, stop_func=<unavailable>, stop_arg=<unavailable>) at thr_exit.c:108
    frame #8: libthr.so.3`_pthread_exit_mask [inlined] thread_unwind at thr_exit.c:173
    frame #9: libthr.so.3`_pthread_exit_mask(status=<unavailable>, mask=<unavailable>) at thr_exit.c:255
    frame #10: libthr.so.3`_pthread_exit(status=<unavailable>) at thr_exit.c:207
    frame #11: libthr.so.3`thread_start(curthread=0x0000000833f24800) at thr_create.c:289
```

https://github.com/freebsd/freebsd/blob/c0f92af24002/lib/libthr/thread/thr_exit.c#L316

Comment 11

[Mike Hommey [:glandium]]

Bug 1365191 has now landed that removed the FreeBSD code (!MOZ_MEMORY). I believe support for FreeBSD code should be more close to support for Linux (using pthread_mutex instead of spinlock) with a workaround for the initialization infinite recursion (probably using _pthread_mutex_init_calloc_cb).

Comment 12

[devnexen]

Hello sorry if my comment is irrelevant but I was thinking, for all the calls of pthread_mutex_init with no attributes, adding explicit PTHREAD_MUTEX_NORMAL attribute type which is the default on Linux but not on *BSD ?