1402647 - memory/build/malloc_decls.h:48:13: error: '::malloc' has not been declared (--disable-jemalloc)

Status: RESOLVED FIXED

(Core :: Memory Allocator, defect)

Description

[Jan Beich]

mozjemalloc isn't supported on many Tier3 platforms, so it's disabled by default.

$ c++ -v
FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
Target: x86_64-unknown-freebsd12.0
Thread model: posix
InstalledDir: /usr/bin

$ MALLOC_CONF=stats_print:true /usr/bin/true 2>&1 | head -2
___ Begin jemalloc statistics ___
Version: 5.0.1-0-g896ed3a8b3f41998d4fb4d625d30ac63ef2d51fb

$ ./mach bootstrap
$ ./mach build
[...]
In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:48:1: error: no member named 'malloc' in the global namespace; did you mean simply
      'malloc'?
MALLOC_DECL(malloc, void *, size_t)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:15:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:48:13: note: 'malloc' declared here
MALLOC_DECL(malloc, void *, size_t)
            ^
memory/build/malloc_decls.h:49:1: error: no member named 'calloc' in the global namespace; did you mean simply
      'calloc'?
MALLOC_DECL(calloc, void *, size_t, size_t)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:15:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:49:13: note: 'calloc' declared here
MALLOC_DECL(calloc, void *, size_t, size_t)
            ^
memory/build/malloc_decls.h:50:1: error: no member named 'realloc' in the global namespace; did you mean simply
      'realloc'?
MALLOC_DECL(realloc, void *, void *, size_t)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:15:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:50:13: note: 'realloc' declared here
MALLOC_DECL(realloc, void *, void *, size_t)
            ^
memory/build/malloc_decls.h:51:1: error: no member named 'free' in the global namespace; did you mean simply
      'free'?
MALLOC_DECL(free, void, void *)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:15:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:51:13: note: 'free' declared here
MALLOC_DECL(free, void, void *)
            ^
memory/build/malloc_decls.h:52:1: error: no member named 'memalign' in the global namespace; did you mean
      simply 'memalign'?
MALLOC_DECL(memalign, void *, size_t, size_t)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:15:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:52:13: note: 'memalign' declared here
MALLOC_DECL(memalign, void *, size_t, size_t)
            ^
5 errors generated.

Also affects OpenBSD:
http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/2215/steps/build/logs/stdio

Comment 1

[Jan Beich]

Obviously, the above Clang suggestion is bogus:

In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:48:1: error: all paths through this function will call
      itself [-Werror,-Winfinite-recursion]
MALLOC_DECL(malloc, void *, size_t)
^
memory/build/fallback.cpp:14:3: note: expanded from macro 'MALLOC_DECL'
  { \
  ^
In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:49:1: error: all paths through this function will call
      itself [-Werror,-Winfinite-recursion]
MALLOC_DECL(calloc, void *, size_t, size_t)
^
memory/build/fallback.cpp:14:3: note: expanded from macro 'MALLOC_DECL'
  { \
  ^
In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:50:1: error: all paths through this function will call
      itself [-Werror,-Winfinite-recursion]
MALLOC_DECL(realloc, void *, void *, size_t)
^
memory/build/fallback.cpp:14:3: note: expanded from macro 'MALLOC_DECL'
  { \
  ^
In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:51:1: error: all paths through this function will call
      itself [-Werror,-Winfinite-recursion]
MALLOC_DECL(free, void, void *)
^
memory/build/fallback.cpp:14:3: note: expanded from macro 'MALLOC_DECL'
  { \
  ^
In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:18:
memory/build/malloc_decls.h:52:1: error: all paths through this function will call
      itself [-Werror,-Winfinite-recursion]
MALLOC_DECL(memalign, void *, size_t, size_t)
^
memory/build/fallback.cpp:14:3: note: expanded from macro 'MALLOC_DECL'
  { \
  ^
5 errors generated.

Comment 2

[Mike Hommey [:glandium]]

Is MALLOC_H undefined?

Comment 3

[Jan Beich]

MALLOC_H is <malloc_np.h> on FreeBSD and declares malloc_usable_size() + jemalloc-specific extensions (unprefixed). It doesn't bootleg <stdlib.h>.

MALLOC_H is not defined on DragonFly and OpenBSD.

https://github.com/freebsd/freebsd/commit/2ffad81b17575
https://github.com/DragonFlyBSD/DragonFlyBSD/commit/02b66c54cac9
https://github.com/openbsd/src/commit/d88f57029e5a

Comment 4

[Jan Beich]

Explicitly adding #include <stdlib.h> to memory/build/fallback.cpp isn't enough:

In file included from objdir/memory/build/Unified_cpp_memory_build0.cpp:2:
In file included from memory/build/fallback.cpp:19:
memory/build/malloc_decls.h:52:1: error: no member named 'memalign' in the
      global namespace; did you mean simply 'memalign'?
MALLOC_DECL(memalign, void *, size_t, size_t)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memory/build/fallback.cpp:16:12: note: expanded from macro 'MALLOC_DECL'
    return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
           ^~
memory/build/malloc_decls.h:52:13: note: 'memalign' declared here
MALLOC_DECL(memalign, void *, size_t, size_t)
            ^
1 error generated.

memalign() is Linux-specific extension. Other systems use aligned_alloc (C11/C++17), _aligned_malloc (Windows) or posix_memalign (any Unix). Upstream jemalloc doesn't provide memalign() unless system malloc library does.

Comment 5

[Jan Beich]

Also, MALLOC_FUNCS_JEMALLOC is confusingly named. No upstream jemalloc version supports those.

Comment 6

[Jan Beich]

(In reply to Jan Beich from comment #3)
> MALLOC_H is not defined on DragonFly and OpenBSD.

Doh, MALLOC_H is <sys/malloc.h> on DragonFly and declares kernel malloc. For userland it's mostly a nop bootlegging some types.

Comment 7

[Petr Sumbera]

For record. On Solaris I see this:

 0:16.42 gmake[5]: Entering directory '/scratch/firefox/obj-x86_64-pc-solaris2.11/memory/build'
 0:16.44 /usr/bin/g++ -std=gnu++11 -o Unified_cpp_memory_build0.o -c -I/scratch/firefox/obj-x86_64-pc-solaris2.11/dist/system_wrappers -include /scratch/firefox/config/gcc_hidden.h -DNDEBUG=1 -DTRIMMED=1 -DMOZ_MEMORY_IMPL -DMOZ_HAS_MOZGLUE -I/scratch/firefox/memory/build -I/scratch/firefox/obj-x86_64-pc-solaris2.11/memory/build -I/scratch/firefox/obj-x86_64-pc-solaris2.11/dist/include -I/scratch/firefox/obj-x86_64-pc-solaris2.11/dist/include/nspr -I/scratch/firefox/obj-x86_64-pc-solaris2.11/dist/include/nss  -fPIC  -DMOZILLA_CLIENT -include /scratch/firefox/obj-x86_64-pc-solaris2.11/mozilla-config.h -MD -MP -MF .deps/Unified_cpp_memory_build0.o.pp  -Wall -Wc++11-compat -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++14-compat -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wformat -fno-exceptions -fno-strict-aliasing -fno-rtti -fno-exceptions -fno-math-errno -pthread -pipe  -g -O -fno-omit-frame-pointer   -Wno-unused-function -Wno-error=uninitialized
   /scratch/firefox/obj-x86_64-pc-solaris2.11/memory/build/Unified_cpp_memory_build0.cpp
 0:16.51 In file included from /scratch/firefox/obj-x86_64-pc-solaris2.11/memory/build/Unified_cpp_memory_build0.cpp:2:0:
 0:16.51 /scratch/firefox/memory/build/malloc_decls.h: In static member function 'static void* SystemMalloc::memalign(std::size_t, std::size_t)':
 0:16.51 /scratch/firefox/memory/build/fallback.cpp:15:12: error: '::memalign' has not been declared
 0:16.52      return ::name(ARGS_HELPER(ARGS, ##__VA_ARGS__)); \
 0:16.52             ^
 0:16.52 /scratch/firefox/memory/build/malloc_decls.h:52:1: note: in expansion of macro 'MALLOC_DECL'
 0:16.52  MALLOC_DECL(memalign, void *, size_t, size_t)
 0:16.52  ^

Comment 8

[Mike Hommey [:glandium]]

Attachment: Bug 1402647 - Add missing stdlib.h header for system allocator.

Review commit: https://reviewboard.mozilla.org/r/183420/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/183420/

Comment 9

[Mike Hommey [:glandium]]

Attachment: Bug 1402647 - Add a memalign implementation on platforms that don't have one.

Review commit: https://reviewboard.mozilla.org/r/183422/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/183422/

Comment 10

[Jan Beich]

Comment on attachment 8912046 [details]
Bug 1402647 - Add a memalign implementation on platforms that don't have one.

I confirm, this unbreaks build on FreeBSD.

Comment 11

[Nicholas Nethercote [inactive]]

Comment on attachment 8912045 [details]
Bug 1402647 - Add missing stdlib.h header for system allocator.

https://reviewboard.mozilla.org/r/183420/#review188584

Comment 12

[Nicholas Nethercote [inactive]]

Comment on attachment 8912046 [details]
Bug 1402647 - Add a memalign implementation on platforms that don't have one.

https://reviewboard.mozilla.org/r/183422/#review188586

::: memory/build/fallback.cpp:20
(Diff revision 1)
> +{
> +#ifdef XP_WIN
> +  return _aligned_malloc(aSize, aAlignment);
> +#else
> +  void* ret;
> +  if (posix_memalign(&ret, aAlignment, aSize)) {

I'd add '!= 0' to the condition.

Comment 13

[Mike Hommey [:glandium]]

Comment on attachment 8912046 [details]
Bug 1402647 - Add a memalign implementation on platforms that don't have one.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/183422/diff/1-2/

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b29d40633a3a
Add missing stdlib.h header for system allocator. r=njn
https://hg.mozilla.org/integration/autoland/rev/4735518e975a
Add a memalign implementation on platforms that don't have one. r=njn

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b29d40633a3a
https://hg.mozilla.org/mozilla-central/rev/4735518e975a