973310 - Bug 967556 (and 748598) broke OpenBSD "No memalign implementation found"

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Landry Breuil (:gaston)]

Cf bug 748598 comment 64 and logs at http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/1034/steps/build/logs/stdio

Comment 1

[Landry Breuil (:gaston)]

After digging a bit with :mwu on irc, it looks like on OpenBSD the autoconf test for posix_memalign (also used for valloc and strndup (which we have)) is failing weirdly with clang or gcc 4.6, and behaving fine with base gcc 4.2 (which i of course cant use for mozilla)

=============================
configure:14015: checking for posix_memalign
configure:14047: clang++ -o conftest  -fno-exceptions -fno-strict-aliasing -fno-rtti -fno-exceptions -fno-math-errno -std=gnu++0x -pthread -Qunused-arguments   -I/usr/X11R6/include -pthread -Wl,--no-keep-memory -Wl,-z,noexecstack -Wl,-z,text conftest.C  1>&5
/tmp/conftest-Sm9eT9.o(.text+0x10): In function `main':
: undefined reference to `innocuous_posix_memalign'
clang-3.3: error: linker command failed with exit code 1 (use -v to see invocation)
configure: failed program was:
#line 14020 "configure"
#define posix_memalign innocuous_posix_memalign
#include "confdefs.h"
/* System header to define __stub macros and hopefully few prototypes,
    which can conflict with char posix_memalign(); below.  */
#include <assert.h>
/* Override any gcc2 internal prototype to avoid an error.  */
#ifdef __cplusplus
extern "C"
#endif
/* We use char because int might match the return type of a gcc2
    builtin and then its argument prototype would still apply.  */
char posix_memalign();

int main() {

/* The GNU C library defines this for functions which it implements
    to always fail with ENOSYS.  Some functions are actually named
    something starting with __ and the normal name is an alias.  */
#if defined (__stub_posix_memalign) || defined (__stub___posix_memalign)
choke me
#else
posix_memalign();
#endif

; return 0; }
=============================

(so those tests failing are something wrong, for a start)

But strangely, this doesnt seem to do a lot, since there's no HAVE_POSIX_MEMALIGN #define nor in generated mozilla-config.h nor anywhere else in tree (besides a strange one in js/src on weird os/archs, cf http://mxr.mozilla.org/mozilla-central/ident?i=HAVE_POSIX_MEMALIGN)

MOZ_MEMORY doesnt look defined on OpenBSD either (nor MOZ_MEMORY_BSD), since we dont use jemalloc.

$grep MOZ_MEMORY /usr/obj/m-c/*         
/usr/obj/m-c/config.status:    (''' MOZ_MEMORY ''', r'''  '''),

So the new codepath used in memory/mozalloc/VolatileBufferFallback.cpp since bug 748598 falls on the #error side..

Maybe HAVE_POSIX_MEMALIGN should be defined somewhere by the autoconf test ?

Comment 2

[Michael Wu [:mwu]]

HAVE_POSIX_MEMALIGN does get defined if the test passes, though the path by which that happens is a little weird.

For reference, this is how things fail:

    [18:46] dawn:/tmp/ $CC=clang make t
    clang -O2 -pipe    -o t t.c
    /tmp/t-OBXiUn.o(.text+0x7): In function `main':
    : undefined reference to `innocuous_posix_memalign'
    clang-3.3: error: linker command failed with exit code 1 (use -v to see invocation) 

It looks like mm_malloc.h defines a function that uses posix_memalign, and our #define tricks from bug 967556 breaks that. (not that it was likely to be working before, though)

Comment 3

[Landry Breuil (:gaston)]

Commenting out build/autoconf/hotfixes.m4 inclusion in aclocal.m4

-builtin(include, build/autoconf/hotfixes.m4)dnl
+dnl builtin(include, build/autoconf/hotfixes.m4)dnl

makes configure happier for me, be it with clang 3.3 or gcc 4.6:

checking for strndup... yes
checking for posix_memalign... yes
checking for memalign... no
checking for valloc... yes

And HAVE_POSIX_MEMALIGN is defined:
/usr/obj/m-c/config.status:    (''' HAVE_POSIX_MEMALIGN ''', ' 1 '),

I'll let my full build proceed to see if the actual code in memory/mozalloc/VolatileBufferFallback.cpp compiles, but that is probably the cause, fallout of bug 967556 and not 748598 as i originally thought (well, that's the one exposing the breakage)

Comment 4

[Landry Breuil (:gaston)]

And my build succeeded, so adjusting dependencies to match reality. Mike, any hint on how to properly fix this ?

Comment 5

[Landry Breuil (:gaston)]

Mike ? I'd like to get my builds green again, to ensure no other breakage like for example bug 974272 pile on top...

Comment 6

[Mike Hommey [:glandium]]

Check the configure source for the C code it's trying to compile for that test. Paste it here. Then try to compile it.

Comment 7

[Landry Breuil (:gaston)]

(In reply to Mike Hommey [:glandium] from comment #6)
> Check the configure source for the C code it's trying to compile for that
> test. Paste it here. Then try to compile it.

Well, already done that in comment 1, but here is it again.

===========================

[08:12] dawn:/tmp/ $cat t.c                                                                                                               
#line 14020 "configure"
#define posix_memalign innocuous_posix_memalign
#include "confdefs.h"
/* System header to define __stub macros and hopefully few prototypes,
    which can conflict with char posix_memalign(); below.  */
#include <assert.h>
/* Override any gcc2 internal prototype to avoid an error.  */
#ifdef __cplusplus
extern "C"
#endif
/* We use char because int might match the return type of a gcc2
    builtin and then its argument prototype would still apply.  */
char posix_memalign();

int main() {

/* The GNU C library defines this for functions which it implements
    to always fail with ENOSYS.  Some functions are actually named
    something starting with __ and the normal name is an alias.  */
#if defined (__stub_posix_memalign) || defined (__stub___posix_memalign)
choke me
#else
posix_memalign();
#endif

; return 0; }

[08:12] dawn:/tmp/ $CC=clang make t
clang -O2 -pipe    -o t t.c 
configure:14021:10: fatal error: 'confdefs.h' file not found
#include "confdefs.h"
         ^
1 error generated.
*** Error 1 in /tmp (<sys.mk>:85 't')

===========================

- indeed, no confdefs.h here - so if i remove the inclusion....

[08:12] dawn:/tmp/ $CC=clang make t 
clang -O2 -pipe    -o t t.c 
/tmp/t-A3e5rL.o(.text+0x7): In function `main':
: undefined reference to `innocuous_posix_memalign'
clang-3.3: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error 1 in /tmp (<sys.mk>:85 't')


Note that i dont see a #undef posix_memalign as the comment says there should be... not an inclusion of another header ? Should i try with a newer autoconf, as this is apparently an issue with 2.13 ?

Comment 8

[Mike Hommey [:glandium]]

Huh, I missed comment 1. So yeah, the #undef should be there, but you won't find any version of autoconf more recent than 2.13 that works. It might be something fishy in your autoconf 2.13 or m4 (more likely to be a m4 issue).

Comment 9

[Landry Breuil (:gaston)]

Attachment: Add trailing $ to stop matching on EOL

So after a debugging session on irc, it appears our m4'regexp is greedy and matches all the AC_CHECK_FUNC macro content with #include.*, and doesnt stop at the end of the matching line. Adding a $ ensures it stops matching at the EOL, and it does properly insert the #undef right after the matched #include.

Comment 10

[Landry Breuil (:gaston)]

Mike ? I've been running this on my builder the past 3 days to get working builds..

Comment 11

[Landry Breuil (:gaston)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c5dc06a580e1

Was pondering about touching clobber to ensure all builders now take this into account, as this was done in 967556..

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/c5dc06a580e1