1321093 - logalloc make check bustage with gcc 6.2.0

Status: RESOLVED FIXED

(Core :: Memory Allocator, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

https://treeherder.mozilla.org/logviewer.html#?job_id=32036318&repo=try

INFO - make[2]: Entering directory `/home/worker/workspace/build/src/obj-firefox/memory/replace'
INFO - make[3]: Entering directory `/home/worker/workspace/build/src/obj-firefox/memory/replace/logalloc'
INFO - make[4]: Entering directory `/home/worker/workspace/build/src/obj-firefox/memory/replace/logalloc/replay'
INFO - grep "^1 " /home/worker/workspace/build/src/memory/replace/logalloc/replay/replay.log > expected_output.log
INFO - MALLOC_LOG=1 LD_PRELOAD=/home/worker/workspace/build/src/obj-firefox/memory/replace/logalloc/replay/../liblogalloc.so ./logalloc-replay < /home/worker/workspace/build/src/memory/replace/logalloc/replay/replay.log | /home/worker/workspace/build/src/obj-firefox/_virtualenv/bin/python /home/worker/workspace/build/src/memory/replace/logalloc/replay/logalloc_munge.py | diff -w - expected_output.log
INFO - #10 mapped: 2097152; allocated: 30816; waste: 0; dirty: 0; bookkeep: 9280; binunused: 66400
INFO - 1,3c1,5
INFO - < 1 1 malloc(18944)=#1
INFO - < 1 1 malloc(42)=#2
INFO - < 1 1 malloc(24)=#3
INFO - ---
INFO - > 1 1 malloc(42)=#1
INFO - > 1 1 malloc(24)=#2
INFO - > 1 1 free(#1)
INFO - > 1 1 posix_memalign(4096,1024)=#1
INFO - > 1 1 calloc(4,42)=#3
INFO - 5,11c7,10
INFO - < 1 1 posix_memalign(4096,1024)=#2
INFO - < 1 1 calloc(4,42)=#4
INFO - < 1 1 free(#3)
INFO - < 1 1 realloc(#4,84)=#3
INFO - < 1 1 aligned_alloc(512,1024)=#4
INFO - < 1 1 memalign(512,1024)=#5
INFO - < 1 1 valloc(1024)=#6
INFO - ---
INFO - > 1 1 realloc(#3,84)=#2
INFO - > 1 1 aligned_alloc(512,1024)=#3
INFO - > 1 1 memalign(512,1024)=#4
INFO - > 1 1 valloc(1024)=#5
INFO - 13d11
INFO - < 1 1 free(#6)
INFO - 17a16
INFO - > 1 1 free(#1)
INFO - make[4]: *** [check] Error 1
INFO - make[4]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/memory/replace/logalloc/replay'
INFO - make[3]: *** [check] Error 2
INFO - make[3]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/memory/replace/logalloc'
INFO - make[2]: *** [check] Error 2
INFO - make[2]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/memory/replace'
INFO - make[1]: *** [check] Error 2
INFO - make[1]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/memory'
INFO - make: *** [check] Error 2

Comment 1

[Nathan Froyd [:froydnj]]

Not even |diff -u|?  Boo! :)

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1321093 - Avoid logalloc-replay make check failure when libstdc++ allocates memory in some static initializer.

Review commit: https://reviewboard.mozilla.org/r/96380/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/96380/

Comment 3

[Nicholas Nethercote [inactive]]

Comment on attachment 8815479 [details]
Bug 1321093 - Avoid logalloc-replay make check failure when libstdc++ allocates memory in some static initializer.

https://reviewboard.mozilla.org/r/96380/#review96586

::: memory/replace/logalloc/replay/Makefile.in:30
(Diff revision 1)
>  # Test with MALLOC_LOG as a file descriptor number
> -	MALLOC_LOG=1 $(LOGALLOC) ./$(PROGRAM) < $< | $(PYTHON) $(srcdir)/logalloc_munge.py | diff -w - expected_output.log
> +# We filter out anything happening before the first jemalloc_stats (first
> +# command in replay.log) because starting with libstdc++ 5, a static
> +# initializer in the STL allocates memory, which we obviously don't have
> +# in expected_output.log.
> +	MALLOC_LOG=1 $(LOGALLOC) ./$(PROGRAM) < $< | sed -n '/jemalloc_stats/,$$p' | $(PYTHON) $(srcdir)/logalloc_munge.py | diff -w - expected_output.log

Add the -u flag to diff while you're here, as froydnj suggests?

::: memory/replace/logalloc/replay/Makefile.in:34
(Diff revision 1)
> +# in expected_output.log.
> +	MALLOC_LOG=1 $(LOGALLOC) ./$(PROGRAM) < $< | sed -n '/jemalloc_stats/,$$p' | $(PYTHON) $(srcdir)/logalloc_munge.py | diff -w - expected_output.log
>  # Test with MALLOC_LOG as a file name
>  	$(RM) test_output.log
>  	MALLOC_LOG=test_output.log $(LOGALLOC) ./$(PROGRAM) < $<
> -	$(PYTHON) $(srcdir)/logalloc_munge.py < test_output.log | diff -w - expected_output.log
> +	sed -n '/jemalloc_stats/,$$p' test_output.log | $(PYTHON) $(srcdir)/logalloc_munge.py | diff -w - expected_output.log

Ditto.

Comment 4

[Mike Hommey [:glandium]]

I'm not all that convinced about -u... because here's the output with -u with the bug unfixed:

--- -	2016-11-29 23:32:09.252069623 +0000
+++ expected_output.log	2016-11-29 23:31:26.015367543 +0000
@@ -1,17 +1,16 @@
-1 1 malloc(72704)=#1
-1 1 malloc(42)=#2
-1 1 malloc(24)=#3
+1 1 malloc(42)=#1
+1 1 malloc(24)=#2
+1 1 free(#1)
+1 1 posix_memalign(4096,1024)=#1
+1 1 calloc(4,42)=#3
 1 1 free(#2)
-1 1 posix_memalign(4096,1024)=#2
-1 1 calloc(4,42)=#4
-1 1 free(#3)
-1 1 realloc(#4,84)=#3
-1 1 aligned_alloc(512,1024)=#4
-1 1 memalign(512,1024)=#5
-1 1 valloc(1024)=#6
+1 1 realloc(#3,84)=#2
+1 1 aligned_alloc(512,1024)=#3
+1 1 memalign(512,1024)=#4
+1 1 valloc(1024)=#5
 1 1 jemalloc_stats()
-1 1 free(#6)
 1 1 free(#5)
 1 1 free(#4)
 1 1 free(#3)
 1 1 free(#2)
+1 1 free(#1)

It's not convincingly better, is it?

A git diff --color-words=. would be much better, but would require colors to be legible (and would require temporary files). Wdiff would be another better option, but is likely not available on automation, especially on windows.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/739ac5ad4db5
Avoid logalloc-replay make check failure when libstdc++ allocates memory in some static initializer. r=njn

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/739ac5ad4db5