1340576 - mismatched free / delete / delete [] at large

Status: NEW

(Core :: Memory Allocator, defect, P3)

Description

[ISHIKAWA, Chiaki]

Attachment: bug-3x3.txt: classification done by 3 stackframes (for deallocation) and 3 stackframes (for allocation)

I noticed many mismated free / delete / delete [] at large by running C-C TB under valgrind/memcheck during its {make mozmill} test.

But I am sure this is dormant and existing in M-C FF, too.

I am attaching the classification based on 
stacktrace (the last three frame) leading to
free / delete / delete [] ANDv the stacktrace (the last three frame) leading up to the original allocation.

Since the valgrind/memtest run slowed down the test so much, only about 800+ tests among 1200+ tests completed. (Many tests timed out.)
Within that log (log is huge, 256MB or so, and 16+ MB even compression. I can make it available on request. Just tell me where to post.), based on the classification using the above pair of the stacktraces,
I count 66 different cases.
I am uploading these cases.
The listing removed the process number and the hexadecimal numerical address at the beginning of the line reported by valgrind/memcheck.

At least a few of them are related to the pattern reported in
bug 1325470.
(But some are not as is clearly shown in the next upload that shows only the two stacktop frames for classification. I mean there seem to be a few variations: some are reported to be done in libstdc++ but some are reported in basic_string.tcc? I am not libstdc++ expert and so I can't say exactly the differences are. I can see the root cause may be related.
At lest I can say, there are _M_{construct,assign,mutate,erase,erase_aux} variants. _M_erase, _M_erase_aux are involved in the freeing side of the process.)


The version of GCC and kernel used is:
gcc --version
gcc (Debian 6.3.0-6) 6.3.0 20170205
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ishikawa@ip030:/NREF-COMM-CENTRAL/comm-central$ uname -a
Linux ip030 3.19.5 #1 SMP Mon Apr 20 08:50:21 JST 2015 x86_64 GNU/Linux
ishikawa@ip030:/NREF-COMM-CENTRAL/comm-central$ 

I hope someone can look into this issue soon.

TIA

Comment 1

[ISHIKAWA, Chiaki]

Attachment: bug-2x2.txt: classification done by two stackframes (for deallocation) and two stackframes (for allocation)

This should make it clear that
there are
_M_{construct,assign,mutate} variants for the allocation side. 
In *SOME* cases, _M_erase, _M_erase_aux are involved in the deallocation phase of the operation, but not all.

TIA

Comment 2

[ISHIKAWA, Chiaki]

Attachment: pick-mismatched-free-delete.awk

This is the awk script I used to process the logfile from valgrind/memcheck to produce the previous two uploads.
It should be very helpful for quickly homing into the cause of the issue at hand.

Comment 3

[Andrew McCreight [:mccr8]]

IIRC, Julian does Valgrind runs with mismatched allocator checking disabled or something, because it has a lot of false positives.

Comment 4

[ISHIKAWA, Chiaki]

(In reply to Andrew McCreight [:mccr8] from comment #3)
> IIRC, Julian does Valgrind runs with mismatched allocator checking disabled
> or something, because it has a lot of false positives.

Thank you. That is why I put Bug 1325470 (reported by Julian) as See also  bug.
Maybe I should ask for the feedback.

Julian, what course of action should I take from here?

Comment 5

[Julian Seward [:jseward]]

(In reply to ISHIKAWA, Chiaki from comment #4)
> Julian, what course of action should I take from here?

The simple answer is, run with --show-mismatched-frees=no.  Most of
them are false positives caused by inconsistent inlining of malloc
into new vs free into delete.

The more complex answer is, we'd have to look at them on an
individual basis.  Bug 1325470 is an example which Mike Hommey
believes is a real bug.  But those are relatively rare.  Mostly
Valgrind is reporting false positives here.

Comment 6

[ISHIKAWA, Chiaki]

(In reply to Julian Seward [:jseward] from comment #5)

Thank you for the comment.

> (In reply to ISHIKAWA, Chiaki from comment #4)
> > Julian, what course of action should I take from here?
> 
> The simple answer is, run with --show-mismatched-frees=no.  Most of
> them are false positives caused by inconsistent inlining of malloc
> into new vs free into delete.

Oh, tough. Is there something we can do to make this consistent?
- maybe changing the header files where these inlining is done?
  (For example, inside M-C, C-C tree we can disable inlining?)
  Come to think of it, the bulk of issues are reported in libstdc++, that means we need to
  recompile libstdc++ after tweaking it? Hmm.
 
> The more complex answer is, we'd have to look at them on an
> individual basis.  Bug 1325470 is an example which Mike Hommey
> believes is a real bug.  But those are relatively rare.  

This is indeed tough nut to crack, and time-consuming to analyze.

> Mostly Valgrind is reporting false positives here.

This is unfortunate.
But again, if there are ways to make to replacement more consistent, I would not mind spending time
to change headers of C-C, M-C and even try re-compiling libstdc++.
The reason I say this is that at least until last summer, these false positives were not there IIRC.
So this inconsistency could be
 - GCC6 issue, and/or
 - the latest libstdc++.

TIA