1055224 - Linux gtest crash stacks are not symbolized

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Andrew McCreight [:mccr8]]

I added an intentional crash in jemalloc, and the resulting crashes in gtest don't show a useful symbol, outside of jemalloc itself.  (This isn't actually an immediate problem for me, because every other test suite is also crashing, so this is almost certainly just the same issue.)

Thread 0 (crashed)
 0  firefox!jemalloc_crash [jemalloc.c:989fad4ef7ee : 1572 + 0x0]
    rbx = 0x00007fd25c663d78   r12 = 0x0000000000000000
    r13 = 0x0000000000000000   r14 = 0x00007fd24b889000
    r15 = 0x00000147dc206a6e   rip = 0x000000000041467b
    rsp = 0x00007fffbb52a880   rbp = 0x00007fffbb52a890
    Found by: given as instruction pointer in context
 1  firefox!je_realloc [jemalloc.c:989fad4ef7ee : 6438 + 0x6]
    rbx = 0x00007fd24b8238d0   r12 = 0x0000000000000000
    r13 = 0x0000000000000000   r14 = 0x00007fd24b889000
    r15 = 0x00000147dc206a6e   rip = 0x0000000000418055
    rsp = 0x00007fffbb52a8a0   rbp = 0x00007fffbb52a980
    Found by: call frame info
 2  libxul.so + 0x9fad5a
    rbx = 0x00007fd24b8238d0   r12 = 0x0000000000000050
    r13 = 0x0000000000000000   r14 = 0x00007fd24b889000
    r15 = 0x00000147dc206a6e   rip = 0x00007fd2562add5b
    rsp = 0x00007fffbb52a990   rbp = 0x00007fffbb52a9b0
    Found by: call frame info
 3  libxul.so + 0x2339f21
    rip = 0x00007fd257becf22   rsp = 0x00007fffbb52a9c0
    rbp = 0x00007fffbb52a9e0
    Found by: stack scanning
 4  libxul.so + 0x233a51c
    rip = 0x00007fd257bed51d   rsp = 0x00007fffbb52a9f0
    rbp = 0x00007fffbb52aa10
    Found by: stack scanning
 5  libxul.so + 0x233ae3f
    rip = 0x00007fd257bede40   rsp = 0x00007fffbb52aa20
    rbp = 0x00007fffbb52aa30
    Found by: stack scanning
 6  libxul.so + 0x951a21
    rip = 0x00007fd256204a22   rsp = 0x00007fffbb52aa40
    rbp = 0x00007fffbb52ab50
    Found by: stack scanning

Comment 1

[(not currently active) Ted Mielczarek]

The problem here is that we actually build the gtest libxul as part of `make check`:
http://hg.mozilla.org/mozilla-central/annotate/111a1da2a95d/testing/gtest/Makefile.in#l24

...which is after we've run `make buildsymbols`.

The secondary problem we'd have if we fixed that is that for symbolizing assertion stacks we use fix_stack_using_bpsyms, which isn't smart enough to differentiate between symbol files for libraries with the same name:
http://hg.mozilla.org/mozilla-central/annotate/111a1da2a95d/tools/rb/fix_stack_using_bpsyms.py#l88

Comment 2

[Chris Manchester (limited bugmail, email directly)]

These should be in good shape now that we're running gtests from the test package. I'll use this bug for my patches to provide symbols for assertion stacks.

Comment 3

[Chris Manchester (limited bugmail, email directly)]

Attachment: MozReview Request: Bug 1055224 - Run gtest output through a stack fixer. r=ahal

Review commit: https://reviewboard.mozilla.org/r/30995/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30995/

Comment 4

[Andrew Halberstadt [:ahal]]

Comment on attachment 8708136 [details]
MozReview Request: Bug 1055224 - Run gtest output through a stack fixer. r=ahal

https://reviewboard.mozilla.org/r/30995/#review27863

Lgtm, r+ with a few minor nits.

::: testing/gtest/rungtests.py:36
(Diff revision 1)
>                          symbol files for producing stack traces on crash.

Add utility path to the docstring

::: testing/gtest/rungtests.py:124
(Diff revision 1)
>          # mozinfo is not set up properly to detect if ASan is enabled, so just always set these.

Remove this comment

::: testing/mozbase/mozprocess/mozprocess/processhandler.py:16
(Diff revision 1)
> -__all__ = ['ProcessHandlerMixin', 'ProcessHandler']
> +__all__ = ['ProcessHandlerMixin', 'ProcessHandler', 'StreamOutput']

Might as well be consistent and export LogOutput and StoreOutput as well.

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e0163593bc99

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/e0163593bc99