1658963 - No symbols for libxul.so in assertion stack in mochitest-chrome debug run on Linux

Status: RESOLVED WORKSFORME

(Toolkit :: Crash Reporting, defect, P3)

Description

[Markus Stange [:mstange]]

This try job encountered an assertion failure, but the printed stack does not have any symbols for libxul:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312944925&repo=try&lineNumber=37654

This is a "Linux 18.04 x64 debug Mochitests test-linux1804-64/debug-mochitest-devtools-chrome-e10s dt3" job.

(Luckily, the same assertion failure happened on macOS, with intact symbols:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312945260&repo=try&lineNumber=38087 )

Extract from the log, for posterity:

task 2020-08-13T18:34:16.713Z] 18:34:16     INFO - GECKO(4194) | Assertion failure: aInstanceSize == 0 || entry->GetClassSize() == aInstanceSize (Mismatched sizes were recorded in the memory leak logging table. The usual cause of this is having a templated class that uses MOZ_COUNT_{C,D}TOR in the constructor or destructor, respectively. As a workaround, the MOZ_COUNT_{C,D}TOR calls can be moved to a non-templated base class. Another possible cause is a runnable with an mName that matches another refcounted class.), at /builds/worker/checkouts/gecko/xpcom/base/nsTraceRefcnt.cpp:320
[task 2020-08-13T18:34:16.715Z] 18:34:16     INFO - GECKO(4194) | #01: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x40552aa]
[task 2020-08-13T18:34:16.717Z] 18:34:16     INFO - GECKO(4194) | #02: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4f0aee4]
[task 2020-08-13T18:34:16.718Z] 18:34:16     INFO - GECKO(4194) | #03: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4ff06ea]
[task 2020-08-13T18:34:16.720Z] 18:34:16     INFO - GECKO(4194) | #04: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4f00a39]
[task 2020-08-13T18:34:16.721Z] 18:34:16     INFO - GECKO(4194) | #05: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4f0010b]
[task 2020-08-13T18:34:16.722Z] 18:34:16     INFO - GECKO(4194) | #06: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4efff7b]
[task 2020-08-13T18:34:16.724Z] 18:34:16     INFO - GECKO(4194) | #07: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4fecbd0]
[task 2020-08-13T18:34:16.724Z] 18:34:16     INFO - GECKO(4194) | #08: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4febfc2]
[task 2020-08-13T18:34:16.725Z] 18:34:16     INFO - GECKO(4194) | #09: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x4febc21]
[task 2020-08-13T18:34:16.726Z] 18:34:16     INFO - GECKO(4194) | #10: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x500d913]
[task 2020-08-13T18:34:16.742Z] 18:34:16     INFO - GECKO(4194) | #11: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x5018118]
[task 2020-08-13T18:34:16.745Z] 18:34:16     INFO - GECKO(4194) | #12: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x5029a1e]
[task 2020-08-13T18:34:16.746Z] 18:34:16     INFO - GECKO(4194) | #13: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x40fbc9f]
[task 2020-08-13T18:34:16.747Z] 18:34:16     INFO - GECKO(4194) | #14: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x40ffb23]
[task 2020-08-13T18:34:16.751Z] 18:34:16     INFO - GECKO(4194) | #15: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x47307fb]
[task 2020-08-13T18:34:16.752Z] 18:34:16     INFO - GECKO(4194) | #16: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x46ddcaf]
[task 2020-08-13T18:34:16.759Z] 18:34:16     INFO - GECKO(4194) | #17: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x46ddc09]
[task 2020-08-13T18:34:16.762Z] 18:34:16     INFO - GECKO(4194) | #18: ??? [/builds/worker/workspace/build/application/firefox/libxul.so + 0x40f8fcc]
[task 2020-08-13T18:34:16.798Z] 18:34:16     INFO - GECKO(4194) | #19: _pt_root [nsprpub/pr/src/pthreads/ptthread.c:204]
[task 2020-08-13T18:34:16.802Z] 18:34:16     INFO -  fix-stacks error: failed to read breakpad symbols dir `/builds/worker/workspace/build/symbols/libpthread.so.0` for `/lib/x86_64-linux-gnu/libpthread.so.0`
[task 2020-08-13T18:34:16.803Z] 18:34:16     INFO -  fix-stacks note:  this is expected and harmless for system libraries on debug automation runs
[task 2020-08-13T18:34:16.804Z] 18:34:16     INFO - GECKO(4194) | #20: ??? [/lib/x86_64-linux-gnu/libpthread.so.0 + 0x76db]
[task 2020-08-13T18:34:16.807Z] 18:34:16     INFO -  fix-stacks error: failed to read breakpad symbols dir `/builds/worker/workspace/build/symbols/libc.so.6` for `/lib/x86_64-linux-gnu/libc.so.6`
[task 2020-08-13T18:34:16.808Z] 18:34:16     INFO -  fix-stacks note:  this is expected and harmless for system libraries on debug automation runs
[task 2020-08-13T18:34:16.811Z] 18:34:16     INFO - GECKO(4194) | #21: clone [/lib/x86_64-linux-gnu/libc.so.6 + 0x121a3f]
[task 2020-08-13T18:34:16.811Z] 18:34:16     INFO - GECKO(4194) | #22: ??? (???:???)
[task 2020-08-13T18:34:17.677Z] 18:34:17     INFO - GECKO(4194) | ExceptionHandler::GenerateDump cloned child 4728
[task 2020-08-13T18:34:17.678Z] 18:34:17     INFO - GECKO(4194) | ExceptionHandler::SendContinueSignalToChild sent continue signal to child
[task 2020-08-13T18:34:17.679Z] 18:34:17     INFO - GECKO(4194) | ExceptionHandler::WaitForContinueSignal waiting for continue signal...
[task 2020-08-13T18:34:17.962Z] 18:34:17     INFO - GECKO(4194) | Exiting due to channel error.

Notably, fix-stacks prints warnings about libpthread.so.0 and about libc.so.6, but not about libxul.so.

Comment 1

[Nicholas Nethercote [inactive]]

The two notable things in the output:

• These stack traces start with "??? [" (note the space before the '[') which is output from fix-stacks.
• There are no complaints about libxul being missing.

These two things indicate that libxul was successfully read by fix-stacks, but (for whatever reason) it failed to find source location info for these addresses. There's no obvious reason why, unless we've suddenly started using separate debuginfo files for Linux on CI, but that seems unlikely.

Comment 2

[Gabriele Svelto [:gsvelto]]

There's been issues with missing symbols for various builds, they should be fixed now. Unfortunately we don't have a bug where the fix was tracked so I can't dup against that.

Comment 3

[Nicholas Nethercote [inactive]]

Let's close this, then. We can reopen later if necessary.