Closed Bug 1381043 Opened 2 years ago Closed 2 years ago

Restore full Rust debug symbols / address dsymutil crashes

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox58 fixed)

RESOLVED FIXED
mozilla58
Tracking Status
firefox58 --- fixed

People

(Reporter: gps, Assigned: glandium)

References

Details

Attachments

(2 files)

This bug is to track undoing the hack in bug 1380381 related to dsymutil crashing on full Rust debug symbols.
bug 1380381 comment 9 had a stack:
[task 2017-07-13T15:09:16.295282Z] 15:09:16     INFO -  #0 0x00000000007a8945 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x7a8945)
[task 2017-07-13T15:09:16.295504Z] 15:09:16     INFO -  #1 0x00000000007a8f96 SignalHandler(int) (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x7a8f96)
[task 2017-07-13T15:09:16.295689Z] 15:09:16     INFO -  #2 0x00007ff047cd17e0 __restore_rt (/lib64/libpthread.so.0+0xf7e0)
[task 2017-07-13T15:09:16.295978Z] 15:09:16     INFO -  #3 0x000000000042e1d8 llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x42e1d8)

(frames 4-16 all seem to be the same as frame 3)

[task 2017-07-13T15:09:16.298978Z] 15:09:16     INFO -  #17 0x000000000042aa4d llvm::dsymutil::(anonymous namespace)::DwarfLinker::link(llvm::dsymutil::DebugMap const&) (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x42aa4d)
[task 2017-07-13T15:09:16.299186Z] 15:09:16     INFO -  #18 0x00000000004285b9 llvm::dsymutil::linkDwarf(llvm::StringRef, llvm::dsymutil::DebugMap const&, llvm::dsymutil::LinkOptions const&) (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x4285b9)
[task 2017-07-13T15:09:16.299354Z] 15:09:16     INFO -  #19 0x000000000041b938 main (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x41b938)
[task 2017-07-13T15:09:16.299533Z] 15:09:16     INFO -  #20 0x00007ff046e90d5d __libc_start_main (/lib64/libc.so.6+0x1ed5d)
[task 2017-07-13T15:09:16.299723Z] 15:09:16     INFO -  #21 0x000000000041b1d9 _start (/home/worker/workspace/build/src/clang/bin/llvm-dsymutil+0x41b1d9)
[task 2017-07-13T15:09:16.299883Z] 15:09:16     INFO -  Stack dump:
[task 2017-07-13T15:09:16.300105Z] 15:09:16     INFO -  0.	Program arguments: /home/worker/workspace/build/src/clang/bin/llvm-dsymutil --arch=x86_64 /home/worker/workspace/build/src/obj-firefox/toolkit/library/XUL

Nathan: do you know anything about this part of LLVM? Do you know anyone who does? Maybe we should just file an LLVM bug with this stack and see if it's obvious to LLVM developers?
Flags: needinfo?(nfroyd)
I hit this again today locally.

MacBook-Pro-104:~ mikeconley$ /usr/bin/dsymutil --version

Apple LLVM 7.0.2 (clang-700.1.81)

I attached lldb, and have the following stack:

(lldb) bt
* thread #1: tid = 0x109b634, 0x0000000100017312 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 1762, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000100017312 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 1762
    frame #1: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #2: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #3: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #4: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #5: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #6: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #7: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #8: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #9: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #10: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #11: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #12: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #13: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #14: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #15: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #16: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #17: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #18: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #19: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #20: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #21: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #22: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #23: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #24: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #25: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #26: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #27: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #28: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #29: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #30: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #31: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #32: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #33: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #34: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #35: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #36: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #37: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #38: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #39: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #40: 0x0000000100017649 dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2585
    frame #41: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #42: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #43: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #44: 0x00000001000176ff dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::lookForDIEsToKeep(llvm::dsymutil::(anonymous namespace)::DwarfLinker::RelocationManager&, llvm::DWARFDebugInfoEntryMinimal const&, llvm::dsymutil::DebugMapObject const&, llvm::dsymutil::(anonymous namespace)::CompileUnit&, unsigned int) + 2767
    frame #45: 0x000000010000a86d dsymutil`llvm::dsymutil::(anonymous namespace)::DwarfLinker::link(llvm::dsymutil::DebugMap const&) + 12061
    frame #46: 0x0000000100030c72 dsymutil`main + 24226
    frame #47: 0x00007fff94e695c9 libdyld.dylib`start + 1
    frame #48: 0x00007fff94e695c9 libdyld.dylib`start + 1

I got a core dump too. I'll see if I can find a place to upload it.
I got a similar crash when trying to build with rustc 1.20 beta on try:
https://public-artifacts.taskcluster.net/DPiAQgT5SNqfF2tlmkG5Lw/0/public/logs/live_backing.log

Considering how deep the frames go, I'm wondering if it's not a stack overflow.
So, my crash happens on this line of code from llvm:
  uint32_t Offset = Die.getOffset() + getULEB128Size(Abbrev->getCode());
with a null deref on Abbrev, which is null.

Adding a null check makes that null deref crash go away (obviously) but I then hit an assert:
  DwarfLinker.cpp:2389: unsigned int llvm::dsymutil::{anonymous}::DwarfLinker::DIECloner::cloneDieReferenceAttribute(llvm::DIE&, const llvm::DWARFDie&, llvm::dsymutil::{anonymous}::DwarfLinker::DIECloner::AttributeSpec, unsigned int, const llvm::DWARFFormValue&, llvm::dsymutil::{anonymous}::CompileUnit&): Assertion `Ref > InputDIE.getOffset()' failed.

If I comment out the assert, it works and goes through without crashing.

I ended up narrowing down that this was coming from libgkrust_gtest.a(gkrust_gtest-037d1f7bf69d09f3.0.o).

Sure enough, if I rebuild the .a excluding that .o, an unmodified dsymutil goes to completion without crashing.

But also, if I remove everything from the objdir except that .a and gtest/XUL, I can reproduce the crash, much faster. I can even narrow down by removing everything from the .a except the gkrust_gtest .o, which allows to publish a testcase that is 210MB uncompressed instead of 5.5GB for an entire objdir. I'll file a llvm bug with that reduced testcase. Note I haven't dug whether llvm or rust is at fault here, which is why I'm wary about patching our llvm-dsymutil with those two changes. Well, the null deref is a no-brainer, but that doesn't say whether the dwarf is well formed in the first place, and the assert is a different matter. However, briefly looking at the dwarf dump around the offsets where the errors occur doesn't show anything blatantly wrong.

Anyways, I verified I could still reproduce the crash from bug 1380381 comment 6 by running the job again in a one-click loaner, then tried again with a fixed llvm-dsymutil, and the crash was gone there too.

So the bad news is that reducing the scope of debuginfo like we did in bug 1380381 doesn't entirely work around the problem. It may or may not happen again with the current version of rust, but surely will happen with 1.20.
The good news is that the above is great progress towards a solution.
And fwiw, this patch is confirmed to work:
https://hg.mozilla.org/try/rev/7e0dcabfcc99aad408a76251d3e2f8de050dcaf3

(this is on top of all my WIP for bug 1313111, so clang is built with the patch and used by the build job automatically ; no, I haven't hooked up rust yet thus the rustup hack)

The assert doesn't need patching because, being an assert, it's not actually compiled in release mode (and we build clang in release mode). So technically speaking, this patch can't really hurt. I mean, we might as well already be hitting the assert without knowing (although a quick test suggests we're not).
Looks like glandium is on the case!
Flags: needinfo?(nfroyd)
(In reply to Mike Hommey [:glandium] from comment #6)
> Filed https://bugs.llvm.org/show_bug.cgi?id=33873

Thanks for digging into this and filing the upstream bug!
mw: I don't know if you got roped into this, but we were seeing crashes again in Firefox CI running llvm-dsymutil due to DWARF produced by rustc.
Some relevant info came in the llvm bug:

https://bugs.llvm.org/show_bug.cgi?id=33873#c9

The issue is caused by the DIE referencing a NULL DIE, in this case as its abstract origin:

0x00283dc2:               DW_TAG_inlined_subroutine
                            DW_AT_abstract_origin (cu + 0x2b6844)
                            DW_AT_ranges  (0x002d6070
                               [0x0000000000201c59 - 0x0000000000201c8e)
                               [0x0000000000201c98 - 0x0000000000201cc9))
                            DW_AT_call_file ("/home/worker/workspace/build/src/third_party/rust/app_units/src/app_unit.rs")
                            DW_AT_call_line (12)

Those references are just bogus and in some cases they happen to be coincidentally referencing a NULL DIE. But because the offset is valid, resolveDIEReference returns the a valid NULL DIE which causes trouble in keepDIEAndDependencies but also elsewhere (e.g. cloneDieReferenceAttribute).

So rust would be to blame (but llvm shouldn't crash anyways).
Flags: needinfo?(mwoerister)
This looks like a problem with an LLVM optimization pass because we never emit DW_TAG_inlined_subroutine or DW_AT_abstract_origin in rustc. Any such DIE/tag is generated by the LLVM inliner. We might violate some undocumented pre-condition somewhere but I don't think we are doing anything fancy.
Flags: needinfo?(mwoerister)
Duplicate of this bug: 1397382
Assignee: nobody → mh+mozilla
(In reply to Mike Hommey [:glandium] from comment #14)
> Ideally, we'd backport it to the clang 4 toolchain too, but that results
> in silently(!) missing crash symbols for libxul.

Filed bug 1407944 about the fact that we can silently miss crash symbols.
Comment on attachment 8917744 [details]
Bug 1381043 - Backport llvm r313872 to 3.9 to avoid llvm-dsymutil crashing on bad rust DWARF data.

https://reviewboard.mozilla.org/r/188680/#review194084

Maybe we should think about upgrading to clang 4 or even clang 5 at some point...
Attachment #8917744 - Flags: review?(nfroyd) → review+
Comment on attachment 8917745 [details]
Bug 1381043 - Enable fuller debug info for rust code on OSX cross builds.

https://reviewboard.mozilla.org/r/188682/#review194086
Attachment #8917745 - Flags: review?(nfroyd) → review+
(In reply to Nathan Froyd [:froydnj] from comment #17)
> Comment on attachment 8917744 [details]
> Bug 1381043 - Backport llvm r313872 to 3.9 to avoid llvm-dsymutil crashing
> on bad rust DWARF data.
> 
> https://reviewboard.mozilla.org/r/188680/#review194084
> 
> Maybe we should think about upgrading to clang 4 or even clang 5 at some
> point...

Well, as per the commit message, 4 is not possible. Trunk works, but I haven't tested 5.
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/a3225b6f6193
Backport llvm r313872 to 3.9 to avoid llvm-dsymutil crashing on bad rust DWARF data. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/95555b11aaaf
Enable fuller debug info for rust code on OSX cross builds. r=froydnj
https://hg.mozilla.org/mozilla-central/rev/a3225b6f6193
https://hg.mozilla.org/mozilla-central/rev/95555b11aaaf
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla58
(In reply to Michael Woerister from comment #12)
> This looks like a problem with an LLVM optimization pass because we never
> emit DW_TAG_inlined_subroutine or DW_AT_abstract_origin in rustc. Any such
> DIE/tag is generated by the LLVM inliner. We might violate some undocumented
> pre-condition somewhere but I don't think we are doing anything fancy.

Has a bug been filed on rust and/or on LLVM about this?
Flags: needinfo?(mwoerister)
Yes, I think so:
https://bugs.llvm.org/show_bug.cgi?id=35562
Flags: needinfo?(mwoerister)
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.