Create symbols with "-gdwarf-2" instead of just "-g"

RESOLVED FIXED in Firefox 55

Status

Firefox Build System
General
RESOLVED FIXED
a year ago
5 months ago

People

(Reporter: gkw, Assigned: sfink)

Tracking

({regression})

Trunk
mozilla55
regression

Firefox Tracking Flags

(firefox-esr52 unaffected, firefox53 unaffected, firefox54 unaffected, firefox55 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

a year ago
+++ This bug was initially created as a clone of Bug #1346381 +++

I had made fuzzing on releng machines work again (bug 1272137 comment 10) with js shells that even had symbols because they were inadvertently unstriped.

Now that this change in bug 1346381 has occurred, we no longer have symbols, so deduplication of fuzzing testcases is impossible.


Unstriped:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000100
0x000000010032e255 in js::jit::SnapshotIterator::allocationValue ()
(gdb) bt
#0  0x000000010032e255 in js::jit::SnapshotIterator::allocationValue ()
warning: .o file "/builds/slave/m-cen-m64-d-000000000000000000/build/src/obj-firefox/js/src/shell/../libjs_static.a(Unified_cpp_js_src9.o)" more recent than executable timestamp in "/Users/cltbld/aa20170310/js"
warning: Couldn't open object file '/builds/slave/m-cen-m64-d-000000000000000000/build/src/obj-firefox/js/src/shell/../libjs_static.a(Unified_cpp_js_src9.o)'
#1  0x0000000100adc20b in InitFromBailout ()
#2  0x0000000100ada84f in js::jit::BailoutIonToBaseline ()
#3  0x0000000100adb8bd in js::jit::ExceptionHandlerBailout ()
#4  0x000000010032a47a in js::jit::HandleException ()
(gdb)


Striped:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000100
0x000000010032faf5 in start ()
(gdb) bt
#0  0x000000010032faf5 in start ()
(gdb)


Not sure what the best way forward is. Perhaps have 2 builds?
Flags: needinfo?(mh+mozilla)
(Assignee)

Updated

a year ago
Summary: Having a striped js shell breaks fuzzing on releng js shells that we ship → Having a stripped js shell breaks fuzzing on releng js shells that we ship
(Assignee)

Comment 1

a year ago
One option would be to put the debug info into a separate file. Then fuzzers could use that, but AMO or whoever could have their smaller download.
(Reporter)

Comment 2

a year ago
(In reply to Steve Fink [:sfink] [:s:] from comment #1)
> One option would be to put the debug info into a separate file. Then fuzzers
> could use that, but AMO or whoever could have their smaller download.

This compromise would work for me.
The crashreporter-symbols-full.zip files have the dwarf debug info for the js shell. The smaller crashreporter-symbols.zip files have the breakpad symbol files for the js shell (note they are also in the -full.zip files).

Just use that?
Flags: needinfo?(mh+mozilla)
Blocks: 1346381
No longer depends on: 1346381
(Reporter)

Comment 4

a year ago
(In reply to Mike Hommey [:glandium] from comment #3)
> The crashreporter-symbols-full.zip files have the dwarf debug info for the
> js shell. The smaller crashreporter-symbols.zip files have the breakpad
> symbol files for the js shell (note they are also in the -full.zip files).

:glandium and I spoke on IRC over the past days and he made the following suggestions for the loaner 10.7 Mac machine I had:

* We should use a newer version of gdb *or*
* We should use lldb *or*
* We should try compiling with "-gdwarf-2" instead of "-g"

I first installed Homebrew, which seemed to work even though 10.7 is unsupported. I then installed gdb 7.12 (latest possible version) but lldb / llvm didn't work, not even the oldest version available, LLVM 3.7.

Homebrew gdb needed to be codesigned, so I manually VPN'ed (screen sharing) in to create a self-signed cert, after which the loaner needed to be rebooted. After that, homebrew gdb then could only be codesigned by VPN'ing in and doing it via Terminal.app (not via ssh).[1]

After all jumping through all these hoops, the default symbols with "-g" still didn't work, it only worked with symbols from a try build with "-gdwarf-2".

fwiw, I tried on another macOS 10.12 Sierra machine which could access the "-gdwarf-2" symbols, only that one could not set the build_src directory to be the current working directory or lldb (version 360.1.70) will assert at:

Assertion failed: (m_die_array.empty() && "Compile unit DIE already added"), function AddCompileUnitDIE, file /Library/Caches/com.apple.xbs/Sources/lldb/lldb-360.1.70/source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp, line 312.

In conclusion, the symbols only work with a stripped js binary exec if:

* We have a newer version of gdb or lldb
* The symbols are produced with "-gdwarf-2" and not "-g"

Need to test next with cdb/windbg.

[1] https://gcc.gnu.org/onlinedocs/gcc-4.8.1/gnat_ugn_unw/Codesigning-the-Debugger.html
Summary: Having a stripped js shell breaks fuzzing on releng js shells that we ship → Create symbols with "-gdwarf-2" instead of just "-g"
Comment on attachment 8848684 [details] [diff] [review]
Patch v1

Review of attachment 8848684 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/moz.configure/toolchain.configure
@@ +849,5 @@
>  def default_debug_flags(compiler_info):
>      # Debug info is ON by default.
>      if compiler_info.type in ('msvc', 'clang-cl'):
>          return '-Zi'
> +    return '-gdwarf-2'

It would be better to set this for the specific build you need it for rather than everything else. There are actually benefits from using -g instead by default (like, better, and usually smaller debug info)
Attachment #8848684 - Flags: review?(mh+mozilla)
status-firefox55: affected → fix-optional
Yeah, I'd rather not have our release builds use -gdwarf-2.
(Assignee)

Comment 8

a year ago
Gary, I don't know if fuzzing is using the SM(f) builds yet. But if so, I could switch (just) those to -gdwarf-2.
Flags: needinfo?(gary)
(Reporter)

Comment 9

a year ago
Works for me for now. I'd like to punt this to you, Steve.
Flags: needinfo?(gary) → needinfo?(sphink)
(Assignee)

Comment 10

a year ago
Created attachment 8866579 [details] [diff] [review]
Generate old-style DWARF info for fuzzing builds
Attachment #8866579 - Flags: review?(mh+mozilla)
(Assignee)

Updated

a year ago
Assignee: nobody → sphink
Status: NEW → ASSIGNED
(Reporter)

Updated

a year ago
Attachment #8848684 - Attachment is obsolete: true
Attachment #8866579 - Flags: review?(mh+mozilla) → review+

Comment 11

a year ago
Pushed by sfink@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2aa6d03565fa
Generate old-style DWARF info for fuzzing builds, r=glandium

Comment 12

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/2aa6d03565fa
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
status-firefox55: fix-optional → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
status-firefox53: --- → unaffected
status-firefox54: --- → affected
status-firefox-esr52: --- → unaffected
Whiteboard: [checkin-needed-beta]
The fuzzing builds only exist on 55, so I think there's nothing to do for 54 here. But feel free to correct me if that's wrong.
status-firefox54: affected → unaffected
Flags: needinfo?(sphink)
Whiteboard: [checkin-needed-beta]
(Assignee)

Comment 14

a year ago
That's correct. Bug 1346016 landed in 55.

Updated

5 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.