1003085 - Update in-tree dump_syms binaries

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[(not currently active) Ted Mielczarek]

Upstream Breakpad made some changes that can make dump_syms much faster:
https://code.google.com/p/google-breakpad/source/detail?r=1316

dmajor tested in bug 669384 and it looks like this makes buildsymbols actually usable for our Win64 builds, so we should update our in-tree dump_syms binaries:
http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/tools/win32/

We've got binaries built with VC2010, 2012 and 2013 in there, we should really update all three.

Comment 1

[Makoto Kato [:m_kato]]

Attachment: Part 1. Import r1316 into tree

Comment 2

[Makoto Kato [:m_kato]]

Attachment: Part 2. Update dump_syms.exe

Comment 3

[Makoto Kato [:m_kato]]

Comment on attachment 8416327 [details] [diff] [review]
Part 1. Import r1316 into tree

Just importing r1316 into tree.

result of win64 debug...

  adding: xul.pdb/122499929D5C490CBFF757B36E309C581/xul.sym (140 bytes security) (deflated 74%)
program finished with exit code 0
elapsedTime=340.011000
========= Finished 'python c:/builds/moz2_slave/try-w64-d-00000000000000000000/build/build/pymake/make.py ...' (results: 0, elapsed: 5 mins, 40 secs) (at 2014-05-01 21:42:31.392773) =========

Comment 4

[Makoto Kato [:m_kato]]

Comment on attachment 8416328 [details] [diff] [review]
Part 2. Update dump_syms.exe

build by 2010 sp1, 2012 update 4 and 2013 update 1.

Comment 5

[(not currently active) Ted Mielczarek]

Comment on attachment 8416327 [details] [diff] [review]
Part 1. Import r1316 into tree

Review of attachment 8416327 [details] [diff] [review]:
-----------------------------------------------------------------

Since we're not building from source I don't think this is strictly necessary. (Also since this patch is already upstream it's not that big of a deal, we'll pick it up next time we sync anyway.)

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 8416328 [details] [diff] [review]
Part 2. Update dump_syms.exe

Review of attachment 8416328 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks! We should probably let the CrashKill team know to keep an eye on nightly crash reports for any weirdness that's fallout from this.

Comment 7

[Makoto Kato [:m_kato]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> Comment on attachment 8416327 [details] [diff] [review]
> Part 1. Import r1316 into tree
> 
> Review of attachment 8416327 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Since we're not building from source I don't think this is strictly
> necessary. (Also since this patch is already upstream it's not that big of a
> deal, we'll pick it up next time we sync anyway.)

OK.

FYI, upstream cannot build on 2010 or 2012 due to https://code.google.com/p/google-breakpad/issues/detail?id=584.  2010 or 2012 needs #undef max after #include <set>.

Comment 8

[Makoto Kato [:m_kato]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/46b2a27092f5

I also watch crash reporter data.

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/46b2a27092f5

Comment 10

[(not currently active) Ted Mielczarek]

We need to back this out. RyanVM pointed me at a crash on TBPL that displayed no symbols:
https://tbpl.mozilla.org/php/getParsedLog.php?id=39285462&tree=Mozilla-Inbound

It turns out that the one change the upstream patch did make (outputting PUBLIC lines along with FUNC lines) is harmful because it outputs numerous PUBLIC lines at the same address, which BasicSourceLineResolver records as errors. When it hits 100 errors it fails parsing and gives up loading the symbol file.

Comment 11

[(not currently active) Ted Mielczarek]

It actually looks like this was fixed upstream in rev 1319:
https://code.google.com/p/google-breakpad/source/detail?r=1319

This ought to work okay if we rebuild the binaries at that rev (or newer).

Comment 12

[Makoto Kato [:m_kato]]

Attachment: dump_syms r1319 builds

built by r1319 with #undef max

Comment 13

[Makoto Kato [:m_kato]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d6a0c7c471e2

Comment 14

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d6a0c7c471e2

Comment 15

[Bob Clary [:bc] (inactive)]

I have vs 2010 and 2012 installed. I'm getting oodles of CoCreateInstance CLSID_DiaSource {3BFCEA48-620F-4B6B-81F7-B9AF75454C7D} failed (msdia*.dll unregistered?) errors on win7 64bit vs 2012 update 4 on building nightly but not beta or aurora.  What compiler was used to build these?

PS. Can we set the executable permission when checking executables like these into the tree?

Comment 16

[Bob Clary [:bc] (inactive)]

Looks like 3BFCEA48-620F-4B6B-81F7-B9AF75454C7D is msdia120.dll distributed with vs 2013. Shouldn't dump_syms_vc1700.exe be built with vs 2012?

Comment 17

[Makoto Kato [:m_kato]]

(In reply to Bob Clary [:bc:] from comment #16)
> Looks like 3BFCEA48-620F-4B6B-81F7-B9AF75454C7D is msdia120.dll distributed
> with vs 2013. Shouldn't dump_syms_vc1700.exe be built with vs 2012?

I will rebuild this.  VS2013 installation seems to overwrites 2012's SDK.

Comment 18

[Makoto Kato [:m_kato]]

Also, I check 20140514 buiild, crash reporter appears file path of source code.

Comment 19

[Bob Clary [:bc] (inactive)]

m_kato: do you have an estimated time for the new dump_syms? I am unable to test windows 64bit nightly builds until this is fixed.

Comment 20

[Makoto Kato [:m_kato]]

(In reply to Bob Clary [:bc:] from comment #19)
> m_kato: do you have an estimated time for the new dump_syms? I am unable to
> test windows 64bit nightly builds until this is fixed.

I will bulild this today, then send a review.

Comment 21

[Makoto Kato [:m_kato]]

Attachment: Rebuild dump_syms.exe for VS11

Comment 22

[Makoto Kato [:m_kato]]

Comment on attachment 8424646 [details] [diff] [review]
Rebuild dump_syms.exe for VS11

I setup VS11 only desktop, then it work fine.

Comment 23

[Bob Clary [:bc] (inactive)]

I applied the patch and dump_syms-vc1700.exe works for me.

Comment 24

[Makoto Kato [:m_kato]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/4ce72b03591e

Comment 25

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/4ce72b03591e