706 bytes, patch
|Details | Diff | Splinter Review|
784 bytes, patch
|Details | Diff | Splinter Review|
444 bytes, patch
|Details | Diff | Splinter Review|
I crashed today and was unable to load symbol store. ... thunderbird trunk nightly w32 buildsymbols broken on comm-central broken - missing at http://symbols.mozilla.org/thunderbird per Ted, also seamonkey, possibly only w32 ... "look at the build log, see if anything funny shows up under the "buildsymbols" <ted> if you know the buildid, you can check <ted> there's always a .txt file that lists the symbols <ted> http://symbols.mozilla.org/thunderbird/thunderbird-3.0b1pre-WINNT-20080813031200-symbols.txt <ted> there totally isn't a thunderbird.pdb in there <wsmwk> ted: that's the one <ted> have a look at the build log, see if anything funny shows up under the "buildsymbols" step? <wsmwk> ted: would there be a related reason why breakpad didn't trigger from my crash? crash threw me into MS dialog <ted> not afaik <ted> you know, i think buildsymbols might be kind of broken for you <ted> since the move to comm-central <wsmwk> could be :) <ted> you should file a bug on that <ted> i think it only scans the moz objdir for pdb files <ted> so it won't ever find thunderbird.pdb
(In reply to comment #0) > <wsmwk> ted: would there be a related reason why breakpad didn't trigger from > my crash? crash threw me into MS dialog > <ted> not afaik I don't think breakpad is working at all in current trunk. Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a2pre) Gecko/20080813031200 Shredder/3.0b1pre ID:20080813031200 I have a few reproducable crashes, if I can confirm, I'll open a new bug.
Opened bug 450519 for the lack of breakpad reports
It's busted for at least 2 different reasons. Slightly unrelated, but on Linux, dump_syms crashes on various things, like thunderbird-bin. When this happens, symbols are simply skipped for that particular object. See Bug 445090. On Win32, it's a very simple problem, the symbol dumping code needs to find .pdb files everywhere within the objdir, and unfortunately, it now runs within objdir/mozilla and isn't seeing all .pdb files. The offending line is this one: http://mxr.mozilla.org/comm-central/source/mozilla/Makefile.in#157 And for our usage, replacing '.' with '..' would work, but the real solution is to use $OBJDIR or something similar that will work both with mozilla-central alone, and with the comm-central/mozilla-central nesting setup of Thunderbird. A possible temporary fix would be to run $> make buildsymbols MAKE_SYM_STORE_PATH=.. Until the buildsystem is fixed. I've applied that change temporarly on the nightly buildbot master, lets see what happens to tonight symbols...
(In reply to comment #3) > $> make buildsymbols MAKE_SYM_STORE_PATH=.. The correct solution might be to create a makefile target in comm-central for this and call the mozilla/ makefile with this variable override.
Created attachment 334217 [details] [diff] [review] [checked in] comm-central buildsymbols fixup Something like this would do the trick, like recommended in comment #4. However, doesn't that mean that everything that builds a project like comm-central does, with a mozilla-central checkout one directory down have to replicate this code ? Feels too me like mozilla-central's make file shouldn't use '.' in specifying where to find binaries, but rather use a more precise make variable that we could overried more easily. MAKE_SYM_STORE_PATH feels like internal details that might change in the future.
(In reply to comment #5) > However, doesn't that mean that everything that builds a project like > comm-central does, with a mozilla-central checkout one directory down have to > replicate this code ? True, if you call this "code" at all ;-) I don't expect too many projects building up something like comm-central though. I'd add a comment there pointing to this bug to tell why this was done, but else it looks good to me - if it actually works.
Comment on attachment 334217 [details] [diff] [review] [checked in] comm-central buildsymbols fixup I've tested it on my own build, and it worked as expected, really only making a difference on win32.
Comment on attachment 334217 [details] [diff] [review] [checked in] comm-central buildsymbols fixup Looks good to me. I neither have win32 nor a build with crashreporter enabled around, so I can't actually test, but I trust what you said and it looks good from code inspection.
Comment on attachment 334217 [details] [diff] [review] [checked in] comm-central buildsymbols fixup http://hg.mozilla.org/comm-central/index.cgi/rev/138 changeset: 138:418432362df6 tag: tip user: Philippe M. Chiasson <email@example.com> date: Mon Aug 18 15:27:39 2008 -0400
I don't have a crash to test, but looks good for 20080819 build. http://symbols.mozilla.org/thunderbird/thunderbird-3.0b1pre-WINNT-20080819031251-symbols.txt includes the lines thunderbird.pdb/A405399E4AB4459B8D6992C9DCEAC8061/thunderbird.sym thunderbird.pdb/A405399E4AB4459B8D6992C9DCEAC8061/thunderbird.pdb whether the symbols are in fact available remains to be seen
See comment #10. Somebody with Windows and a reproducible way to crash Thunderbird should be able to verify whether the debugging symbols for thunderbird.exe are pulled in correctly, as it used to.
Sorry, I still can't confirm. With this bug 450519 fixed - No breakpad reports since comm-central move - crash-stats should be getting w32 and linux crashes again. But after 4 hours I still have no response to view a crash I induced at 8am this morning, so I can't confirm. http://crash-stats.mozilla.com/report/pending/512cc60f-74f8-11dd-bb58-001a4bd43ed6
symbols should be there, but whether they are working is another matter. Due to no response from crash-stats we must await fixing of bug 422908 BTW, Ted's crashme extension can be used to force crash in Thunderbird, when breakpad is enabled for the build :) crashme @ https://wiki.mozilla.org/QA/Firefox3/TestPlan/Breakpad#References
symbols found for http://crash-stats.mozilla.com/report/pending/512cc60f-74f8-11dd-bb58-001a4bd43ed6 (some hours later) FIXED
This turns out to be the wrong thing to do for OS X Universal builds. In that case, MAKE_SYM_STORE_PATH should be set to dist/universal, as can be seen here: http://mxr.mozilla.org/mozilla-central/source/Makefile.in#164 The fix to comm-central/Makefile.in should probably look a little like this: +MAKE_SYM_STORE_PATH := .. +ifeq ($(OS_ARCH),Darwin) +ifdef UNIVERSAL_BINARY + MAKE_SYM_STORE_PATH := dist/universal +endif +endif + # http://bugzilla.mozilla.org/show_bug.cgi?id=450485 buildsymbols :: - $(MAKE) -C mozilla $@ MAKE_SYM_STORE_PATH=.. + $(MAKE) -C mozilla $@ MAKE_SYM_STORE_PATH=$(MAKE_SYM_STORE_PATH) I haven't tested this, and I am not sure all these variables are available at that point, OS_ARCH, UNIVERSAL_BINARY, but that's the idea of the fix.
Created attachment 344484 [details] [diff] [review] process universal binaries when generating buildsymbols on OS X [Checkin: Comment 19] Finally gotten around to writing/testing a proper patch. This patch limits symbols extraction to the final universal binaries.
Comment on attachment 344484 [details] [diff] [review] process universal binaries when generating buildsymbols on OS X [Checkin: Comment 19] r=me by code inspection, I take it that you tested this, and it looks reasonable.
(In reply to comment #17) > (From update of attachment 344484 [details] [diff] [review]) > r=me by code inspection, I take it that you tested this, Yes, both on OSX Universal and Linux without breakage > and it looks reasonable. Thanks!
Comment on attachment 344484 [details] [diff] [review] process universal binaries when generating buildsymbols on OS X [Checkin: Comment 19] http://hg.mozilla.org/comm-central/rev/5709136d4b9c
The patch in attachment 344484 [details] [diff] [review] is responsible for bug 464247. The sub-make will not inherit the value of MAKE_SYM_STORE_PATH and will be back to using mozilla's defaults. The proper fix is to explicitely pass this into mozilla's sub-make invocation. The only case where this would have worked is if MAKE_SYM_STORE_PATH was set in the environemnt or passed by the make command-line.
Created attachment 348790 [details] [diff] [review] Pass MAKE_SYM_STORE_PATH down to mozilla's buildsymbols target Explicitely passing down the value for MAKE_SYM_STORE_PATH we've decided on down to mozilla's buildsymbols target is required.
Comment on attachment 348790 [details] [diff] [review] Pass MAKE_SYM_STORE_PATH down to mozilla's buildsymbols target sounds logical to me, I wondered myself if that was the problem when looking at the last patch. r=me by code inspection.
What must have hapenned is that I've either tested this with MAKE_SYM_STORE_PATH in my environment, or invoked make directly with it, possibly. Otherwise, not entirely sure why it would have worked quite right, even on linux.
http://hg.mozilla.org/comm-central/rev/be77b2aa036b changeset: 1143:be77b2aa036b tag: tip user: Philippe M. Chiasson <firstname.lastname@example.org> date: Tue Nov 18 14:01:02 2008 -0500 summary: Bug 450485. Make sure the value of MAKE_SYM_STORE_PATH is passed to mozilla's buildsymbols target. r=KaiRo
yay! we have w32 symbols. I haven't checked Mac. thanks gozer crashme extension results bp-13f2c778-b6cf-475b-bada-ef2620081119 build 20081118031447 crashme.dll nsCrasher::Crash nsCrasher.cpp:21 xpcom_core.dll NS_InvokeByIndex_P xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:101 thunderbird.exe thunderbird.exe@0x9aab0 bp-2cd19882-840c-4fba-a8ac-739720081119 buildid 20081119032441 crashme.dll nsCrasher::Crash nsCrasher.cpp:21 xpcom_core.dll NS_InvokeByIndex_P xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:101 thunderbird.exe XPCWrappedNative::CallMethod js/src/xpconnect/src/xpcwrappednative.cpp:2422 thunderbird.exe XPC_WN_CallMethod js/src/xpconnect/src/xpcwrappednativejsops.cpp:1477