Trying to use http://hg.mozilla.org/build/tools/raw-file/default/breakpad/linux/minidump_stackwalk on one of the test slaves: /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk) /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk) minidump_stackwalk exited with return code 1
What version of cygwin is used on the test slaves? The one I built was with cygwin 1.7. It should be pretty easy to rebuild with CXXFLAGS="-g -O1" ./configure if the slaves are still using Cygiwn 1.5.
This isn't Cygwin, this is Linux. We've hit this a number of times in the past, see bug 578880, bug 526868, bug 554854.
Ted, is this an issue for running the minidump_stackwalk manually that environment variables need to be set appropriately or built for the environment he is testing in? In otherwords I'm having trouble seeing exactly how RelEng would help solve this?
We had this problem running test binaries (probably one of those bugs I linked in comment 2), and we needed to set LD_LIBRARY_PATH in the build environment for it to work, since we're building with a newer libstdc++ thatn what's installed on the slaves.
Is this on a fedora slave? I see that the libstdc++.so.6 library on a standard slave does have both of those strings. Is there something more than the below command output to verify the status of this library? [cltbld@talos-r3-fed-001 ~]$ strings /usr/lib/libstdc++.so.6 | grep GLIBCXX GLIBCXX_3.4 GLIBCXX_3.4.1 GLIBCXX_3.4.2 GLIBCXX_3.4.3 GLIBCXX_3.4.4 GLIBCXX_3.4.5 GLIBCXX_3.4.6 GLIBCXX_3.4.7 GLIBCXX_3.4.8 GLIBCXX_3.4.9 GLIBCXX_3.4.10 GLIBCXX_3.4.11 GLIBCXX_3.4.12 GLIBCXX_3.4.13 GLIBCXX_FORCE_NEW GLIBCXX_DEBUG_MESSAGE_LENGTH
I'm not sure how to get from a fuzzing log to a slave name. Should I add something that writes the slave name to a file?
The fuzzing jobs run on the CentOS builders, which will have an older libstdc++ on them.
It would be nice to get this fixed so I can find Linux-specific crash bugs through fuzzing :)
Assigning this to Rail. Rail, can you look at the issue here - it seems to me that we can set the LD_LIBRARY_PATH env var to point to the GCC environment this script needs. Can you confirm that? and if so, then make a patch to change this script to have that information? thanks!
IS it possible to use statically linked minidump_stackwalk? In this case we can run it on any slave.
Yeah, there shouldn't be any problem with that, I just didn't build it that way.
(In reply to comment #11) > Yeah, there shouldn't be any problem with that, I just didn't build it that > way. Assigning to you in this case. ;) Feel free to push it back to me when we are ready for deployment of new binaries.
Created attachment 497624 [details] [diff] [review] Statically link Linux minidump_stackwalk Here's a patch to build/tools with a statically-linked Linux minidump_stackwalk.
Created attachment 497626 [details] [diff] [review] Statically link Linux minidump_stackwalk Here's a patch to build/tools with a statically-linked Linux minidump_stackwalk.
Comment on attachment 497626 [details] [diff] [review] Statically link Linux minidump_stackwalk Welcome to stampy town.
Comment on attachment 497626 [details] [diff] [review] Statically link Linux minidump_stackwalk http://hg.mozilla.org/build/tools/rev/776728aff443
Still broken, at least for 64-bit: /builds/slave/fuzzer-linux64/build/minidump_stackwalk: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /builds/slave/fuzzer-linux64/build/minidump_stackwalk)
Does seem to be working on 32-bit :)
Ah, crap, I apparently only fixed the 32-bit mdsw. I thought they were symlinks at one point?
Having this working on 32-bit allowed me to find bug 673390 :)
(In reply to comment #19) > Ah, crap, I apparently only fixed the 32-bit mdsw. I thought they were > symlinks at one point? Ted: does this imply that I can simply copy over the already-built statically linked version into linux64 and it will just work?
That ought to be fine, if it's statically linked then it shouldn't have any library dependencies to worry about, so it should work.
Comment on attachment 497626 [details] [diff] [review] Statically link Linux minidump_stackwalk Checked-in for linux64 also: http://hg.mozilla.org/build/tools/rev/9e0852356108
I don't think this works. If I recall correctly, a linux32 executable is only able to deal with linux32 at runtime. So using the linux32 executable on linux64 is not going to work.
It runs fine on my 64-bit Ubuntu system, FWIW.
(In reply to comment #25) > It runs fine on my 64-bit Ubuntu system, FWIW. I must be mixing with dump_symbols.