Closed Bug 610311 Opened 10 years ago Closed 9 years ago

In-tree minidump_stackwalk requires libraries not present on slaves

Categories

(Release Engineering :: General, defect)

x86
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jruderman, Assigned: ted)

Details

(Whiteboard: [automation][sg:want P4])

Attachments

(1 file, 1 obsolete file)

Trying to use http://hg.mozilla.org/build/tools/raw-file/default/breakpad/linux/minidump_stackwalk on one of the test slaves:

/builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk)

/builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /builds/moz2_slave/fuzzer-linux/build/build/minidump_stackwalk)

minidump_stackwalk exited with return code 1
What version of cygwin is used on the test slaves? The one I built was with cygwin 1.7. It should be pretty easy to rebuild with CXXFLAGS="-g -O1" ./configure if the slaves are still using Cygiwn 1.5.
This isn't Cygwin, this is Linux. We've hit this a number of times in the past, see bug 578880, bug 526868, bug 554854.
Ted, is this an issue for running the minidump_stackwalk manually that environment variables need to be set appropriately or built for the environment he is testing in?

In otherwords I'm having trouble seeing exactly how RelEng would help solve this?
Whiteboard: [triagefollowup]
We had this problem running test binaries (probably one of those bugs I linked in comment 2), and we needed to set LD_LIBRARY_PATH in the build environment for it to work, since we're building with a newer libstdc++ thatn what's installed on the slaves.
Is this on a fedora slave?  I see that the libstdc++.so.6 library on a standard slave does have both of those strings.  Is there something more than the below command output to verify the status of this library?

[cltbld@talos-r3-fed-001 ~]$ strings /usr/lib/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_FORCE_NEW
GLIBCXX_DEBUG_MESSAGE_LENGTH
I'm not sure how to get from a fuzzing log to a slave name.  Should I add something that writes the slave name to a file?
The fuzzing jobs run on the CentOS builders, which will have an older libstdc++ on them.
It would be nice to get this fixed so I can find Linux-specific crash bugs through fuzzing :)
Assigning this to Rail.

Rail, can you look at the issue here - it seems to me that we can set the LD_LIBRARY_PATH env var to point to the GCC environment this script needs.

Can you confirm that? and if so, then make a patch to change this script to have that information?

thanks!
Assignee: nobody → rail
IS it possible to use statically linked minidump_stackwalk? In this case we can run it on any slave.
Yeah, there shouldn't be any problem with that, I just didn't build it that way.
(In reply to comment #11)
> Yeah, there shouldn't be any problem with that, I just didn't build it that
> way.

Assigning to you in this case. ;)

Feel free to push it back to me when we are ready for deployment of new binaries.
Assignee: rail → ted.mielczarek
Whiteboard: [triagefollowup] → [automation]
Here's a patch to build/tools with a statically-linked Linux minidump_stackwalk.
Attachment #497624 - Attachment is obsolete: true
Here's a patch to build/tools with a statically-linked Linux minidump_stackwalk.
Attachment #497626 - Flags: review?(bhearsum)
Comment on attachment 497626 [details] [diff] [review]
Statically link Linux minidump_stackwalk

Welcome to stampy town.
Attachment #497626 - Flags: review?(bhearsum) → review+
Whiteboard: [automation] → [automation][sg:want P4]
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Still broken, at least for 64-bit:

/builds/slave/fuzzer-linux64/build/minidump_stackwalk: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /builds/slave/fuzzer-linux64/build/minidump_stackwalk)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Does seem to be working on 32-bit :)
Ah, crap, I apparently only fixed the 32-bit mdsw. I thought they were symlinks at one point?
Having this working on 32-bit allowed me to find bug 673390 :)
(In reply to comment #19)
> Ah, crap, I apparently only fixed the 32-bit mdsw. I thought they were
> symlinks at one point?

Ted: does this imply that I can simply copy over the already-built statically linked version into linux64 and it will just work?
That ought to be fine, if it's statically linked then it shouldn't have any library dependencies to worry about, so it should work.
Comment on attachment 497626 [details] [diff] [review]
Statically link Linux minidump_stackwalk

Checked-in for linux64 also:

http://hg.mozilla.org/build/tools/rev/9e0852356108
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
I don't think this works. If I recall correctly, a linux32 executable is only able to deal with linux32 at runtime. So using the linux32 executable on linux64 is not going to work.
It runs fine on my 64-bit Ubuntu system, FWIW.
(In reply to comment #25)
> It runs fine on my 64-bit Ubuntu system, FWIW.

I must be mixing with dump_symbols.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.