Closed Bug 873667 Opened 12 years ago Closed 10 years ago

Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp, resampler_private_AR2.c, etc.

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: RyanVM, Unassigned)

References

Details

(Keywords: intermittent-failure)

+++ This bug was initially created as a clone of Bug #820796 +++ Cloning from bug 820796 so we can track this as a separate issue from the gcc 4.5 issue. https://tbpl.mozilla.org/php/getParsedLog.php?id=23092428&tree=Mozilla-Inbound Linux x86-64 mozilla-inbound pgo-build on 2013-05-17 12:47:09 PDT for push e5e7221a2c82 slave: bld-linux64-ec2-602 nsTIS620ToUnicode.cpp /builds/slave/m-in-l64-pgo-00000000000000000/build/intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp:22:1: fatal error: error writing to -: Broken pipe compilation terminated. g++: internal compiler error: Segmentation fault (program as) Please submit a full bug report, with preprocessed source if appropriate. See <http://gcc.gnu.org/bugs.html> for instructions. make[8]: *** [nsCP1253ToUnicode.o] Error 4 make[8]: *** Deleting file `nsCP1253ToUnicode.o' make[8]: *** Waiting for unfinished jobs....
Summary: Intermittent Linux gcc 4.7.2 Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp → Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp
See Also: → 820796
The log in comment 1 seems to be a different issue from the description (as crashes in the first gcc in the second). in any case I'm not sure there's much we can do unless we can get access to the objdir when one of these fails and the failure is reproducable given the same profile data.
https://tbpl.mozilla.org/php/getParsedLog.php?id=23137546&tree=Mozilla-Inbound /usr/bin/ccache /tools/gcc-4.7.2-0moz1/bin/gcc -m32 -o resampler_down2.o -c -I../../dist/system_wrappers -include /builds/slave/m-in-lx-pgo-000000000000000000/build/config/gcc_hidden.h -DOPUS_BUILD -DOPUS_VERSION='"draft-11-mozilla"' -DUSE_ALLOCA -Drestrict= -DHAVE_LRINTF -DMOZ_GLUE_IN_PROGRAM -DNO_NSPR_10_SUPPORT -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/include -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/celt -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/silk -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/src -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/silk/float -I/builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus -I. -I../../dist/include -I/builds/slave/m-in-lx-pgo-000000000000000000/build/obj-firefox/dist/include/nspr -I/builds/slave/m-in-lx-pgo-000000000000000000/build/obj-firefox/dist/include/nss -fPIC -Wall -Wpointer-arith -Wdeclaration-after-statement -Werror=return-type -Wtype-limits -Wempty-body -Wsign-compare -Wno-unused -Wcast-align -std=gnu99 -fgnu89-inline -fno-strict-aliasing -ffunction-sections -fdata-sections -pthread -pipe -DNDEBUG -DTRIMMED -g -fprofile-use -fprofile-correction -Wcoverage-mismatch -O3 -fno-omit-frame-pointer -include ../../mozilla-config.h -DMOZILLA_CLIENT -MD -MP -MF .deps/resampler_down2.o.pp /builds/slave/m-in-lx-pgo-000000000000000000/build/media/libopus/silk/resampler_down2.c resampler_private_AR2.c <built-in>:0:0: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See <http://gcc.gnu.org/bugs.html> for instructions. make[6]: *** [resampler.o] Error 1
Summary: Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp → Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp, resampler_private_AR2.c
Depends on: 874528
Is it a drop-all-ICEs bug?
(In reply to Mike Hommey [:glandium] from comment #5) > Is it a drop-all-ICEs bug? seems that atleast, and I can't think of a particularly good argument for having multiple bugs where the only difference is slightly different ICE text.
We should probably try gcc 4.7.3. Maybe a good occasion to try tooltool for gcc as well?
Depends on: 875465
So, I tried doing a normal opt build but with CFLAGS / CXXFLAGS containing those for pgo use in https://tbpl.mozilla.org/?tree=Try&rev=191e3ccc2c91 and was able to reproduce the ICE on one linux build, which I find interesting since one would hope adding those flags when there's no pgo data would have no effect. So I wonder if its worth trying stripping the pgo use flags from the command line when there's no data for the file being compiled as a bit of a work around.
(In reply to Mike Hommey [:glandium] from comment #8) > We should probably try gcc 4.7.3. Maybe a good occasion to try tooltool for > gcc as well? The list of fixed bugs in 4.7.3 doesn't seem to contain anything related to this bug: http://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED&resolution=FIXED&target_milestone=4.7.3 (In reply to Trevor Saunders (:tbsaunde) from comment #9) > So, I tried doing a normal opt build but with CFLAGS / CXXFLAGS containing > those for pgo use in https://tbpl.mozilla.org/?tree=Try&rev=191e3ccc2c91 and > was able to reproduce the ICE on one linux build, which I find interesting > since one would hope adding those flags when there's no pgo data would have > no effect. So I wonder if its worth trying stripping the pgo use flags from > the command line when there's no data for the file being compiled as a bit > of a work around. Did it reproduce consistently? Maybe we should just try to get this fixed in GCC itself.
(In reply to Nathan Froyd (:froydnj) from comment #10) > (In reply to Mike Hommey [:glandium] from comment #8) > > We should probably try gcc 4.7.3. Maybe a good occasion to try tooltool for > > gcc as well? > > The list of fixed bugs in 4.7.3 doesn't seem to contain anything related to > this bug: > > http://gcc.gnu.org/bugzilla/buglist. > cgi?bug_status=RESOLVED&resolution=FIXED&target_milestone=4.7.3 ok :/ I think its probably still worth getting bug 875465 done though I'd probably only use it on try because I hacked dumping a stack into the code to ICE and that seems a lot more useful than "I segfaulted somewhere" messages :) > (In reply to Trevor Saunders (:tbsaunde) from comment #9) > > So, I tried doing a normal opt build but with CFLAGS / CXXFLAGS containing > > those for pgo use in https://tbpl.mozilla.org/?tree=Try&rev=191e3ccc2c91 and > > was able to reproduce the ICE on one linux build, which I find interesting > > since one would hope adding those flags when there's no pgo data would have > > no effect. So I wonder if its worth trying stripping the pgo use flags from > > the command line when there's no data for the file being compiled as a bit > > of a work around. > > Did it reproduce consistently? Maybe we should just try to get this fixed > in GCC itself. I haven't found anything reproduceble :/ I have a tarball of the objdir and srcdir for the build for the buildwhere I tried borrowing the slave, but couldn't get the gcc to ICE when I ran make again by hand. I asked in #gcc about this a while ago which resulted in http://gcc.gnu.org/ml/gcc-patches/2013-05/msg01281.html However I suspect there has been profiling data for the file being compiled in some of the cases we have here. When I look at my local pgo objdir which is fairly stale I see gcda files for some of the files we ICE in here.
https://tbpl.mozilla.org/php/getParsedLog.php?id=23849119&tree=Fx-Team /builds/slave/fx-team-l64-pgo-00000000000000/build/intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp:1:0: internal compiler error: in default_addr_space_address_mode, at targhooks.c:1051
(In reply to Ryan VanderMeulen [:RyanVM] from comment #13) > /builds/slave/fx-team-l64-pgo-00000000000000/build/intl/uconv/ucvlatin/ > nsCP1253ToUnicode.cpp:1:0: internal compiler error: in > default_addr_space_address_mode, at targhooks.c:1051 That is a bizarre error; multiple address spaces aren't supported on x86. We should never be hitting that assert. I smell memory corruption.
(In reply to Nathan Froyd (:froydnj) from comment #14) > (In reply to Ryan VanderMeulen [:RyanVM] from comment #13) > > /builds/slave/fx-team-l64-pgo-00000000000000/build/intl/uconv/ucvlatin/ > > nsCP1253ToUnicode.cpp:1:0: internal compiler error: in > > default_addr_space_address_mode, at targhooks.c:1051 > > That is a bizarre error; multiple address spaces aren't supported on x86. > We should never be hitting that assert. I smell memory corruption. yeah, I wonder if bad hardware being taken out of the pool and new bad hardware showing up can explain the gap of a week or two with no problems.
Summary: Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp, resampler_private_AR2.c → Intermittent Linux gcc 4.7.2 PGO Segmentation fault in intl/uconv/ucvlatin/nsCP1253ToUnicode.cpp, resampler_private_AR2.c, etc.
https://tbpl.mozilla.org/php/getParsedLog.php?id=25171472&tree=Mozilla-Inbound I do find it interesting that many of these cases occur in the intl code.
We switched to GCC 4.7.3 in bug 902104. Will be interesting to see if this goes away.
(In reply to Gregory Szorc [:gps] from comment #50) > We switched to GCC 4.7.3 in bug 902104. Will be interesting to see if this > goes away. https://tbpl.mozilla.org/php/getParsedLog.php?id=26533433&tree=B2g-Inbound Nope.
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #53) > (In reply to Gregory Szorc [:gps] from comment #50) > > We switched to GCC 4.7.3 in bug 902104. Will be interesting to see if this > > goes away. > > https://tbpl.mozilla.org/php/getParsedLog.php?id=26533433&tree=B2g-Inbound > > Nope. Note this one, like comment 0 is an assembler crash (binutils, not gcc) ; other were gcc ICEs, and those might be gone. We may want to investigate upgrading binutils.
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #55) > How about this one? > > https://tbpl.mozilla.org/php/getParsedLog.php?id=26589060&tree=Mozilla- > Inbound That would be an assembler crash: g++: internal compiler error: Segmentation fault (program as) Versus a crash in GCC itself, from comment 52: /builds/slave/m-beta-lx-00000000000000000000/build/dom/workers/WorkerPrivate.cpp: In destructor 'nsTArray_base<Alloc, Copy>::~nsTArray_base() [with Alloc = nsTArrayInfallibleAllocator; Copy = nsTArray_CopyElements<mozilla::dom::workers::WorkerRunnable*>]': /builds/slave/m-beta-lx-00000000000000000000/build/dom/workers/WorkerPrivate.cpp:4462:1: note: correcting inconsistent profile data /builds/slave/m-beta-lx-00000000000000000000/build/dom/workers/WorkerPrivate.cpp:4462:1: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See <http://gcc.gnu.org/bugs.html> for instructions.
So if all the 4.7.3 crashes are in binutils, can we consider switching aurora/beta over to it?
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #63) > So if all the 4.7.3 crashes are in binutils, can we consider switching > aurora/beta over to it? "it" being binutils or 4.7.3?
Sorry, that was incredibly vague on my part. Upgrading to 4.7.3. Seems like a minor release.
https://tbpl.mozilla.org/php/getParsedLog.php?id=27101116&tree=Mozilla-Inbound /builds/slave/m-in-l64-pgo-00000000000000000/build/js/src/vm/Shape.cpp:1503:1: note: correcting inconsistent profile data /builds/slave/m-in-l64-pgo-00000000000000000/build/js/src/vm/Shape.cpp: In static member function 'static bool JSObject::setParent(JSContext*, JS::HandleObject, JS::HandleObject)': /builds/slave/m-in-l64-pgo-00000000000000000/build/js/src/vm/Shape.cpp:1503:1: internal compiler error: Segmentation fault Please submit a full bug report,
Blocks: 910105
Inactive; closing (see bug 1180138).
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.