321244 - Lightning build for Linux crashes official Thunderbird build (trunk only) [@ nsCOMPtr_base::~nsCOMPtr_base()]

Status: RESOLVED FIXED

(Calendar :: Lightning Only, defect)

Description

[Thorsten 'nightrat' Fritz]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051220 Firefox/1.6a1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051220 Firefox/1.6a1

Lightning crashes Thunderbird trunk 1.6a1.
Downloading Lightning from ftp://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-trunk/linux-xpi
Downloading Thunderbird from ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk

Install Lightning in Thunderbird. Restart Thunderbird and it crashes.

I compared the compilers for both builds. Lightning is build with gcc 3.2.3. Thunderbird trunk is compiled with gcc 3.3.3.

I tried a build of Lightning, compiled with gcc-3.3.6, and it worked as it should. No crash.

Lightning should be compiled with the same compiler as Thunderbird trunk is compiled with.

Reproducible: Always

Steps to Reproduce:

Comment 1

[Adam Guthrie]

Do you think you could install Thunderbird with talkback and get a talkback ID? http://kb.mozillazine.org/Talkback

Comment 2

[Thorsten 'nightrat' Fritz]

Talkback ID for that crash tb13167007x

Comment 3

[Adam Guthrie]

I don't know what's going on here, but that ID doesn't seem to have any symbols.

Thorsten, did you completely uninstall and reinstall Thunderbird?

Comment 4

[Thorsten 'nightrat' Fritz]

For that ID i didn't uninstall and reinstall Thunderbird, i created only a new profile.

Now a downloaded new packages of Thunderbird and Lightning. Then i got the following ID tb13168683m

Comment 5

[Stefan Sitter [:ssitter]]

Does it also happens with the thunderbird trunk build found at
http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-trunk/?

Comment 6

[timeless]

still no symbols, but the crash is in storage...

Comment 7

[Thorsten 'nightrat' Fritz]

(In reply to comment #5)
> Does it also happens with the thunderbird trunk build found at
> http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-trunk/?

No. There's no crash installing Lightning in that build. That's the build Lightning is compiled with, so the same compiler.

Comment 8

[Stefan Sitter [:ssitter]]

Confirmed using Thunderbird 1.6a1 (20060109) and Lightning from the links above.
Talkback: TB13741482X

Comment 9

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Someone needs to look at it in the debugger and figure out where it's crashing, since talkback doesn't have symbols for lightning (and hence, not for storage).

Comment 10

[Stefan Sitter [:ssitter]]

Crash also happens with Thunderbird 1.5 and branch build Lightning. Talkback ID is TB14028687K.

I don't know how to debug on Linux but maybe the following about:buildconfig information can help:

================================================================

Thunderbird 1.5 (Lightning Build) from
http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/2006-01-15-07-mozilla1.8/

Build platform
target
i686-pc-linux-gnu

Build tools
Compiler Version  Compiler flags
gcc gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-52) -Wall -W -Wno-unused -Wpointer-arith -Wcast-align -Wno-long-long -pedantic -pthread -pipe
c++ gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-52) -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic -fshort-wchar -pthread -pipe -I/usr/X11R6/include

Configure arguments
--enable-application=mail --enable-extensions=default,lightning --disable-static --enable-shared --disable-tests --disable-debug '--enable-optimize=-O2 -gstabs+' 

================================================================

Thunderbird 1.5 (Thunderbird Build) from
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2006-01-15-05-mozilla1.8/

Build platform
target
i686-pc-linux-gnu

Build tools
Compiler Version       Compiler flags
gcc gcc version 3.3.3  -Wall -W -Wno-unused -Wpointer-arith -Wcast-align -Wno-long-long -pedantic -pthread -pipe
c++ gcc version 3.3.3  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic -fshort-wchar -pthread -pipe -I/usr/X11R6/include

Configure arguments
--enable-application=mail --enable-update-channel=release --disable-debug --enable-update-packaging '--enable-optimize=-O2 -gstabs+' --enable-official-branding --disable-tests --disable-shared --enable-static --enable-xft --disable-freetype2 --enable-default-toolkit=gtk2 
================================================================

It seems that the lightning xpi is build with --enable-shared while the official thunderbird is build with --disable-shared. Could this be the reason for the crash?

Comment 11

[Dan Mosedale (:dmosedale, :dmose)]

This appears to not be gcc ABI lossage, however, as the gcc man page seems to say that g++ 3.2* and 3.3* should both be using ABI version 1.

Comment 12

[Dan Mosedale (:dmosedale, :dmose)]

> It seems that the lightning xpi is build with --enable-shared while the
> official thunderbird is build with --disable-shared. Could this be the reason
> for the crash?

I don't think this should be a problem, no.  The builds have always (necessarily, at least with our current build system) been done this way.

Comment 13

[timeless]

Incident ID: 13741482 
Stack Signature nsCOMPtr_base::~nsCOMPtr_base() 5146b4b6 
Product ID ThunderbirdTrunk 
Build ID 2006010904 
Trigger Time 2006-01-09 06:40:06.0 
Platform LinuxIntel 
Operating System Linux 2.6.12-9-686 
Module libxpcom_core.so + (0003d7f6) 
URL visited  
User Comments  
Since Last Crash 0 sec 
Total Uptime 0 sec 
Trigger Reason SIGSEGV: Segmentation Fault: (signal 11) 
Source File, Line No. /builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/build/nsCOMPtr.cpp, line 80 
Stack Trace  

nsCOMPtr_base::~nsCOMPtr_base()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/build/nsCOMPtr.cpp, line 80]
libstoragecomps.so + 0x3612 (0xb702a612)
XPTC_InvokeByIndex()
XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode)()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappednative.cpp, line 3147]
XPC_WN_CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappednativejsops.cpp, line 1444]
js_Invoke()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 1230]
js_Interpret()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 3779]
js_Invoke()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 1254]
js_Interpret()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 3292]
js_Invoke()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 1254]
nsXPCWrappedJSClass::CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappedjsclass.cpp, line 1380]
nsXPCWrappedJS::CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappedjs.cpp, line 466]
PrepareAndDispatch()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/reflect/xptcall/src/md/unix/xptcstubs_gcc_x86_unix.cpp, line 100]
nsComponentManagerImpl::CreateInstance()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/components/nsComponentManager.cpp, line 1725]
nsComponentManagerImpl::GetService()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/components/nsComponentManager.cpp, line 1931]
nsJSCID::GetService()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcjsid.cpp, line 899]
XPTC_InvokeByIndex()
XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode)()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappednative.cpp, line 3147]
XPC_WN_CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappednativejsops.cpp, line 1444]
js_Invoke()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 1230]
js_Interpret()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 3779]
js_Invoke()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/jsinterp.c, line 1254]
nsXPCWrappedJSClass::CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappedjsclass.cpp, line 1380]
nsXPCWrappedJS::CallMethod()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/js/src/xpconnect/src/xpcwrappedjs.cpp, line 466]
PrepareAndDispatch()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/reflect/xptcall/src/md/unix/xptcstubs_gcc_x86_unix.cpp, line 100]
nsObserverService::NotifyObservers()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/xpcom/ds/nsObserverService.cpp, line 238]
nsXREDirProvider::DoStartup()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/toolkit/xre/nsXREDirProvider.cpp, line 597]
XRE_main()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/toolkit/xre/nsAppRunner.cpp, line 97]
main()  [/builds/tinderbox/thunderbird-trunk/Linux_2.4.18-14_Depend/mozilla/mail/app/nsMailApp.cpp, line 63]
libc.so.6 + 0x14ea2 (0xb745aea2)

Comment 14

[Dan Mosedale (:dmosedale, :dmose)]

I've confirmed that both Mac and Windows branch builds of Lightning work with the 1.5 Thunderbird release bits on those platforms.  Which is beginning to make it smell more like ABI lossage, despite my comment 11.

Comment 15

[Stefan Sitter [:ssitter]]

(In reply to comment #14)
> I've confirmed that both Mac and Windows branch builds of Lightning work with
> the 1.5 Thunderbird release bits on those platforms.  Which is beginning to
> make it smell more like ABI lossage, despite my comment 11.

If I interpret http://gcc.gnu.org/onlinedocs/libstdc++/abi.html correct gcc 3.2.3 and gcc 3.3.3 use a different ABI:
  gcc-3.2.3: GLIBCPP_3.2.2, CXXABI_1.2
  gcc-3.3.3: GLIBCPP_3.2.3, CXXABI_1.2.1

Comment 16

[Dan Mosedale (:dmosedale, :dmose)]

Chase: it looks like for the Linux Tinderbox build, we really are going to need to use a toolchain that's closer on the ABI front to the one that Thunderbird 1.5 was built with.  It turns out that all most (all?) C++ XPCOM components depend on libstdc++ for operator new() and operator delete() and variants, so that ABI is relevant too.  :-(

Comment 17

[Chase Phillips]

Mass reassign of open bugs for [e-mail] to [e-mail].

Comment 18

[J. Paul Reed [:preed]]

dmose: would putting these tinderboxen on the same machines that are used for Thunderbird 1.5.0.1 builds be OK?

Comment 19

[Simon Paquet [:sipaq]]

(In reply to comment #18)
> dmose: would putting these tinderboxen on the same machines that are used for
> Thunderbird 1.5.0.1 builds be OK?

I think so, but why don't we just update GCC to 3.3.3 on the current Lightning tinderbox machines?

Comment 20

[J. Paul Reed [:preed]]

(In reply to comment #19)
> I think so, but why don't we just update GCC to 3.3.3 on the current Lightning
> tinderbox machines?

Until we have a more coherrent configuration management story, it's easier to put products that rely on being built with the same toolchain *in* the same environment, as opposed to trying to make one environment look like another.

It's faster for the people doing the work, it's easier, and it's more assertably correct.

Comment 21

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to comment #18)
> dmose: would putting these tinderboxen on the same machines that are used for
> Thunderbird 1.5.0.1 builds be OK?
> 

Assuming that's the same toolchain used to build Thunderbird 1.5, that sounds great.

Comment 22

[Oliver Braun]

Attachment: link components against libsupc++.a and export only NSGetModule

This patch changes COMPONENT=1 libraries on Linux to no longer link against libstdc++.so, but against static libsupc++.a. It also introduces a minimal version script reducing the exported symbols to just NSGetModule.

Comment 23

[Oliver Braun]

My patch is a rather radical approach, since it changes the way all XPCOM components are linked. I also have to admit that I haven't verified if it really fixes the problem (since I do not have a gcc 3.2 at hand currently.

However, I was able to use the lightning.xpi build with gcc 3.4.1 in the official thunderbird 1.5 release and thought it might be worth mentioning.

Comment 24

[timeless]

iirc we had issues with the licensing of the library with which you're trying to link statically.

Comment 25

[Benjamin Smedberg]

It's still in my plans to do something like this, but instead of using libsupc++ to build an in-tree support lib which provides ::operator new and the very few other C++ symbols we typically need.

Comment 26

[Oliver Braun]

(In reply to comment #24)
> iirc we had issues with the licensing of the library with which you're trying
> to link statically.

Hmm, libsupc++ is part of the gcc suite. Are you sure we're talking about the same library ? 

Comment 27

[timeless]

yes.

Comment 28

[Carl Ponder]

Will the latest drop work with Thunderbird 1.5 yet?

Comment 29

[Dan Mosedale (:dmosedale, :dmose)]

preed: for some reason, the download link on the new tinderbox leads to an empty directory.  Can you have a look?

Comment 30

[J. Paul Reed [:preed]]

Attachment: Patch to post-mozilla-rel.pl to handle the lightning deliverables case

Comment 31

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 213529 [details] [diff] [review]
Patch to post-mozilla-rel.pl to handle the lightning deliverables case

r=dmose; thanks muchly!

Comment 32

[Stefan Sitter [:ssitter]]

I'm sorry to disturb the party but ...

I tested with Lightning 2006-03-01-mozilla1.8 downloaded from
http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-mozilla1.8/linux-xpi/

It works now with Thunderbird 1.5 (20051201) downloaded from
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/1.5/linux-i686/en-US/
(from about:buildconfig: gcc version 3.3.3)

But: Still fail with Thunderbird 1.5.0.2 (20060301) downloaded from 
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-mozilla1.8.0/
(from about:buildconfig: gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-53))

I still see a crash: TB15804953W, TB15804529M, TB15804416M

Please clarify if this an issue with the tinderbox used or if I should reopen the bug.

Comment 33

[Dan Mosedale (:dmosedale, :dmose)]

Good catch, Stefan.  I just spun off bug 329047 about that problem.

Comment 34

[Stefan Sitter [:ssitter]]

This was regressed recently on the Linux lightning-linux-tbox (http://tinderbox.mozilla.org/Sunbird-Mozilla1.8/):

Works with lightning-2006-04-06-06-mozilla1.8 linux xpi and Tbird 1.5 (20051201)
Crash with lightning-2006-04-06-12-mozilla1.8 linux xpi and Tbird 1.5 (20051201)
from http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/

According to the build logs Lightning is build again with gcc 3.2.3 instead of gcc 3.3.2:

From 2006-04-06-06 build log:
    PATH=/opt/gcc-3.3.2rh/bin:/usr/kerberos/bin:/usr/local/bin:/bin:...
    Compiler is -- gcc (gcc (GCC) 3.3.2 20031022 (Red Hat Linux 3.3.2-1)

From 2006-04-06-12 build log:
    PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:...
    Compiler is -- gcc (gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-53)

Note: There were some problems with the build between 2006-04-06-06 and 2006-04-06-12. There are also two nightly build available for 2006-04-06. The first is Ok, the second crashes Thunderbird. Somehow the tinderbox configuration was changed between this two times.

Please check.

Comment 35

[Dan Mosedale (:dmosedale, :dmose)]

preed: This is making it impossible for Linux users who don't want to do their own builds to test if bugs they saw in 0.1 still exist in the nightly.  Is this something you might have time to poke at soon?

Comment 36

[Dan Mosedale (:dmosedale, :dmose)]

It was pointed out to me that, in fact, testing is possible by using the Thunderbird next to the nightly XPI.  That'll teach me to post to bugs before my coffee.  It would nonetheless be very helpful to folks who don't want to use bleeding edge versions of Lightning if this could be fixed.  Do I recall correctly that simply changing a PATH might be sufficient to fix it?

Comment 37

[Stefan Sitter [:ssitter]]

After a tip from dmose that it should be fixed now I retested today:

BRANCH:
Tested lightning-2006-06-11-06-mozilla1.8 with Thunderbird 1.5 (20051201) / Thunderbird 1.5.0.4 (20060516) / Thunderbird 2.0a1 (20060611) on Linux - OK, working again.

TRUNK:
Tested lightning-2006-06-11-06-trunk with Thunderbird 3.0a1 (20060611) - Not OK, still crashing. According to Lightning build log GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52) is still used.

Comment 38

[Dan Mosedale (:dmosedale, :dmose)]

Resetting priority to normal, since the critical part has been fixed.

Comment 39

[Stefan Sitter [:ssitter]]

(In reply to comment #37)
Retest with 2006-07-10 builds: Branch build is ok, trunk build still crash.

Comment 40

[Matthew (lilmatt) Willis]

(In reply to comment #39)
> (In reply to comment #37)
> Retest with 2006-07-10 builds: Branch build is ok, trunk build still crash.

We respun the trunk builds. Lightning on Linux was the first to complete.

It doesn't crash for me on Ubuntu.

Thunderbird 3.0a1 20060710
Lightning 2006071013

Comment 41

[Joey Minta]

(In reply to comment #40)
> (In reply to comment #39)
> > (In reply to comment #37)
> > Retest with 2006-07-10 builds: Branch build is ok, trunk build still crash.
> 
> We respun the trunk builds. Lightning on Linux was the first to complete.
> 
> It doesn't crash for me on Ubuntu.
> 
> Thunderbird 3.0a1 20060710
> Lightning 2006071013
> 
So is this bug fixed now?

Comment 42

[Thorsten 'nightrat' Fritz]

Tested with Thunderbird 3.0a1 20060810 and Lightning 0.1+ 20060810, it is still crashing.

Requesting blocking 0.3

Comment 43

[Joey Minta]

(In reply to comment #42)
> Tested with Thunderbird 3.0a1 20060810 and Lightning 0.1+ 20060810, it is still
> crashing.
> 
> Requesting blocking 0.3
> 
Has anyone been doing any kind of QA on Thunderbird trunk?  Do we have any reasonable assurances that it will be shippable come release time?  I'm inclined to say this doesn't block, since Thunderbird trunk won't be ready to ship then, but I'm open to arguments to the contrary.

Comment 44

[Thorsten 'nightrat' Fritz]

I use Thunderbird trunk versions for many months now and it is usable.

I think Lightning 0.3 should run with Thunderbird trunk. So testers of the trunk can test Lighting too and do not have to run Thunderbird branch builds to test Lightning..

Comment 45

[Joey Minta]

(In reply to comment #44)
> I use Thunderbird trunk versions for many months now and it is usable.
> 
> I think Lightning 0.3 should run with Thunderbird trunk. So testers of the
> trunk can test Lighting too and do not have to run Thunderbird branch builds to
> test Lightning..
> 
OK. I'm sold on those points.  My next question is what we gain by blocking the release on this.  My intuition is that users of Thunderbird trunk are more than willing (and even likely) to upgrade to Lightning 0.3+ later.  Does it really make sense to make regular users wait for 0.3?  Releasing 0.3 and then fixing this soon in 0.3+ seems like a more reasonable approach.

Comment 46

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to comment #44)
> I think Lightning 0.3 should run with Thunderbird trunk. 

Lightning 0.3 is defined as the set of bits that we will be releasing from the branch for use with Thunderbird 1.5 / 2.0.  So saying that Lightning 0.3 should run with the trunk isn't really meaningful.

> So testers of the trunk can test Lighting too and do not have to run
> Thunderbird branch builds to test Lightning..

You're right that we do need to fix this bug.  However, tying it to the 0.3 release cycle doesn't really accomplish anything interesting that I can see.

Comment 47

[Dan Mosedale (:dmosedale, :dmose)]

Does not block 0.3; see comment 46.

Comment 48

[Stefan Sitter [:ssitter]]

The build servers have been upgraded with Bug 369874 and Bug 332900 and both servers now use gcc 4.1.2 compiler for building.

I see no crash using Lightning/0.6a1 (2007032503) with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a3pre) Gecko/20070325 Thunderbird/3.0a1

-> Resolving as fixed.