304842 - Talkback data are nearly useless for OS X Firefox: most crashes appear as firefox-bin + offset

Status: VERIFIED FIXED

(Firefox Build System :: General, defect)

Description

[Mark Mentovai]

The talkback server isn't able to decode symbols in firefox-bin on Mac OS X. 
See
http://talkback-public.mozilla.org/reports/firefox/FFTrunk/FFTrunk-topcrashers.html
, and note the abundance of Mac crashes in firefox-bin + OFFSET.

The talkback server is able to figure out symbols in dynamic libraries, and
talkback data is working for Camino again, so it should be able to work.

Is firefox-bin being stripped before talkback symbols are uploaded?

Comment 1

[Mark Mentovai]

Most dylibs are affected too.

Trunk is broken, 1.0.x seems OK.  (Marking as regression.)

Comment 2

[Mike Connor [:mconnor]]

Let's try to get this in if possible.

Comment 3

[Benjamin Smedberg]

Comparing the build logs with those of four weeks ago doesn't show any obvious
problems (no early stripping, and the -g flag to --enable-optimize is there.)

I don't really know how symbol upload works: have there been any tinderbox
configuration changes that might cause us not to upload symbols for firefox-bin ?

Comment 4

[Mark Mentovai]

I initially suspected dead_strip (300047) but it seems that this regressed even
earlier than that.  I don't know if the trunk was ever good.  The 1.0.x branch
has symbols.

Comment 5

[Chase Phillips]

bryner had me add '-g' to the Firefox trunk builds prior to alpha2.  After I
added it he said he was able to get debug information out of Talkback.

(In reply to comment #3)
> I don't really know how symbol upload works: have there been any tinderbox
> configuration changes that might cause us not to upload symbols for firefox-bin ?

What's being generated is being uploaded.  Build logs after Saturday/Sunday are
slightly different from before because symbols are now compressed, uploaded, and
uncompressed instead of being transferred uncompressed over the wire.

bryner, can you confirm or deny this has regressed since alpha2?  Or perhaps you
only wanted a subset of data from the symbols that '-g' got you and this problem
existed then as well?

Comment 6

[Mike Shaver (:shaver emeritus)]

01:10 < bryner> i think the talkback server may or may not require a bogus .elf 
                extension on the binary
01:10 < bryner> and we should try it the opposite way of how it is

Comment 7

[Chase Phillips]

(In reply to comment #6)
> 01:10 < bryner> i think the talkback server may or may not require a bogus .elf 
>                 extension on the binary
> 01:10 < bryner> and we should try it the opposite way of how it is

Here are the symbols on the symbol server from the Mozilla 1.8 branch:

  ~/symbols/MozillaOrg/Firefox15/MacOSX/2005082505
  $ ls -al
  total 459445
  drwxrwxrwx+  2 symbols None         0 Aug 25 06:14 .
  drwxrwxrwx+ 18 symbols None         0 Aug 25 06:41 ..
  -rwxr-xr-x   1 symbols None 444710108 Aug 25 06:14 firefox-bin.elf
  -rwxrwxrwx   1 symbols None    755896 Aug 25 06:14 libjsd.dylib
  -rwxrwxrwx   1 symbols None   2181888 Aug 25 06:14 libmozjs.dylib
  -rwxrwxrwx   1 symbols None    781028 Aug 25 06:14 libnspr4.dylib
  -rwxrwxrwx   1 symbols None    563564 Aug 25 06:14 libnss3.dylib
  -rwxrwxrwx   1 symbols None    285556 Aug 25 06:14 libnssckbi.dylib
  -rwxrwxrwx   1 symbols None    113604 Aug 25 06:14 libplc4.dylib
  -rwxrwxrwx   1 symbols None     97976 Aug 25 06:14 libplds4.dylib
  -rwxrwxrwx   1 symbols None    185424 Aug 25 06:14 libsmime3.dylib
  -rwxrwxrwx   1 symbols None    670528 Aug 25 06:14 libsoftokn3.dylib
  -rwxrwxrwx   1 symbols None    153900 Aug 25 06:14 libssl3.dylib
  -rwxr-xr-x   1 symbols None     28748 Aug 25 06:14 libxpcom.dylib
  -rwxrwxrwx   1 symbols None   1068868 Aug 25 06:14 libxpcom_compat.dylib
  -rwxrwxrwx   1 symbols None  12243696 Aug 25 06:14 libxpcom_core.dylib
  -rwxrwxrwx   1 symbols None   6356420 Aug 25 06:14 libxpinstall.dylib
  -rwxrwxrwx   1 symbols None    267184 Aug 25 06:14 libxpistub.dylib

Symbols for Firefox 1.0 use the same .elf extensions:

  ~/symbols/MozillaOrg/Firefox10/MacOSX/2005082504
  $ ls -l
  total 381995
  -rwxr-xr-x  1 symbols None 363735184 Aug 25 05:07 firefox-bin.elf
  -rwxrwxrwx  1 symbols None   3287740 Aug 25 05:07 libinspector.dylib
  -rwxrwxrwx  1 symbols None    750072 Aug 25 05:07 libjsd.dylib
  -rwxrwxrwx  1 symbols None   1784636 Aug 25 05:07 libmozjs.dylib
  -rwxrwxrwx  1 symbols None    322936 Aug 25 05:07 libnegotiateauth.dylib
  -rwxrwxrwx  1 symbols None    782200 Aug 25 05:07 libnspr4.dylib
  -rwxrwxrwx  1 symbols None    550452 Aug 25 05:07 libnss3.dylib
  -rwxrwxrwx  1 symbols None    265436 Aug 25 05:07 libnssckbi.dylib
  -rwxrwxrwx  1 symbols None    113676 Aug 25 05:07 libplc4.dylib
  -rwxrwxrwx  1 symbols None     97976 Aug 25 05:07 libplds4.dylib
  -rwxrwxrwx  1 symbols None    139740 Aug 25 05:07 libqfaservices.dylib
  -rwxrwxrwx  1 symbols None    184268 Aug 25 05:07 libsmime3.dylib
  -rwxrwxrwx  1 symbols None    665344 Aug 25 05:07 libsoftokn3.dylib
  -rwxrwxrwx  1 symbols None    148940 Aug 25 05:07 libssl3.dylib
  -rwxrwxrwx  1 symbols None  10919264 Aug 25 05:07 libxpcom.dylib
  -rwxrwxrwx  1 symbols None   1017660 Aug 25 05:07 libxpcom_compat.dylib
  -rwxrwxrwx  1 symbols None   6137612 Aug 25 05:07 libxpinstall.dylib
  -rwxrwxrwx  1 symbols None    251140 Aug 25 05:07 libxpistub.dylib

My view of this problem is affected by the knowledge that this worked or didn't
work 1-2 months ago -- I'll ask again:

(In reply to comment #5)
> bryner, can you confirm or deny this has regressed since alpha2?  Or perhaps you
> only wanted a subset of data from the symbols that '-g' got you and this problem
> existed then as well?

Comment 8

[Jay Patel [:jay]]

Looking at the digester logs, I saw this:

$ more digester*.log | grep firefox-bin
08/25 14:36:05 (08E0): Derived module name firefox-bin
08/25 14:36:05 (08E0): Derived module name firefox-bin + (003e09ea)
08/25 14:38:44 (08E0): WARNING: ELFFile::GetELFSectionInfo: Section header
starts after end of file
[C:\builds\supportsoft_branch_r1\ns\talkback\Server\SymbolAccess\ELFFile.cpp,143]08/25
14:38:44 (08E0): spUnixELFSymbolAccess::ResolveProcedure: couldn't find symbol
table08/25 14:38:44 (08E0): Derived calling function name firefox-bin + 0x1cbc34
(0x08213c34)
08/25 14:38:44 (08E0): WARNING: ELFFile::GetELFSectionInfo: Section header
starts after end of file
[C:\builds\supportsoft_branch_r1\ns\talkback\Server\SymbolAccess\ELFFile.cpp,143]08/25
14:38:44 (08E0): spUnixELFSymbolAccess::ResolveProcedure: couldn't find symbol
table08/25 14:38:44 (08E0): Derived calling function name firefox-bin + 0x58dd93
(0x085d5d93)
08/25 14:38:44 (08E0): WARNING: ELFFile::GetELFSectionInfo: Section header
starts after end of file
[C:\builds\supportsoft_branch_r1\ns\talkback\Server\SymbolAccess\ELFFile.cpp,143]08/25
14:38:44 (08E0): spUnixELFSymbolAccess::ResolveProcedure: couldn't find symbol
table08/25 14:38:44 (08E0): Derived calling function name firefox-bin + 0x73935c
(0x0878135c)
08/25 14:38:44 (08E0): WARNING: ELFFile::GetELFSectionInfo: Section header
starts after end of file
[C:\builds\supportsoft_branch_r1\ns\talkback\Server\SymbolAccess\ELFFile.cpp,143]08/25
14:38:44 (08E0): spUnixELFSymbolAccess::ResolveProcedure: couldn't find symbol
table08/25 14:38:44 (08E0): Derived calling function name firefox-bin + 0x2bae3
(0x08073ae3)
08/25 14:45:59 (08E0): spUnixMachOSymbolAccess::MapFileIntoMemory: CreateFile
e:\symbols\MozillaOrg\FirefoxTrunk\MacOSX\2005081506\firefox-bin failed

bryner:  Do those warnings tell us anything about this problem?

cc'ing Shiva as well, in case he can help.

Comment 9

[Chase Phillips]

Bryner filed
http://talkback-public.mozilla.org/talkback/fastfind.jsp?search=2&type=iid&id=8716077
using Deer Park Alpha 2.  Same problem.

He suggests there could be a difference between compile/link flags on
trunk/Mozilla 1.8 branch and Mozilla 1.7/Aviary 1.0/Aviary 1.0.1 branches.  I'm
out of cycles to troubleshoot this right now.

Guys, I'm sure we want symbols for Mac builds but I'm overloaded with the time
crunch and this has a good chance of not getting all of my attention before
Wednesday while I do other things (verify partial update, ensure l10n builds are
being built, interviews, maintenance, AUS2 project coordination).  I'd really
appreciate any help people can give.

Comment 10

[Chase Phillips]

This was in the status whiteboard:

  [probable misconfiguration of branch build machines]

The same build system builds Firefox on Aviary-1.0.1 as builds trunk and Mozilla
1.8.  This seems less than probable to me (not discounting it completely).

Can someone show why some symbols would resolve from .dylib files while others
from firefox-bin won't and how a build system needs to be configured to lead to
that situation in one build tree and not another?

Comment 11

[Chase Phillips]

Josh, can you take a look at this bug so you can become familiar with the
details?  We're going to need your help here.

Comment 12

[Chase Phillips]

Because we can't be sure that rebuilding symbols for Talkback is possible after
that build tree is removed -- we've never done that before (it has high risk)
and there's some doubt (however small) that the build process is deterministic
-- we're going to take another run at this problem with Jay at the helm.  He'll
be looking to Josh, Mike, Mark, Josh, Bryner, and Dbaron for assistance.

Questions that, if answered, may help locate the problem:

  * Did this ever work on the trunk?
    * If so, when was the earliest we know that this regressed there?
  * When did this start working on the AVIARY_1_0_20040515_BRANCH?
  * Does this work on the MOZILLA_1_7_BRANCH?
    * If so, when did it start working?

atlantia is a Firefox Mac OS X build system.  It builds
AVIARY_1_0_1_20050124_BRANCH, HEAD, and MOZILLA_1_8_BRANCH.  If this works on
the Aviary-1.0.1 branch but not the Mozilla 1.8 branch, then build logs from
this system on those trees are a good place to start for comparison.

Bryner suggests looking for how nsAppRunner.cpp is compiled since symbols there
should resolve for every incoming stack trace.

Comment 13

[Scott MacGregor]

As a starting point, here's the output date for compiling nsAppRunner.cpp for
1.8(broken). The line is quite long so I've broken it down into several smaller
pieces that are easier to examine:

1.0.x Build (Good):
  nsAppRunner.cpp 
    c++ -o nsAppRunner.o -c -DOSTYPE=\"Darwin7.9.0\" -DOSARCH=\"Darwin\"
-D_BUILD_STATIC_BIN  

1.8 Build (bad)
nsAppRunner.cpp
c++ -o nsAppRunner.o -c  -DMOZ_UPDATER -DMOZILLA_INTERNAL_API
-DOSTYPE=\"Darwin7.9.0\" -DOSARCH=\"Darwin\" -DBUILD_ID=2005082906
-D_BUILD_STATIC_BIN -DOS_TARGET=\"Darwin\" -DTARGET_XPCOM_ABI=\"ppc-gcc3\"
-DTARGET_OS_ABI=\"Darwin_ppc-gcc3\"  -DTOOLKIT_EM_VERSION=\"1.7+\" 

Note all of the extra defines here with 1.8/trunk builds.

Good:
   -mdynamic-no-pic   -fno-rtti -fno-exceptions -Wall -Wconversion
-Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wno-long-long -

Bad:
  

 -mdynamic-no-pic   -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith
-Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wno-long-long -nostdinc -nostdinc++ -

Note the nostdinc and nostdinc++ flags which are only found in the bad build

Good:
  I/Developer/SDKs/MacOSX10.2.8.sdk/usr/include -fpascal-strings 
  -no-cpp-precomp -fno-common -fshort-wchar 
  -I/Developer/SDKs/MacOSX10.2.8.sdk/Developer/Headers/FlatCarbon -pipe  

Bad:
  I/Developer/SDKs/MacOSX10.2.8.sdk/usr/include/gcc/darwin/3.3/c++ 
  -I/Developer/SDKs/MacOSX10.2.8.sdk/usr/include/gcc/darwin/3.3/c++/ppc-darwin 
  -I/Developer/SDKs/MacOSX10.2.8.sdk/usr/include/gcc/darwin/3.3/c++/backward 
  -isystem /Developer/SDKs/MacOSX10.2.8.sdk/usr/include/gcc/darwin/3.3 -isystem 
  /Developer/SDKs/MacOSX10.2.8.sdk/usr/include   
  -F/Developer/SDKs/MacOSX10.2.8.sdk/System/Library/Frameworks -fpascal-strings 
  -no-cpp-precomp -fno-common -fshort-wchar 
  -I/Developer/SDKs/MacOSX10.2.8.sdk/Developer/Headers/FlatCarbon -pipe
-fexceptions  

There seemed to be a lot of differneces here.

Good:
  -DNDEBUG -DTRIMMED -O2 -g  
  -I/Developer/SDKs/MacOSX10.2.8.sdk/Developer/Headers/FlatCarbon   
  -DMOZILLA_CLIENT -include ../../mozilla-config.h -Wp,-MD,.deps/nsAppRunner.pp
nsAppRunner.cpp

Bad:
  -DNDEBUG -DTRIMMED -O2 -g 
  -I/Developer/SDKs/MacOSX10.2.8.sdk/Developer/Headers/FlatCarbon   
  -DMOZILLA_CLIENT -include ../../mozilla-config.h -Wp,-MD,.deps/nsAppRunner.pp
nsAppRunner.cpp

These two lines look the same to me

Comment 14

[Jay Patel [:jay]]

After some investigation, I believe the problem is related to the naming of the
firefox-bin.elf symbol file.  Although the .elf extension is required for Linux
incidents, it appears as if it's not needed for MacOSX.  

I removed the .elf from today's FirefoxTrunk and Firefox15 builds 20050829xx,
and the incidents that have come in since that change show properly resolved
stack traces.

Here is the test crash bryner submitted:
Incident ID: 8846926
Stack Signature	libSystem.B.dylib.88.0.0 + 0xa778 (0x9000a778) 3908b05b
Product ID	FirefoxTrunk
Build ID	2005082906
Trigger Time	2005-08-29 17:41:58.0
Platform	MacOSX
Operating System	Darwin 8.2.0
Module	libSystem.B.dylib.88.0.0 + (0000a778)
URL visited	
User Comments	test crash
Since Last Crash	56 sec
Total Uptime	56 sec
Trigger Reason	SIGSEGV: Segmentation Violation: (signal 11)
Source File, Line No.	N/A
Stack Trace 	
libSystem.B.dylib.88.0.0 + 0xa778 (0x9000a778)
libSystem.B.dylib.88.0.0 + 0xa6bc (0x9000a6bc)
CoreFoundation.368.12.0 + 0x22c3c (0x9074ac3c)
HIToolbox.221.0.0 + 0x8ac0 (0x93129ac0)
HIToolbox.221.0.0 + 0xed768 (0x9320e768)
HIToolbox.221.0.0 + 0xed51c (0x9320e51c)
HIToolbox.221.0.0 + 0xed47c (0x9320e47c)
nsMacMessagePump::GetEvent()   nsMacMessagePump::DoMessagePump()  
nsAppShell::Run() 
[/builds/tinderbox/Fx-Trunk/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsAppShell.cpp,
line 114]
nsAppStartup::Run()   XRE_main() 
[/builds/tinderbox/Fx-Trunk/Darwin_7.9.0_Depend/mozilla/toolkit/xre/nsAppRunner.cpp,
line 2324]
_start()   start()

And another random one for Firefox15:
Incident ID: 8846423
Stack Signature	nsDocShell::RestoreFromHistory() 292070e2
Product ID	Firefox15
Build ID	2005082905
Trigger Time	2005-08-29 17:20:13.0
Platform	MacOSX
Operating System	Darwin 8.2.0
Module	firefox-bin + (00687174)
URL visited	
User Comments	
Since Last Crash	2337 sec
Total Uptime	2337 sec
Trigger Reason	SIGBUS: Bus Error: (signal 10)
Source File, Line No.	txXSLTNumberCounters.cpp, line 848
Stack Trace 	
nsDocShell::RestoreFromHistory()   nsDocShell::RestoreFromHistory()  
PL_HandleEvent() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/xpcom/threads/plevent.c,
line 689]
PL_ProcessPendingEvents() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/xpcom/threads/plevent.c,
line 623]
CoreFoundation.368.12.0 + 0x23c8c (0x9074bc8c)
CoreFoundation.368.12.0 + 0x231bc (0x9074b1bc)
CoreFoundation.368.12.0 + 0x22c3c (0x9074ac3c)
HIToolbox.221.0.0 + 0x8ac0 (0x93129ac0)
HIToolbox.221.0.0 + 0xed64c (0x9320e64c)
HIToolbox.221.0.0 + 0xed51c (0x9320e51c)
HIToolbox.221.0.0 + 0xed47c (0x9320e47c)
nsMacMessagePump::GetEvent()   nsMacMessagePump::DoMessagePump()  
nsAppShell::Run() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsAppShell.cpp,
line 114]
nsAppStartup::Run()   XRE_main() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/toolkit/xre/nsAppRunner.cpp,
line 2324]
_start()   start()

Both those stacks look good to me, so we just need to figure out where that file
gets created and named and make sure it doesn't have the .elf extension for
MacOSX builds when it is pushed to HAL.

Comment 15

[Chase Phillips]

I modified the build code for Talkback to not append .elf to the Mac binary
image on the MOZILLA_1_8_BRANCH.  I'll respin atlantia on the branch tonight. 
I'm skeptical but Jay and Bryner have described how (oddly) this code was not
being run for the 1.0-1.0.6 releases and sometime after 1.0.6 on the nightlies
things got bad -- .elf was appended and the latest 1.0.x nightlies don't resolve
symbols correctly.

Please test the Mozilla 1.8 branch respin or tomorrow's Mozilla 1.8 build and
let us know what you find.

Checking in Makefile.in;
/mofo/talkback/fullsoft/Makefile.in,v  <--  Makefile.in
new revision: 1.41.2.4; previous revision: 1.41.2.3
done

Comment 16

[Mark Mentovai]

I crashed 2005083006 (1.8) and submitted TB8865418G.  Look for it at a friendly
talkback server near you in the coming hours.  The Apple CrashReporter stack is
included in the notes with the report.  For reference, the top of the
CrashReporter stack:

0   <<00000000>> 	0x14000000 0 + 335544320
1   org.mozilla.firefox 	0x00519d70 ViewWrapper::GetInterface(nsID const&,
void**) + 468
2   org.mozilla.firefox 	0x006a48a8 nsWindow::DispatchEvent(nsGUIEvent*,
nsEventStatus&) + 172
3   org.mozilla.firefox 	0x006a4970 nsWindow::DispatchWindowEvent(nsGUIEvent&,
nsEventStatus&) + 36

(This crash occurred by pressing control-space in the extension manager a few
times.)

Comment 17

[Jay Patel [:jay]]

Here's Mark's crash according to Talkback:
Incident ID: 8865418
Stack Signature	0x14000000 2a479fad
Product ID	Firefox15
Build ID	2005083005
Trigger Time	2005-08-30 08:46:56.0
Platform	MacOSX
Operating System	Darwin 8.2.0
Module	
URL visited	https://bugzilla.mozilla.org/show_bug.cgi?id=304842
User Comments	Test for bug 304842. (Pressed control-space in the extension
manager a few times.) Date/Time: 304842-08-30 11:46:47.291 -0400 OS Version:
10.4.2 (Build 8C46) Report Version: 3 Command: firefox-bin Path:
Since Last Crash	14 sec
Total Uptime	906 sec
Trigger Reason	SIGSEGV: Segmentation Violation: (signal 11)
Source File, Line No.	N/A
Stack Trace 	
0x14000000
nsViewManager::DispatchEvent() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/view/src/nsViewManager.cpp,
line 2020]
nsWindow::DispatchEvent() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 1809]
nsWindow::DispatchWindowEvent() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 1832]
nsWindow::UpdateWidget() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 1635]
nsWindow::PaintUpdateRectProc() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 1285]
nsWindow::HandleUpdateEvent() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 70]
nsWindow::Update() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsWindow.cpp,
line 1241]
nsMacWindow::WindowEventHandler() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsMacWindow.cpp,
line 686]
HIToolbox.221.0.0 + 0x78d4 (0x931288d4)
HIToolbox.221.0.0 + 0x702c (0x9312802c)
HIToolbox.221.0.0 + 0x6ea8 (0x93127ea8)
HIToolbox.221.0.0 + 0xda86c (0x931fb86c)
HIToolbox.221.0.0 + 0xe154 (0x9312f154)
HIToolbox.221.0.0 + 0x7b24 (0x93128b24)
HIToolbox.221.0.0 + 0x702c (0x9312802c)
HIToolbox.221.0.0 + 0xddb0 (0x9312edb0)
HIToolbox.221.0.0 + 0x4ece0 (0x9316fce0)
HIToolbox.221.0.0 + 0xedce4 (0x9320ece4)
HIToolbox.221.0.0 + 0xed938 (0x9320e938)
HIToolbox.221.0.0 + 0xed790 (0x9320e790)
HIToolbox.221.0.0 + 0xed51c (0x9320e51c)
HIToolbox.221.0.0 + 0xed47c (0x9320e47c)
nsMacMessagePump::GetEvent()   nsMacMessagePump::DoMessagePump()  
nsAppShell::Run() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/widget/src/mac/nsAppShell.cpp,
line 114]
nsAppStartup::Run()   XRE_main() 
[/builds/tinderbox/Fx-Mozilla1.8/Darwin_7.9.0_Depend/mozilla/toolkit/xre/nsAppRunner.cpp,
line 2324]
_start()   start()

Looks nice enough to me. ;-)  Resolved Fixed?  We should probably wait until
other tinderboxes have been updated with this latest fix.

Comment 18

[Chase Phillips]

Landed this fix on the trunk, as well.

Comment 19

[Jesse Ruderman]

Does Firefox 1.0.7 RC3 suffer from this problem?  It crashed once and the
Talkback stack was full of "firefox-bin" lines.  TB9436880E

Comment 20

[Mark Mentovai]

Reopening, requesting blocking, and bumping severity per comment 19. 
TB9436880E, 1.0.7RC build 091409, has no symbols from firefox-bin but does have
symbols from libxpcom.dylib.  TB9464402Y, build 091516, also has no symbols in
firefox-bin.  Looks like the .elf suffix needs to be dropped on the aviary 1.0.x
branch too.

http://talkback-public.mozilla.org/talkback/fastfind.jsp?search=1&searchby=stacksig&match=contains&searchfor=&vendor=MozillaOrg&product=Firefox10&platform=MacOSX&buildid=200509

Comment 21

[Mark Mentovai]

Firefox 1.0.7 was released with build 2005091516 without this having been fixed.

TB9581228Q and
http://talkback-public.mozilla.org/talkback/fastfind.jsp?search=1&searchby=stacksig&match=contains&searchfor=&vendor=MozillaOrg&product=Firefox10&platform=MacOSX&buildid=2005091516&sdate=&stime=&edate=&etime=&sortby=bbid

There are no symbols for firefox-bin, but dylibs look OK.

Jay, can you flip firefox-bin.elf over to firefox-bin on hal and do whatever
other magic is necessary to pick up the symbols?

Chase, can .elf be dropped on the 1.0 branch so that if there's ever a 1.0.8,
this won't be such a headache?

Comment 22

[Jay Patel [:jay]]

Mark:  I have removed the .elf from firefox-bin for Firefox10MacOSX2005091516
(1.0.7) so that all incoming incidents from now on get pretty stacks. Please
keep an eye on new 1.0.7 crashes and make sure things look ok.  Thanks.

Leaving reopened so that Chase can make the build side changes so that we'll be
ok with future 1.0.x builds.

Comment 23

[Mark Mentovai]

This can't block 1.0.7 because 1.0.7 has shipped.

Over to Chase.

Comment 24

[Chase Phillips]

Fixed.  The fix should be verifiable in the next 1.0.x Mac nightly respin.

Comment 25

[Jay Patel [:jay]]

v.fixed on 1.0.x branch.

Comment 26

[Jay Patel [:jay]]

this is actually fixed everywhere now (no more .elf extension for Mac firefox-bin files).

The last build on MacOS X with symbols from was 20060109, so something else is broken, but the firefox-bin file does not have the .elf extension, so this bug is fixed.