301266 - Unable to get useful stacks for Linux builds from talkback

Status: RESOLVED FIXED

(Core Graveyard :: Talkback Client, defect)

Description

[Boris Zbarsky [:bzbarsky]]

I've had a few crashes over the last day, and I sent in talkbacks, but the
talkback server is just showing me library names in the stacks...  See
http://talkback-public.mozilla.org/talkback/fastfind.jsp?search=1&searchby=comments&match=contains&searchfor=zbarsky&vendor=All&product=All&platform=All&buildid=&sdate=&stime=&edate=&etime=&sortby=bbid
for the stacks (should be one or two Firefox and one Seamonkey).

I've been seeing a lot of crashes on trunk lately, and I'd really like to try to
sort out what's going on....

Comment 1

[Jay Patel [:jay]]

This is most likely due to symbols not being uploaded to HAL (the Talkback
processing server and symbols repository).  We run out of space fairly quickly
these days because we only have about 136 GB of space to store symbols for all
of our products on 3 platforms and for both the trunk and branches. I have to
manually delete old symbols on a regular basis.  

I am currently out of town and did not get around to doing that this past
weekend (just got to it today), so we probably won't be getting good data for
any builds from 7/17-7/19.  Sorry for any inconvenience this has caused... I
have brought up the space issue and hopefully we can get things rolling to fix
the problem soon.

Comment 2

[Jay Patel [:jay]]

Boris, if you can try to reproduce your crashes with the latest nightlies it
will be nice to verify if your problems with the stacks was due to missing
symbols.  

Comment 3

[Boris Zbarsky [:bzbarsky]]

By "latest nightlies" I assume you mean the 7/20 builds once those exist
tomorrow?  Given that 7/17-7/19 is what's supposed to not work and that the 7/19
builds don't even exist yet...

Comment 4

[Jay Patel [:jay]]

Yes, I meant to say the 7/20 builds.  And after looking at the symbols dirs...it
looks like things are back to normal after I cleaned up some space.  We should
be good for a while, and we are working on adding more storage to the servers as
well.

Comment 5

[Boris Zbarsky [:bzbarsky]]

So... I can verify that Firefox works now.  Seamonkey is still not working --
see
http://talkback-public.mozilla.org/talkback/fastfind.jsp?search=2&type=iid&id=7650206
for example (which is a July 20 build).

Comment 6

[Jay Patel [:jay]]

Reopening.  I haven't been paying much attention to the Seamonkey Trunk symbols,
but after a quick look, it seems as if we have not had any symbols for Mac or
Linux builds since 2/17/2005 (20050217).  

Since this probably isn't a Talkback server side issue and more likely a build
config/automation issue, I'm reassigning to Chase for more investigation.

Comment 7

[Jay Patel [:jay]]

This seems to only be a problem for the MozillaTrunk symbols for MacOSX and
LinuxIntel... Mozilla17 symbols look ok on all platforms.

Comment 8

[Chase Phillips]

(In reply to comment #6)
> Since this probably isn't a Talkback server side issue and more likely a build
> config/automation issue, I'm reassigning to Chase for more investigation.

I don't have time to look into this right now.  Jay, you would be helping big
time if you could keep looking into this by checking out the log files and
looking on the symbol server.

Comment 9

[Chase Phillips]

cc'ing coop.  Maybe he can scan some of those log files, too.

Comment 10

[Jay Patel [:jay]]

I'll be gone until Sunday night, so I can look into this next week.  If Coop has
a chance to do some digging before then, great.  If not, I'll see what I can do
when I get back from my vacation.

Comment 11

[Chris Cooper [:coop] (he/him)]

I'll look at the logs for the tinderboxen in question and make sure we're
actually trying to upload symbols and such.

Comment 12

[Chase Phillips]

This problem came about due to the rebranding.  The build error:

  make -C /builds/tinderbox/Mozilla-Trunk/Darwin_6.8_Depend/mozilla/fullsoft
fullcircle-push
  rm -f ../dist/bin/mozilla-bin.elf
  cp ../dist/bin/mozilla-bin ../dist/bin/mozilla-bin.elf
  cp: ../dist/bin/mozilla-bin: No such file or directory
  make: *** [../dist/bin/mozilla-bin.elf] Error 1

The problem was in fullsoft/Makefile.in where it was still using 'mozilla' for
the program base name instead of 'seamonkey'.  I updated that file to use the
new binary name.