Closed Bug 711179 Opened 8 years ago Closed 8 years ago

No longer getting stacks + symbols for crashes on tinderbox

Categories

(Release Engineering :: General, defect, P3, critical)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: justin.lebar+bug, Assigned: bear)

References

Details

(Whiteboard: [symbols])

Attachments

(1 file)

See [1].  The crashstack contains entries like

 1  libxul.so + 0x174766d

instead of a function + line no.

I don't know why this is happening, but it certainly makes debugging less pleasant.  bz and ted remember that we used to get useful stacks on both release and debug builds, so I presume this isn't fallout from bug 669953?

[1] Debug mochitest-1 log on Linux-64: https://tbpl.mozilla.org/php/getParsedLog.php?id=7957621&tree=Try#error0
10.6 from what I've got handy - https://tbpl.mozilla.org/php/getParsedLog.php?id=7956447&tree=Mozilla-Inbound

I started out thinking it was 10.7-only, and then I finally saw one (out of probably ten) of them which wasn't busted.
OS: Linux → All
Hardware: x86_64 → All
Windows too: https://tbpl.mozilla.org/php/getParsedLog.php?id=7958437&tree=Try#error0
Summary: No longer getting stacks + symbols for crashes on tinderbox (at least Linux, maybe elsewhere) → No longer getting stacks + symbols for crashes on tinderbox
I know the cm-ixstore01 outage has probably made it hard, but has anyone looked back in time to find the window for this yet?
Priority: -- → P3
Whiteboard: [symbols]
I complained about <https://tbpl.mozilla.org/php/getParsedLog.php?id=7981661&tree=Firefox> on IRC and ted found:

<ted> Ms2ger: i think RelEng broke this
<ted> looking at that log file, they unzip the symbols to   PWD=/Users/cltbld/talos-slave/test/build/symbols
<ted> but then they run
<ted> python reftest/runreftest.py --appname=./FirefoxNightly.app/Contents/MacOS/firefox-bin --utility-path=bin --extra-profile-file=bin/plugins --symbols-path=../symbols reftest/tests/layout/reftests/reftest.list
<ted>   PWD=/Users/cltbld/talos-slave/test/build
<ted> so it's looking for symbols in /Users/cltbld/talos-slave/test/symbols
<ted> Ms2ger: probably fallout from https://bugzilla.mozilla.org/show_bug.cgi?id=693352
Joel,

from what the IRC conversation above says, our change to get crashdumps into tegra output may have broken this.  Can you look to see if it's something we need to adjust?
Assignee: nobody → bear
The work done on the mobile stuff was on the foopies, not the test slaves.  The only thing I can think of is that there was some common code changed for downloading the symbols that affected the foopies and the test slaves.
Can we get this fixed up on Monday?  Diagnosing crashes on try on platforms you don't have is impossible now.
Severity: normal → critical
I'm not really sure of all the changes that were made, but the patch in attachment 578011 [details] [diff] [review] makes a lot of changes to set symbols-path=../symbols. Perhaps whatever corresponding change downloads the symbols to the right directory didn't make it to the desktop test runs?
new rule: thou shalt not use s///g
Attachment #582827 - Flags: review?(catlee)
Attachment #582827 - Flags: review?(catlee) → review+
Comment on attachment 582827 [details] [diff] [review]
remove ../ from symbols path for *non* tegra tests

committed changeset 1958:7f86840c319d
Attachment #582827 - Flags: checked-in+
reconfig has landed - this should be un-hork'd
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.