Enable crash stack reporting on mozmill tests in tinderbox

RESOLVED FIXED in Thunderbird 3.1b1

Status

defect
RESOLVED FIXED
10 years ago
10 years ago

People

(Reporter: standard8, Assigned: rain1)

Tracking

Trunk
Thunderbird 3.1b1
Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird3.1 beta1-fixed)

Details

(Whiteboard: [notacrash])

Attachments

(4 attachments)

Crash stack reporting won't currently work with mozmill. To get it to work we probably want to do a combination of the following:

1) Fix the unittest mozconfigs for Mac to specify -gdwarf-2 (unit test build boxes already do make buildsymbols).
2) Switch the runtest.py for mozmill from automation.process to automation.runApp.

I've not looked at item 2 in detail, if we can't do that we may have to do a combination of copy/pasting and setting environment variables, but hopefully not.
Whiteboard: [notacrash]
Component: Build Config → Testing Infrastructure
QA Contact: build-config → testing-infrastructure
Actually, comment 0 seems completely bogus.

The main part to do here is to implement a call to the checkForCrashes function as we do in the bloat tests:

http://mxr.mozilla.org/comm-central/source/mailnews/test/performance/bloat/runtest.py#222

I think that should get all the information necessary.

iirc this can be tested to some extent by creating a file in the PROFILE/minidumps directory (it just won't actually be able to process the stack).
One thing I noticed while writing this patch was that automationutils specifies that --symbols-path should be absolute <http://mxr.mozilla.org/comm-central/source/mozilla/build/automationutils.py#82>, but we seem to pass in a relative path to checkForCrashes for the bloat tests. This is because DIST is relative at <http://mxr.mozilla.org/comm-central/source/mailnews/testsuite-targets.mk#44> -- and this is easily verified by running make mailbloat or by looking at tinderbox output.

Should this be fixed?
Assignee: nobody → sid.bugzilla
Status: NEW → ASSIGNED
(In reply to comment #2)
> Should this be fixed?

It probably doesn't hurt to fix it, but I do know (from a bustage a couple of days ago) that bloat crash stacks are working on trunk.
Depends on: 547740
Posted patch patch, v1Splinter Review
Tested with a fake dump inside the minidumps folder.
Attachment #428729 - Flags: review?(bugzilla)
Comment on attachment 428729 [details] [diff] [review]
patch, v1

>+        if checkForCrashes(os.path.join(self._profile_dir, 'minidumps'),
>+                           SYMBOLS_PATH, TEST_NAME):
>+            print >> sys.stderr, 'TinderboxPrint: ' + TEST_NAME + '<br/><em class="testfail">CRASH</em>'
>+            sys.exit(-1)

nit: -1 isn't supported by windows, 1 would be better.
Attachment #428729 - Flags: review?(bugzilla) → review+
Checked in: http://hg.mozilla.org/comm-central/rev/3ce9d0c923d4
Target Milestone: --- → Thunderbird 3.1b1
What seems to be happening with failures like <http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTest/1267204120.1267207018.18085.gz#err101> is that MozMill is raising an exception much before it decides to clean up. This patch should make sure the crash detection code is executed regardless of whether MozMill throws an exception.
Attachment #429170 - Flags: review?(bugzilla)
Comment on attachment 429170 [details] [diff] [review]
catch exceptions raised by mozmill too

Yep, let's give this a try.
Attachment #429170 - Flags: review?(bugzilla) → review+
Checked in: http://hg.mozilla.org/comm-central/rev/4b3a1e2c92eb

It appears the only issue remaining is to get runtestlist.py to actually pass symbols path to runtest.py....
ouch, how in the world did I miss that :(
Attachment #429297 - Flags: review?(bugzilla)
Attachment #429297 - Attachment is patch: true
Attachment #429297 - Attachment mime type: application/octet-stream → text/plain
Comment on attachment 429297 [details] [diff] [review]
pass in the symbols path from runtestlist.py

This looks better. Checked in as well:

http://hg.mozilla.org/comm-central/rev/400c636c00c9
Attachment #429297 - Flags: review?(bugzilla) → review+
Depends on: 549067
This patch is on top of the one in bug 545085.

(I actually tested this locally :p )
Attachment #430065 - Flags: review?(bugzilla)
Attachment #430065 - Flags: review?(bugzilla) → review+
Confirmed to work: http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTest/1267855861.1267858631.14684.gz
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.