483061 - Make minidump spew appear in brief logs

Status: RESOLVED FIXED

(Webtools Graveyard :: Tinderbox, defect)

Description

[Blake Kaplan (:mrbkap) (inactive)]

The crash stacks as a result of the minidump spew on tinderbox don't appear in the brief logs. As the full logs are very large, it would be nice if this were not the case.

It wasn't clear where to file this bug or where the right place to fix it is. Starting it out in RelEng for now.

For reference, this is what *does* appear in the brief log:
TEST-UNEXPECTED-FAIL | (automation.py) | Exited with code 1 during test run
INFO | (automation.py) | Application ran for: 0:13:55.148331
TEST-UNEXPECTED-FAIL | (automation.py) | Browser crashed (minidump found)
Operating system: Linux
                  0.0.0 Linux 2.6.18-53.1.19.el5 #1 SMP Wed May 7 08:20:19 EDT 2008 i686 GNU/Linux
CPU: x86
     GenuineIntel family 10 model 15 stepping 8
     1 CPU

 Skipping 437 Lines...

0x07c25000 - 0x07c2afff  libxpcomsample.so  ???
...

Comment 1

[(not currently active) Ted Mielczarek]

Yeah. So this is a combination of the way we're outputting the info, and tinderbox's error parser behavior. We spit out those TEST-UNEXPECTED-FAIL lines, which ep_unittest.pl regards as error lines. Good so far. Tinderbox has a couple of globals:
http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/showlog.cgi#34
my $LINES_AFTER_ERROR = 5;
my $LINES_BEFORE_ERROR = 30;
...that do exactly what you'd expect them to. These make sense for compiler errors, since there's usually useful info before the actual error. Here, the useful info is happening after the error. We could:
a) hack tinderbox to either increase LINES_AFTER_ERROR, or hack the error parser to give us more context
b) hack our output to trigger the error parser more frequently

Comment 3

[Phil Ringnalda (:philor)]

s/would be nice/would be the only way that people will ever do anything about crashes other than wait for another build or for someone to go ahead and check in on unstarred orange to produce another build/ - this is the number one cause of ignored orange by probably 20:1.

Comment 4

[Aki Sasaki (not active)]

Could we get the unit test script to prepend Error: in front of every line?

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Bug 536534 describes a simple solution to the log highlighter that would fix this.

Comment 6

[Aki Sasaki (not active)]

> The regex we want would be something like (depending on the particular regex
syntax):
>  ^Thread \d\+ \(crashed\)$

I'm currently busy w/ Fennec 1.0rc1, but if someone wants to take a stab at a patch, the perl-style regex would go in here:

http://mxr.mozilla.org/webtools/source/tinderbox/ep_unittest.pl#28

Comment 7

[Benjamin Smedberg]

Attachment: Add PROCESS-CRASH and stack information to the log, rev. 1

Comment 8

[Benjamin Smedberg]

Checking in ep_unittest.pl;
/cvsroot/mozilla/webtools/tinderbox/ep_unittest.pl,v  <--  ep_unittest.pl
new revision: 1.6; previous revision: 1.5
done