576072 - automation.py should return non-zero when the test run crashes

Status: RESOLVED INVALID

(Testing :: General, defect)

Description

[Armen [:armenzg]]

Attachment: [WIP] If crash is detected then let's change the returning exit code

Currentyl the windows 7 reftests-d2d tests are going GREEN because the main application crashes before any test fails and therefore the build doesn't go ORANGE or RED.

I have generated a patch that I still have to test but please have a look at it. It is only 3 lines.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1277892360.1277893657.8631.gz&fulltext=1

> PROCESS-CRASH | Main app process exited normally | application crashed (minidump found)
> 
> 0x77ae0000 - 0x77af8fff  sechost.dll  6.1.7600.16385
> 0x77b00000 - 0x77babfff  msvcrt.dll  7.0.7600.16385
> WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!
>
> REFTEST INFO | runreftest.py | Running tests: end.
> program finished with exit code 0
> elapsedTime=833.131000
> TinderboxPrint: reftest<br/>4445/0/236

Comment 1

[Armen [:armenzg]]

This is important to fix specially now that I am trying to clear up the oranges on the Windows 7 slaves.

Comment 2

[(not currently active) Ted Mielczarek]

This is not an app crash, these are crashes in plugin processes, and they're all expected crashes. If you look at the top of the stack, every one of them is in nptest.dll!IntentionalCrash. This should not cause any orange.

Comment 3

[Jesse Ruderman]

Right, see bug 539823.