572847 - Sisyphus - Crash Automation - improve log performance handling

Status: RESOLVED FIXED

(Testing Graveyard :: Sisyphus, defect)

Description

[Bob Clary [:bc] (inactive)]

Attachment: patch

Currently there are several areas where log performance processing suffers.

The regular expressions are not anchored to the start of the line and the code does not "short circuit" the loop execution when a match is found. These problems are insignificant in comparison to the performance hit when posting an external attachment to couchdb using an inline buffer rather than using the curl based external attachment saver. Large log files (>10M) can take hours to upload using couchdb.py:saveAttachment rather than seconds when using couchdb.py:saveFileAttachment.

This patch fixes the regexp processing to use BOL anchors and to short circuit the loop when matches are detected.

It also changes crashworker.py and unittestworker.py to use saveFileAttachment.

A SIGUSR1 handler is added to worker to allow clean termination upon receipt of a SIGUSR1 signal. This is useful when running under a profiler such as cProfile.py where shutting down cleanly is necessary in order to obtain profiler output.

Comment 1

[Bob Clary [:bc] (inactive)]

http://hg.mozilla.org/automation/sisyphus/rev/33e56bd4fbe4