Sisyphus - Crash Automation - improve log performance handling

RESOLVED FIXED

Status

RESOLVED FIXED
8 years ago
8 years ago

People

(Reporter: bc, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

8 years ago
Created attachment 452064 [details] [diff] [review]
patch

Currently there are several areas where log performance processing suffers.

The regular expressions are not anchored to the start of the line and the code does not "short circuit" the loop execution when a match is found. These problems are insignificant in comparison to the performance hit when posting an external attachment to couchdb using an inline buffer rather than using the curl based external attachment saver. Large log files (>10M) can take hours to upload using couchdb.py:saveAttachment rather than seconds when using couchdb.py:saveFileAttachment.

This patch fixes the regexp processing to use BOL anchors and to short circuit the loop when matches are detected.

It also changes crashworker.py and unittestworker.py to use saveFileAttachment.

A SIGUSR1 handler is added to worker to allow clean termination upon receipt of a SIGUSR1 signal. This is useful when running under a profiler such as cProfile.py where shutting down cleanly is necessary in order to obtain profiler output.
Attachment #452064 - Flags: review?(ctalbert)

Updated

8 years ago
Attachment #452064 - Flags: review?(ctalbert) → review+
(Reporter)

Comment 1

8 years ago
http://hg.mozilla.org/automation/sisyphus/rev/33e56bd4fbe4
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.