Closed Bug 601216 Opened 11 years ago Closed 10 years ago
Create buildbot logparser for generating data for waronorange hbase system
I've implemented a log parser for the buildbot logs produced by our test runs. It scans these logs and produces a line of output in JSON format, which contains the kinds of data we think we could be useful in analyzing random oranges and other factors. The exact format of the output is described at http://hg.mozilla.org/automation/logparser/file/d7dfabcc25b8/logparser/logparser.py#l549 and http://hg.mozilla.org/automation/logparser/file/d7dfabcc25b8/logparser/logparser.py#l139 Examples are at http://hg.mozilla.org/automation/logparser/file/tip/logparser/tests Ideally, we'd want some additional data that isn't in the raw buildbot logs, but is in the processed tinderbox logs. This info includes OS version and starttime. If we're going to use this with processed tinderbox logs instead of raw buildbot logs, I'll have to update the script to get that additional info; otherwise we'll have to figure out an alternate mechanism to get this data. As far as I've been able to determine, the life cycle of a log is this: 1 - test machine produces raw buildbot log, and uploads it to FTP server 2 - test machine e-mails raw buildbot log to tinderbox, along with some extra data that isn't in the raw log; this extra data looks like this: tinderbox: tree: Firefox tinderbox: builddate: 1285956707 tinderbox: status: testfailed tinderbox: build: Rev3 Fedora 12 mozilla-central debug test mochitest-other tinderbox: errorparser: unittest tinderbox: binaryurl: tinderbox: logcompression: gzip tinderbox: logencoding: base64 tinderbox: END 3 - tinderbox tracks comments for failed test cases in a separate file, e.g. http://tinderbox.mozilla.org/Firefox/notes.txt 4 - tinderbox creates status pages based on 2) and 3) It isn't clear to me at what point in this cycle this logparser will operate; I'm currently assuming it's on the raw buildbot logs from step 1).
I neglected to mention that the parser lives at http://hg.mozilla.org/automation/logparser/
This has been functioning for quite a while now.
Assignee: jgriffin → nobody
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Component: General → Orange Factor
QA Contact: general → orangefactor
Resolution: --- → FIXED
Product: Testing → Tree Management
You need to log in before you can comment on or make changes to this bug.