Closed Bug 717018 Opened 9 years ago Closed 9 years ago
merge robotium logs into a single logfile
robotium is run one test at a time and reports results for a given test; we need to make robotium run all tests at once or find a way to merge the log files into a single unified log to work with tbpl
Assignee: nobody → jmaher
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows Vista → Android
Hardware: x86 → ARM
Version: unspecified → Trunk
This is an interesting scenario. Right now we have to wait for the test to complete (bug 717725), which means there is no harness level wait to check for a timeout. If we could timeout and prove that we could run 10 test files in a single 'am instrument' command, it would be nice to not run each test one at a time. We have done a lot of fixes to make robotium more reliable, so it might be possible to do this. If we choose to run one test file at a time, then we need to parse all the logs. I am not sure the best method for this. There is some code in maemkit (http://hg.mozilla.org/qa/maemkit) to do this, and we could use that. I would say the log merging should be a temporary fix.
We will want to do this in the short term once we get robocop running on tbpl. This way we can report all the results.
this will be a python harness based log parser that will be able to return the proper return values so we can be orange/red/green.
basic patch to create a single unified mochitest style log at the end of a test run while handling the return value/exit codes from mochitest.
Comment on attachment 595289 [details] [diff] [review] stitch logs together and return proper error code (1.0) + with open(self.localLog) as currentLog: This is a 2.5ism. Do we need to code to 2.4 here? probably better doing lines = currentLog.readlines() vs the data.split() + if reend.match(line): + end_found = True Should just return at this point It would be nice to use logparser. I guess this is not possible? Or worth considering?
Attachment #595289 - Flags: review?(jhammel) → review+
Comment on attachment 595289 [details] [diff] [review] stitch logs together and return proper error code (1.0) Review of attachment 595289 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/mochitest/runtestsremote.py @@ -349,0 +351,26 @@ > > + def addLogData(self): > > + with open(self.localLog) as currentLog: > > + data = currentLog.read() > > + NaN more ... I'll just mention that for me personally, I would like to see a big bold visual separation before the summary: logFile.append("==================== TEST SUMMARY ====================") @@ -425,3 +482,4 @@ > > > > try: > > retVal = mochitest.runTests(options) > > + logData = mochitest.addLogData() addLogData() does not return anything. Did you want to accumulate the log data in a return value and then pass that to printLog? That might be neater...but the member variable works fine.
Attachment #595289 - Flags: review?(gbrown) → review+
(In reply to Geoff Brown [:gbrown] from comment #6) > NaN more ... Oh bugzilla, why do you do that? The intended context there was: + def printLog(self): + passed = 0 + failed = 0 + todo = 0 + incr = 1 + logFile =  + logFile.append("0 INFO SimpleTest START")
added a few minor comments and edits before push. I have to make this match the log file parsers that we have so our systems can properly annotate and turn green/orange/red. inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/7be822540243
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla13
You need to log in before you can comment on or make changes to this bug.