644792 - test run summary print statements should be real logging messages

Status: RESOLVED FIXED

(Testing Graveyard :: Mozmill, defect)

Description

[Heather Arthur [:harth]]

Right now at the end of a test run we print() these:

INFO Passed: 27
INFO Failed: 1
INFO Skipped: 0


This syntax looks much like the syntax for our logging.INFO statments, but not quite! They should be turned into real logging statements to take advantage of our logging system, like being parseable and logging to both file and console.

Then they would look like:
INFO | Passed: 27
INFO | Failed: 1
INFO | Skipped: 0

Comment 1

[Heather Arthur [:harth]]

Attachment: turn summary print statements into logging.info() s and set default console logging level to INFO

This patch turns the print statements into logging.info()s and also set the default logging level to INFO so that they still show up by default after a test run.

Comment 2

[Jeff Hammel]

Looks good, but I'm not sure if we want it to look different than what the other harnesses do (i.e. the pipes versus the spaces).  I'm fine with this, but it isn't what e.g. mochitest does:

20439 INFO Passed: 19943
20440 INFO Failed: 2
20441 INFO Todo:   43

(from: http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1300997663.1300998308.16563.gz&fulltext=1) 

buildbotcustom would have to be changed.  So....I don't know.  These should be logged, but this is again where we run up against something being hard to do with python logging :( 

:ctalbert? ideas?

Comment 3

[Jeff Hammel]

Comment on attachment 521662 [details] [diff] [review]
turn summary print statements into logging.info() s and set default console logging level to INFO

Giving this to you, :ctalbert, as I'm not sure what to do with it.  We want to log this, but python logging makes this difficult.  Not sure where to go from here :/

Comment 4

[cmtalbert]

Comment on attachment 521662 [details] [diff] [review]
turn summary print statements into logging.info() s and set default console logging level to INFO

This looks good. From our quick review of the logging styles we should go with the "INFO | <foo>" style.  This conforms to reftest, xpcshell, and browser-chrome (note that browser chrome prepends "test", "TEST-INFO | <foo>"), but reftest and xpcshell do "INFO | <foo>" style.

Mochitest is the only outlier that uses "INFO <foo>".

Comment 5

[Heather Arthur [:harth]]

master:
https://github.com/mozautomation/mozmill/commit/79a4ca7f9bb5c4b9737fc491ebe750809552aa64