Improve logging for mochitest-remote and mochitest-robocop

RESOLVED FIXED in mozilla26

Status

Testing
Mochitest
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

unspecified
mozilla26
x86
Mac OS X
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

5 years ago
There are a couple of logging issues bugging me. Best to explain with patches...
(Assignee)

Comment 1

5 years ago
Created attachment 794128 [details] [diff] [review]
(1) tidy xpcshell args logging

Before:

args: [u'/home/mozdev/objdirs/firefox/dist/bin/xpcshell', '-g', '/home/mozdev/objdirs/firefox/dist/bin', '-v', '170', '-f', '/home/mozdev/objdirs/x86/_tests/testing/mochitest/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmp5nVpcc';const _SERVER_PORT = '8888'; const _SERVER_ADDR = '192.168.0.50';\n                     const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', './server.js']

After:

INFO | automation.py | Launching: [u'/builds/tegra-160/test/build/hostutils/bin/xpcshell', '-g', '/builds/tegra-160/test/build/hostutils/xre', '-v', '170', '-f', '/builds/tegra-160/test/build/hostutils/bin/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpnHocm7'; const _SERVER_PORT = '30160'; const _SERVER_ADDR = '10.250.49.164'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', './server.js']
Attachment #794128 - Flags: review?(jmaher)
(Assignee)

Comment 2

5 years ago
Created attachment 794138 [details] [diff] [review]
(2) use mozlog in runtestsremote.py

Use mozlog calls instead of "print" in mochitest/runtestsremote.py.

My main motivation for this is that I have been seeing some long lines split recently -- especially SCREENSHOT lines. I have not tracked down the cause, but I think it best if we use one logging interface consistently. I have not seen any split lines with this patch.

Try run, with intentional test failures, at https://tbpl.mozilla.org/?tree=Try&rev=4dc8fddeecf3
Attachment #794138 - Flags: review?(jmaher)
(Assignee)

Comment 3

5 years ago
Comment on attachment 794138 [details] [diff] [review]
(2) use mozlog in runtestsremote.py

Oops - I noticed a problem in the try run.
Attachment #794138 - Flags: review?(jmaher)
Comment on attachment 794128 [details] [diff] [review]
(1) tidy xpcshell args logging

Review of attachment 794128 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/automation.py.in
@@ +217,5 @@
>                   universal_newlines=False,
>                   startupinfo=None,
>                   creationflags=0):
>        args = automationutils.wrapCommand(args)
> +      _log.info("INFO | automation.py | Launching: %s", args)

I would like to make this easier to copy/paste for running, right now args is a python list.  We could either add another command or adjust this one:
_log.info("INFO | automation.py | Launching: %s", subprocess.list2cmdline(args))
Attachment #794128 - Flags: review?(jmaher) → review+
(Assignee)

Comment 5

5 years ago
(1) landed with list2cmdline change:

https://hg.mozilla.org/integration/mozilla-inbound/rev/c6a15f08e51e

I still aspire to land patch (2)...
Whiteboard: [leave open]
(Assignee)

Comment 7

4 years ago
Created attachment 798871 [details] [diff] [review]
(2) use mozlog in runtestsremote.py

Functionally, the main change here is to use mozlog instead of print for dumping screenshots. I do not have conclusive proof that this fixes the intermittent problem with split screenshots in test logs, but I have not seen the problem in any of my try runs.

Other print->mozlog changes make the output a little tidier and more consistent.

A few print's remain, particularly for the test summary. Changing these to mozlog changes the order of output in the final logs, which seems to confuse the tbpl summary.

Here's a try run that includes an intentional failure in a robocop test, to show both success and failure logging: https://tbpl.mozilla.org/?tree=Try&rev=2ca6212d6a1c
Attachment #794138 - Attachment is obsolete: true
Attachment #798871 - Flags: review?(jmaher)
Comment on attachment 798871 [details] [diff] [review]
(2) use mozlog in runtestsremote.py

Review of attachment 798871 [details] [diff] [review]:
-----------------------------------------------------------------

nice!

::: testing/mochitest/runtestsremote.py
@@ +479,5 @@
>      def printDeviceInfo(self, printLogcat=False):
>          try:
>              if printLogcat:
>                  logcat = self._dm.getLogcat(filterOutRegexps=fennecLogcatFilters)
> +                log.info('\n'+(''.join(logcat)))

I assume this was tested, it just looks odd.  Why the () around the .join ?
Attachment #798871 - Flags: review?(jmaher) → review+
(Assignee)

Comment 9

4 years ago
(In reply to Joel Maher (:jmaher) from comment #8)
> > +                log.info('\n'+(''.join(logcat)))
> 
> I assume this was tested, it just looks odd.  Why the () around the .join ?

I thought it looked odd without the ():

'\n'+''.join(logcat)

I wanted to make clear that we're logging a newline + (the joined logcat), distinct from '\n'.join(logcat), which might be expected.
https://hg.mozilla.org/mozilla-central/rev/d66d26298e85
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
You need to log in before you can comment on or make changes to this bug.