813779 - Make Marionette's xunit output compatible with Jenkins

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Jonathan Griffin (:jgriffin)]

An example file which does work with jenkins is:

http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com.trunk/2912/artifact/results.xml

Mozmill code which generates this is:

https://github.com/whimboo/mozmill-automation/blob/master/mozmill_automation/reports.py#L98

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: Make Marionette's xunit output compatible with Jenkins,

This makes output consistent with http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com.trunk/2912/artifact/results.xml

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 684092 [details] [diff] [review]
Make Marionette's xunit output compatible with Jenkins,

Review of attachment 684092 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/marionette/client/marionette/runtests.py
@@ +492,4 @@
>  
> +            if result in ['failure', 'error', 'skipped']:
> +                f = doc.createElement(result)
> +                f.setAttribute('message', 'test %s' % result)

Is it possible to include the elapsed time for the individual tests? If not, can we file a followup bug to add this?

@@ +499,5 @@
>          doc = dom.Document()
>  
> +        testsuite = doc.createElement('testsuite')
> +        testsuite.setAttribute('name', 'Marionette')
> +        testsuite.setAttribute('time', str(self.elapsedtime))

Elapsed time needs to be in milliseconds. The current format is: 0:01:14.989096

Comment 3

[Jonathan Griffin (:jgriffin)]

Attachment: Make Marionette's xunit output compatible with Jenkins,

This puts the time in ms.  Let's file a separate bug for adding times of all the individual tests; that's a bit more work.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 685375 [details] [diff] [review]
Make Marionette's xunit output compatible with Jenkins,

Review of attachment 685375 [details] [diff] [review]:
-----------------------------------------------------------------

Looks great, thanks!

Comment 5

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/73fdd2c4f8f0

Comment 6

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/73fdd2c4f8f0

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/5a057c06e172
https://hg.mozilla.org/releases/mozilla-beta/rev/c9d0b3dabf0a