786848 - Need CI script for B2G xpcshell tests

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Jonathan Griffin (:jgriffin)]

This script would parse the output of an xpcshell run on B2G, and upload the results to autolog.  

A similar script that handles this for B2G mochitests is here:  https://github.com/jonallengriffin/b2gautomation/blob/master/b2gautomation/mochilog.py

Comment 1

[Mihnea Dobrescu-Balaur (:mihneadb)]

Attachment: post_to_autolog functionality in runxpcshelltests.py

I kept the hardcoded parts as they are in Marionette's post_to_autolog.

Also there are some whitespace fixes.

Comment 2

[Jonathan Griffin (:jgriffin)]

Pushed to try to make sure it doesn't break existing xpcshell runs in buildbot:  https://tbpl.mozilla.org/?tree=Try&rev=5183c8314ad2

Comment 3

[Jonathan Griffin (:jgriffin)]

Comment on attachment 657499 [details] [diff] [review]
post_to_autolog functionality in runxpcshelltests.py

Review of attachment 657499 [details] [diff] [review]:
-----------------------------------------------------------------

In general this looks good.  But there are a couple of things that seem awkward:

- it seems awkward to force code to supply actual and expected results when it has what moztest thinks of as a calculated result.  I think it makes sense to allow moztest to accept a calculated result, in which case it can infer the actual and expected results.
- it seems strange to fake the start and end times.  Many of our test harnesses (for good or bad) just track the test duration, not explicit start and end times.  I think moztest should optionally accept a test duration, rather than forcing it to be a calculation.

Comment 4

[Mihnea Dobrescu-Balaur (:mihneadb)]

Attachment: modify according to feedback

This assumes the changes in #788828 have landed.

Comment 5

[Jonathan Griffin (:jgriffin)]

Comment on attachment 658667 [details] [diff] [review]
modify according to feedback

Review of attachment 658667 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, thanks. I'll land after bug 788842.

Comment 6

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/1485815e2524

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/1485815e2524