595866 - Investigate making xpcshell junit compatible

Status: RESOLVED WONTFIX

(Testing :: XPCShell Harness, defect)

Description

[cmtalbert]

I found this gem of effort from the qa newsgroup.

All,

FYI I've written up a howto for xpcshell based tests with junit output (to integrate with Hudson) at http://www.sirgroane.net/2010/09/automated-javascript-unit-tests-with-xpcshell-and-hudson

It took a major update to runxpcshelltests.py and head.js to produce different output types (I implemented TAP as well) - I'd really like an experienced mozilla dev to review it and include in core mozilla.

This effectively implements a wishlist item from http://wiki.mozilla.org/SoftwareTesting

Regards,

Ian 

== What's Next ==
We should take a look and investigate how this integration is achieved, if this is something we want to take and merge into the existing xpcshell test harness, and what (if anything) this new functionality will break of the core set of tests we have now.

Comment 1

[Ian Rogers]

I also implemented the current output format (as "olde", though that's a little perjorative so I don't mind if you rename it "mozilla" or some such). If I got it right then no core functionality should break - but obviously it needs testing. 

The implementation is very simple, it was just a refactoring exercise really.

Comment 2

[(not currently active) Ted Mielczarek]

I don't really know anything about junit. I suspect that SoftwareTesting wishlist was written quite a while ago, I'm not sure if this is something that's actually useful to Mozilla.

The current output format, while totally arbitrary, is useful to us because we've made it (mostly) consistent across all of our unittest harnesses.

Comment 3

[Ian Rogers]

The current output is fine for testing solely within the Mozilla system.  But my work was prompted by the need to integrate testing of a Thunderbird plugin with the rest of my site's development (ie. with Hudson in my case)...

Other people will have this problem - are there currently *any* tools to help plugin developers do regression testing? - and using "public" standards for test output formats wins you/us the ability to integrate with other (massively used and developed) tools. There's no plugin to integrate MATOF "Mozilla arbitrary test output format" with Hudson or any other tool as far as I know...

Simply taking the code I've written, with no other work, and including it in the core will help anyone wanting to integrate with external (to Mozilla) test tools.

In the long run perhaps it can be seen there's a one-to-one mapping between MATOF and TAP.

Comment 4

[Geoff Brown [:gbrown]]

Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.