407990 - add links to documentation about running tests to Tinderbox

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Dan Mosedale (:dmosedale, :dmose)]

Last week, a few of us were trying to figure out ways to lessen the amount of tinderbox test breakage that goes on.  One idea we came up with was to make it easy for people to find documentation on how to run a given set of tests so that they could (as appropriate) do it before checking in.  The initial thought was to come up with a wiki page documenting how to do that for the various test suites.

I just spent a bunch of time poking around the various test documentation on the wiki, and it seems to me that that quite a bit of it already exists, but on different pages.  Rather than copying that around and setting ourselves up for non-single-sourced data problems, I think a better strategy would be to alter the Tinderbox output slightly so that it links directly to documentation for the tests that are being run.

Specifically, we could link the name of the test on the waterfall page to its "how to run me" documentation.  For example, right now, we have (effectively):

TUnit 428/0

in an example test unit test run.  I propose changing this to

<a href="http://developer.mozilla.org/en/docs/Writing_xpcshell-based_unit_tests#Running_unit_tests">TUnit</a> 428/0

This seems like a good idea for tests that are straightforward to run locally.  For performance stuff, and other things that are harder to reproduce locally, we might just want to link to the overview page for that test, rather than the "how to run this" page.

Thoughts?

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Looks like a simple mod to lines like this:
http://mxr.mozilla.org/seamonkey/source/tools/buildbot-configs/testing/unittest/mozbuild.py#230

Comment 2

[[e-mail] (:bhearsum)]

Will take patches for this, but not working on it.

Comment 3

[[e-mail] (:bhearsum)]

We've got simple Makefile targets for all testsuites at this point, make {check,xpcshell-tests,reftest,crashtest} and mochitest-{plain,chrome,browser-chrome,a11y}. I'd bet there isn't much of a need for this.

Do feel free to re-open if you disagree, but I'm closing this for now.

Comment 4

[Chris Cooper [:coop] (he/him)]

Moving closed Future bugs into Release Engineering in preparation for removing the Future component.