1121939 - Introduce a testing-modules resource to register and use shared code modules

Status: RESOLVED WORKSFORME

(Testing :: General, defect)

Description

[Mike de Boer [:mikedeboer]]

Right now we share code for unit tests using the 'head.js' code inclusion mechanism. There are several downsides to using this approach, of which I'll three important ones:

1. shared code to perform common tasks in unit tests can not be shared outside the test directory where each head.js script is placed.
2. Each included head.js exposes many additional globals, which is considered an anti-pattern.
3. Downside no. 1 leads to code duplication in various areas where unit tests require similar boilerplate to setup and/ or tear down a test.

A solution I propose here is:

1. Introduce a 'testing-modules' namespace which allows loading shared code as `Cu.import("resource://testing-modules/HTTPServerMock.jsm")`, similar to 'testing-common' in use by XPCShell and Mochitest-browser, but available to _all_ test frameworks.
2. Register new code modules by adding a directive to moz.build files; something like:

TEST_MODULES += [
  'test/modules/HTTPServerMock.jsm'
]

... which will 1) break the build if a module name is already defined elsewhere, hinting developers that they're likely to be duplicating code and 2) enables a central register for modules that mach, for example, may use to provide a list with optional documentation.

The scope of this work might be too big for this single bug, in which case we can track the work here.

Gregory, what do you think of this idea?

Comment 1

[:Paolo Amadini]

It looks you're precisely describing resource://testing-common/ and TESTING_JS_MODULES in moz.build?

https://developer.mozilla.org/en-US/docs/Mozilla/QA/Developing_tests#JavaScript_test-only_modules

For example:

http://mxr.mozilla.org/mozilla-central/source/testing/modules/moz.build

> ... which will 1) break the build if a module name is already defined
> elsewhere, hinting developers that they're likely to be duplicating code and
> 2) enables a central register for modules that mach, for example, may use to
> provide a list with optional documentation.

However I'm not sure if the moz.build directives have these properties.

Comment 2

[Gregory Szorc [:gps]]

I implemented TESTING_JS_MODULES way back in 2012 in bug 748490. It does everything you should need. Although there may be a few test harnesses that don't load the resource handler. But those are patches waiting to be written. moz.build should complain if the same destination path is defined multiple times. If it doesn't, file a Core :: Build Config bug.

Comment 3

[Mike de Boer [:mikedeboer]]

(In reply to Gregory Szorc [:gps] from comment #2)
> I implemented TESTING_JS_MODULES way back in 2012 in bug 748490. It does
> everything you should need. Although there may be a few test harnesses that
> don't load the resource handler. But those are patches waiting to be
> written. moz.build should complain if the same destination path is defined
> multiple times. If it doesn't, file a Core :: Build Config bug.

Awesome! So why don't I see this used excessively, I wonder... Should I being doing some investigation about this and post something to dev-platform/ fx-team with my findings?

Comment 4

[Gregory Szorc [:gps]]

I imagine some people don't know about it. I've tried to encourage people to write testing JSMs instead of adding stuff in head.js as part of review. It might be worth reminding people, especially reviewers.

If I had my way, we'd kill head.js support and provide a better mechanism within the test harness for programmatically declaring functions that should be called before and after test suite and individual test execution. Weren't you working on such an API for declaring and executing JS tests?