924958 - Provide a KumaScript macro test process

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: KumaScript, defect)

Description

[Trevor]

When editing a template it would be good to have a method of testing the changes made against the actual uses on the site as well as some hand crafted test cases.

It would be helpful to have the ability to:
* see the rendering of two different revisions of the template side by side
* change the language used
* specify various env.x values
* run the test cases against a different template and compare side by side the results of the two templates, which would be useful when testing changes in a separate template

Comment 1

[Jean-Yves Perrier [:teoli]]

Moving this to Kumascript as I believe it is more appropriate. Feel free to move it elsewhere if I was mistaken.

Comment 2

[John Whitlock [:jwhitlock]]

We moved KumaScript templates to GitHub in late 2016:

https://github.com/mozilla/kumascript/tree/master/macros

One strategy is an integration test framework that:

1) simulates Kuma document rendering requests
2) expects and simulated API responses, and 
3) verifies the rendered HTML

These tests could be run during template development and automatically as regression testing.

Comment 3

[[github robot]]

Commit pushed to jenkins-macro-test-framework-924958 at https://github.com/mozilla/kumascript

https://github.com/mozilla/kumascript/commit/0145585a2402d475fb2589840a56d56fa6ef339c
bug 924958: use mocha as new kumascript test framework

* add "mocha" and related packages to "package.json"
  and create new "npm-shrinkwrap.json"
* update "README.md" to reflect "mocha" rather than
  "nodeunit"
* remove unused "HTTPLoader" from "loaders.js"
* update and refactor all tests
* add new functions to "test-utils.js"
* remove unused code from "test-utils.js"
* update Makefile and Jenkinsfile

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kumascript

https://github.com/mozilla/kumascript/commit/0145585a2402d475fb2589840a56d56fa6ef339c
bug 924958: use mocha as new kumascript test framework

https://github.com/mozilla/kumascript/commit/200a75b9405b937d39afc8fe60d6815b45a4cf46
Merge pull request #188 from escattone/macro-test-framework-924958

bug 924958: use mocha as new kumascript test framework

r=jwhitlock,Elchi3

Comment 5

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kumascript

https://github.com/mozilla/kumascript/commit/6cbcbf7577ba860d6416ba762f18d75e85afc69e
bug 924958: first-pass at test framework for macros

* tests/macros/utils.js defines a set of functions
  that comprise a mocha-based test framework for
  simplifying the testing of Kumascript macros
* fixes the EJSTemplate's "execute" method so
  that it no longer re-calls its callback if the
  callback itself throws an error
* adds a "test-macros" command to the Makefile
* adds an explicit timeout, which defaults to 10s,
  to the test commands (fixes an issue where some
  some tests would occassionly exceed the default
  mocha timeout of 2s)
* exercises the new test framework with tests (in
  the tests/macros/ directory) for five simple
  macros that are frequently used
* adds "chai-as-promised" npm package to "package.json"
* add "make test-macros" to the "Jenkinsfile" and
  ".travis.yml" files.

https://github.com/mozilla/kumascript/commit/558e54deb7e11a9a74fef59d6059ed0ac98203ee
Merge pull request #204 from escattone/test-framework-for-macros-924958

bug 924958: first-pass at test framework for macros

Comment 6

[Florian Scholz (Open Web Docs)]

We can now test macros :)