697405 - cfx testall should tell if all tests pass or not

Status: RESOLVED DUPLICATE of bug 1093385

(Add-on SDK Graveyard :: General, defect, P3)

Description

[Irakli Gozalishvili [:irakli] [:gozala] [@gozala]]

cfx testall should tell how many tests failed in total in all packages it was running, otherwise it hard to track what failed and where.

Comment 1

[Alexandre Poirot [:ochameau]]

Before no runtime search, we used to have at the end of cfx testall -v execution:
4900 of 4906 tests passed.

The following tests failed:
  test-ipc.testOne: timed out
  test-low-level-ipc.testIPC: timed out
  test-content-proxy.testXPathResult: timed out
  test-sandbox.testLoadSubscript: timed out
  test-l10n.testExactMatching: exception
  test-l10n.testPartialMatching: exception

FAIL
Total time: 334.473000 seconds
Program terminated unsuccessfully.
Some tests were unsuccessful.

Now, with work from https://github.com/mozilla/addon-sdk/pull/249
each package is executed independently, so that we don't get the result of addon-kit+api-utils. 
The problem is that even before this change, we didn't had any idea of example or testcfx failures.
So I think landing pull request 249 is fine, we can postpone such improvement.

At the end of cfx testall, we should know, at least, which package or test failed
Then it would be really cool, with -v option, to have the list of all failing unit tests.

Comment 2

[Erik Vold [:erikvold] (please needinfo? me)]

Irakli can we wontfix this one?

Comment 3

[Erik Vold [:erikvold] (please needinfo? me)]

running `npm test` on the addon-sdk repo does this now.

Comment 4

[Erik Vold [:erikvold] (please needinfo? me)]

removing this ni