1032218 - integration test for the build system have unhelpful output

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Alexandre Poirot [:ochameau]]

See bug 970658 for detailed story, but the issue is that when an assertion fails in these tests, you only get:
14:59:43     INFO -  [0m  1) Build Integration tests make with DEBUG=1:
14:59:43     INFO -  [0m[31m     Uncaught expected false to be truthy[0m[90m

Ok, you know which test function failed, here, "make with DEBUG=1", but then you get "Uncaught expected false to be truthy", there you have no idea which assertion we are talking about. We should have a stack trace! Passing --verbose to mocha doesn't help.

But that's not all, let me raise all the pain with these tests:
* I haven't found any way to run one particular test, and it is extremelly painful as each test for the build system takes aaaaggggeeeeeee. I just ended up hacking build/common.mk, but I wish it was easier and affordable to anyone.

* For some reason, on tbpl stdout logs, you don't see the assertion failure message (i.e. "Uncaught expected false to be truthy" in my example). You can only see the assertion message at the test summary at the very end of tbpl logs. It is important to not that in brief view, tbpl doesn't link to the test summary but to the test failure.
See: https://tbpl.mozilla.org/php/getParsedLog.php?id=42173808&tree=Gaia-Try#error0

* On top of that, the nice colors you have when running locally pollute tbpl output with all these color characters being inserted... It ends up hiding the test summary with cryptic charactes:
14:59:43     INFO -  [92m [0m[32m 21 passing[0m[90m (19m)[0m
14:59:43     INFO -  [31m  1 failing[0m
14:59:43     INFO -  [0m  1) Build Integration tests make with DEBUG=1:
14:59:43     INFO -  [0m[31m     Uncaught expected false to be truthy[0m[90m
14:59:43     INFO -  [0m
* The color are great when you run locally
These tests are full of pain:
First pain is that, ok, we get nice colors when running locally

Comment 1

[BMO Automation]

Firefox OS is not being worked on