782841 - Gaia unit test errors on CI

Status: RESOLVED INCOMPLETE

(Firefox OS Graveyard :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

When run on the CI, the Gaia unit tests usually fall over (but not always).  Sometimes, they simply stall and never end.  At other times, one of these errors appear:

Loading config file '/data/jenkins/workspace/gaia-unit-test/tools/test-agent/test-agent-server.js'
Running tests

./tools/test-agent/node_modules/test-agent/bin/js-test-agent
*** UTM:SVC TimerManager:notify - notified @mozilla.org/browser/search-service;1

/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/concurrent-reporting-events.js:213
      this.envQueue[envId].push(arguments);
                           ^
TypeError: Cannot call method 'push' of undefined
    at Object.emit (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/concurrent-reporting-events.js:213:28)
    at Object.onTest (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/runner-stream-proxy.js:99:16)
    at /data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:156:20
    at Array.forEach (native)
    at Object.emit (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:155:19)
    at Object.respond (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:81:17)
    at Object.respond (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/reporter.js:138:23)
    at Object._onTestData (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/common/mocha-test-events.js:95:21)
    at /data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:156:20
    at Array.forEach (native)
GAIA: TESTS FAIL

- or -

Loading config file '/data/jenkins/workspace/gaia-unit-test/tools/test-agent/test-agent-server.js'
Running tests

./tools/test-agent/node_modules/test-agent/bin/js-test-agent

/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/reporter.js:138
    return this.proxy.respond([data.event, data.data]);
                      ^
TypeError: Cannot call method 'respond' of undefined
    at Object.respond (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/mocha/reporter.js:138:23)
    at Object._onTestData (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/common/mocha-test-events.js:95:21)
    at /data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:156:20
    at Array.forEach (native)
    at Object.emit (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:155:19)
    at Object.respond (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:81:17)
    at Object._processMessage (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/websocket-client.js:145:10)
    at /data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:156:20
    at Array.forEach (native)
    at Object.emit (/data/jenkins/workspace/gaia-unit-test/tools/test-agent/node_modules/test-agent/lib/test-agent/responder.js:155:19)
GAIA: TESTS FAIL

Comment 1

[James Lal [:lightsofapollo]]

These are definitely test agent (https://github.com/lightsofapollo/js-test-agent) related bugs. These errors are known to happen with multiple b2g instances running (with test-agent open) and multiple node processes listening to 8789. 

My best guess is there is some race condition in the code or startup (b2g-desktop.sh) scripts. I will track it down and fix it upstream and use this bug as a tracking item.

Comment 2

[James Lal [:lightsofapollo]]

We are not using jenkins (and have much better solutions) and also test-agent fixes.