1023878 - make test-perf should exit with a non-zero exit code if there are failures

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Performance, defect, P1)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The jobs currently running on Jenkins are passing, despite there being failures such as:

npm ERR! Error: failed to fetch from registry: mime
npm ERR!     at /usr/share/npm/lib/utils/npm-registry-client/get.js:139:12
npm ERR!     at cb (/usr/share/npm/lib/utils/npm-registry-client/request.js:31:9)
npm ERR!     at Request._callback (/usr/share/npm/lib/utils/npm-registry-client/request.js:136:18)
npm ERR!     at Request.callback (/usr/lib/nodejs/request/main.js:119:22)
npm ERR!     at Request.<anonymous> (/usr/lib/nodejs/request/main.js:212:58)
npm ERR!     at Request.emit (events.js:88:20)
npm ERR!     at ClientRequest.<anonymous> (/usr/lib/nodejs/request/main.js:412:12)
npm ERR!     at ClientRequest.emit (events.js:67:17)
npm ERR!     at HTTPParser.onIncoming (http.js:1261:11)
npm ERR!     at HTTPParser.onHeadersComplete (http.js:102:31)

I also noticed that if npm is not installed we still get a success:

/bin/bash: npm: command not found

In the case of such a failure we should exit immediately with a non-zero exit code so that Jenkins will fail the build.

Comment 1

[Hubert Figuiere [:hub]]

it is not make test-perf per see, it is a common target for the other tests too.

Also

http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Perf/job/b2g.hamachi.mozilla-central.master.mozperftest/3387/consoleText

Pinging jlal.

Comment 2

[Hubert Figuiere [:hub]]

Also, by default Make should stop the rule if a non-zero return code occur. But this ain't happening.

Comment 3

[Hubert Figuiere [:hub]]

ok, so I get what's happening here: gaia-perf-marionette calls make to regenerate node_modules and doesn't check the error code.

Nonetheless, the error is caused by James last change.

Comment 4

[Hubert Figuiere [:hub]]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

Comment 5

[Julien Wajsberg [:julienw]]

Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

I gave 3 simpler alternatives, please pick one.

Comment 6

[Hubert Figuiere [:hub]]

Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

I picked one. Thanks !

Comment 7

[Julien Wajsberg [:julienw]]

Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

r=me
yeah :)

Comment 8

[Hubert Figuiere [:hub]]

https://github.com/mozilla-b2g/gaia/commit/41db6954a67efc55016744bc8f6591ae9e07a285