Closed Bug 1023878 Opened 7 years ago Closed 7 years ago

make test-perf should exit with a non-zero exit code if there are failures

Categories

(Firefox OS Graveyard :: Performance, defect, P1)

ARM
Gonk (Firefox OS)
defect

Tracking

(b2g-v1.3T affected, b2g-v1.4 wontfix, b2g-v2.0 affected, b2g-v2.1 fixed)

RESOLVED FIXED
2.0 S4 (20june)
Tracking Status
b2g-v1.3T --- affected
b2g-v1.4 --- wontfix
b2g-v2.0 --- affected
b2g-v2.1 --- fixed

People

(Reporter: davehunt, Assigned: hub)

Details

(Whiteboard: [c=automation p=1 s= u=])

Attachments

(1 file)

The jobs currently running on Jenkins are passing, despite there being failures such as:

npm ERR! Error: failed to fetch from registry: mime
npm ERR!     at /usr/share/npm/lib/utils/npm-registry-client/get.js:139:12
npm ERR!     at cb (/usr/share/npm/lib/utils/npm-registry-client/request.js:31:9)
npm ERR!     at Request._callback (/usr/share/npm/lib/utils/npm-registry-client/request.js:136:18)
npm ERR!     at Request.callback (/usr/lib/nodejs/request/main.js:119:22)
npm ERR!     at Request.<anonymous> (/usr/lib/nodejs/request/main.js:212:58)
npm ERR!     at Request.emit (events.js:88:20)
npm ERR!     at ClientRequest.<anonymous> (/usr/lib/nodejs/request/main.js:412:12)
npm ERR!     at ClientRequest.emit (events.js:67:17)
npm ERR!     at HTTPParser.onIncoming (http.js:1261:11)
npm ERR!     at HTTPParser.onHeadersComplete (http.js:102:31)

I also noticed that if npm is not installed we still get a success:

/bin/bash: npm: command not found

In the case of such a failure we should exit immediately with a non-zero exit code so that Jenkins will fail the build.
it is not make test-perf per see, it is a common target for the other tests too.

Also

http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Perf/job/b2g.hamachi.mozilla-central.master.mozperftest/3387/consoleText

Pinging jlal.
Assignee: nobody → hub
Priority: -- → P1
Whiteboard: [c=automation p= s= u=]
Also, by default Make should stop the rule if a non-zero return code occur. But this ain't happening.
ok, so I get what's happening here: gaia-perf-marionette calls make to regenerate node_modules and doesn't check the error code.

Nonetheless, the error is caused by James last change.
Status: NEW → ASSIGNED
Whiteboard: [c=automation p= s= u=] → [c=automation p=1 s= u=]
Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

I gave 3 simpler alternatives, please pick one.
Attachment #8438458 - Flags: review?(felash)
Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

I picked one. Thanks !
Attachment #8438458 - Flags: review?(felash)
Comment on attachment 8438458 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20366

r=me
yeah :)
Attachment #8438458 - Flags: review?(felash) → review+
https://github.com/mozilla-b2g/gaia/commit/41db6954a67efc55016744bc8f6591ae9e07a285
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2.0 S4 (20june)
You need to log in before you can comment on or make changes to this bug.