Travis JS job is green even though the log shows tests are failing

RESOLVED FIXED

Status

Tree Management
Treeherder: Docs & Development
P1
normal
RESOLVED FIXED
a year ago
11 months ago

People

(Reporter: emorley, Assigned: emorley)

Tracking

(Blocks: 1 bug)

Details

Attachments

(1 attachment)

(Assignee)

Description

a year ago
There are multiple failures visible in the logs of the JS run on Travis, however `yarn test` is still returning zero, which means the overall job shows as passing/green:
https://travis-ci.org/mozilla/treeherder/jobs/230892456

We need to:
1) Figure out why `yarn test` isn't exiting non-zero
2) Fix the test failures that have crept in whilst this has been broken

I'm guessing this is a regression from the Neutrino switch.
(Assignee)

Updated

a year ago
Depends on: 1364031
(Assignee)

Updated

a year ago
Depends on: 1364033
(Assignee)

Updated

a year ago
Blocks: 1353014
(Assignee)

Comment 1

11 months ago
Travis reports 'The command "yarn test" exited with 0.'.

`yarn test` runs `neutrino test --presets ./neutrino-custom/test.js`:
https://github.com/mozilla/treeherder/blob/d9045d01b1dd14a61e5f003649b3cb51fac40ba7/neutrino-custom/test.js

...which makes use of neutrino-preset-karma v4.2.0:
https://github.com/mozilla-neutrino/neutrino-dev/blob/85ed67f035de81a861d49a74f97f6dbede61c004/packages/neutrino-preset-karma/src/index.js

Whilst we're on an old version of Neutrino, it doesn't look like that preset has changed much between v4 and master:
https://github.com/mozilla-neutrino/neutrino-dev/blob/master/packages/neutrino-preset-karma/index.js

...so I've filed an issue against Neutrino to check it's working as expected:
https://github.com/mozilla-neutrino/neutrino-dev/issues/268
Assignee: nobody → emorley

Comment 2

11 months ago
Created attachment 8879313 [details] [review]
[treeherder] mozilla:fix-travis-js-tests > mozilla:master
(Assignee)

Updated

11 months ago
Depends on: 1374478

Comment 3

11 months ago
Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/72a3ce8140600b6fa4712cf6e60a2fb98c707808
Bug 1364030 - Update to neutrino-preset-karma 4.2.1

To pick up the fix for failing JS tests not making the Travis run
fail (mozilla-neutrino/neutrino-dev#268).
(Assignee)

Updated

11 months ago
Status: NEW → RESOLVED
Last Resolved: 11 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.