Open Bug 1453074 Opened 3 years ago Updated 3 years ago

Reftest failures can be missed if the output is split across multiple lines

Categories

(Testing :: Reftest, defect)

59 Branch
defect
Not set
normal

Tracking

(Not tracked)

People

(Reporter: kats, Unassigned)

Details

See https://treeherder.mozilla.org/logviewer.html#?job_id=172906991&repo=try - this job is passing, but if you open the raw log and search for "UnexpectedPass" you'll see that one of the reftests is failing with an unexpected-pass result. But the json "strucutured output" is broken across multiple lines with DOMWINDOW spew in between, and presumably that's why it doesn't get picked up by whatever is parsing the log for results.

The try push is at https://treeherder.mozilla.org/#/jobs?repo=try&revision=6dd6c81630bf4e1cdf060012e60e49526e52464f - all the Linux R3 things should be orange but some of them are green because of this issue.
Component: General → Reftest
You need to log in before you can comment on or make changes to this bug.