789602 - reftest harness should print pixel delta information on the UNEXPECTED-FAIL line

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Attachment: patch

Right now the reftest harness prints the pixel errors (number of pixels and maximum color difference) below the images.  Since they're not on the same line as the TEST-UNEXPECTED-FAIL, they don't end up in intermittent-orange bugs.  In some cases, having it in the bugs would be very useful for annotating the problem as a known failure.

This would, however, be very useful data to have on intermittent orange bugs, so we ought to have this on the same line as the TEST-UNEXPECTED-FAIL message.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/acb63df12b7a

Comment 2

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/acb63df12b7a

Comment 3

[Ed Morley [:emorley]]

Landed on beta so we have this for ESR17 for the next year.

https://hg.mozilla.org/releases/mozilla-beta/rev/8739f223b6e2

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e8c41ecfc045

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e8c41ecfc045