853989 - Reftest analyzer should label the images instead of just calling them "Image 1" and "Image 2"

Status: RESOLVED DUPLICATE of bug 1016260

(Testing :: Reftest, defect)

Description

[Seth Fowler [:seth] [:s2h]]

When looking at reftests which you're unfamiliar with, it's occasionally unclear which image is the reference and which is the failing case - sometimes they both look reasonable. It'd be great if the images had more meaningful labels.

Comment 1

[Ed Morley [:emorley]]

TBPL uses (in a rather dirty way) the reftest analyser from mozilla-central:
https://hg.mozilla.org/mozilla-central/file/tip/layout/tools/reftest/reftest-analyzer.xhtml

(Loaded from https://hg.mozilla.org/webtools/tbpl/file/01c330f101de/js/UserInterface.js#l1510)

Agree it would be good to rename them :-)

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Well, the use of "test" and "reference" is really a convention rather than the only way to use reftest, so I'm a little hesitant to label the images as other than "image 1" and "image 2".  In a test with:

== file1 file2

"image 1" is the image from file1, and "image 2" is the image from file2; it's just the order of arguments to the == operation.


By "other ways to use reftest", I mean things like this:

(from layout/reftests/table-width/reftest.list)
== shrink-wrap-pref.html shrink-wrap-pref-ref.html
== shrink-wrap-avail.html shrink-wrap-avail-ref.html
== shrink-wrap-min.html shrink-wrap-min-ref.html
!= shrink-wrap-min-ref.html shrink-wrap-pref-ref.html
!= shrink-wrap-pref-ref.html shrink-wrap-avail-ref.html
!= shrink-wrap-avail-ref.html shrink-wrap-min-ref.html

(from layout/reftests/bugs/reftest.list... though perhaps this one is bad style)
== 404030-1.html 404030-1-ref.html
!= 404030-1-notref.html 404030-1.html
!= 404030-1-notref2.html 404030-1.html

Comment 3

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

That said, I guess reftest does bias towards the first file by using the first of the two arguments to identify the test.

Comment 4

[Seth Fowler [:seth] [:s2h]]

Perhaps we could use "Test (file.html)" and "Reference (file-ref.html)". In the common case the Test/Reference labelling would be correct, but in cases where it isn't, the filenames should clear things up.