Open Bug 654218 Opened 9 years ago Updated 7 years ago
reftest harness prints only one images for tests which pass unexpectedly
STR: run reftests in layout/reftests/bugs/reftest.list and make sure the browser window doesn't have the focus. You will get this in the logs: REFTEST TEST-START | file:///Users/ehsanakhgari/moz/mozilla-central/layout/reftests/bugs/503531-1.html REFTEST TEST-START | file:///Users/ehsanakhgari/moz/mozilla-central/layout/reftests/bugs/503531-1-ref.html EFTEST TEST-UNEXPECTED-PASS | file:///Users/ehsanakhgari/moz/mozilla-central/layout/reftests/bugs/503531-1.html | image comparison (==) REFTEST IMAGE: data:image/png;base64,... REFTEST INFO | Loading a blank page This confuses reftest-analyzer.xhtml...
Since an unexpected pass on a "==" comparison means the two images are identical, I think it's sensible enough that it only prints it out once. What confusion does it cause reftest-analyzer.xhtml?
I thought reftest-analyzer.xhtml handled this fine. In any case, the behavior of the harness is as-expected.
I guess I've reported this in another way in the past. reftest-analyzer treats this type of situation by just showing one image, which has confused me on multiple occasions. I'd rather see two identical images, with maybe a piece of text explicitly saying that they're identical. But if you both think this is a better UI, then I think I'll probably just change my local copy...
I think it would make sense for reftest-analyzer to have something at the top that says "One identical image" or something like that (instead of the selector for flipping between them). That would be a useful reminder.
You need to log in before you can comment on or make changes to this bug.