Failure in layout/reftests/reftest-sanity/test-displayport-2.html is not analyzed by reftest analyzer



5 years ago
4 years ago


(Reporter: kats, Unassigned)


Firefox Tracking Flags

(Not tracked)


See - the reftest analyzer can't analyze the reftest failure there. The log looks something like this:

19:27:50     INFO -  REFTEST TEST-START | file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/reftest-sanity/test-displayport-2.html
19:27:50     INFO -  REFTEST TEST-LOAD | file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/reftest-sanity/test-displayport-2.html | 70 / 119 (58%)
19:27:51     INFO -  REFTEST   IMAGE: data:image/png;base64,iVBORw0KGgoAA...
19:27:51     INFO -  REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/reftest-sanity/test-displayport-2.html | image comparison (!=)
19:27:51     INFO -  REFTEST INFO | Saved log: START file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/reftest-sanity/test-displayport-2.html
19:27:51     INFO -  REFTEST INFO | Saved log: [CONTENT] Setting viewport to <w=100, h=100>

And I think the reftest analyzer can't deal with the REFTEST IMAGE line because it comes before the UNEXPECTED-FAIL line. I don't know if this is a bug in the analyzer or the reftest log file generator.
Blocks: 883646
If I paste the log into the reftest analyzer page manually with the image appearing after the unexpected-fail line then it works.
Doesn't the reftest analyzer intentionally not try to deal with != tests, where there's only a single image in the log because the two were supposed to look different but failed to look different, so there's no difference between them which it can highlight?
I don't think so; just the other day I opened a reftest analyzer using the TBPL link and it showed me failures with just the one image.
You need to log in before you can comment on or make changes to this bug.