Bug 1542002 Comment 4 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

Original comment by
on 
Relevant snippet from the log:
> INFO	Allowed 0-1 pixels different, maximum difference per channel 8-8
> INFO	Found 0 pixels different, maximum difference per channel 0
> TEST_END: FAIL, expected PASS - Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> 
> TEST-UNEXPECTED-FAIL | /css/CSS2/lists/list-style-021.xht | Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> REFTEST   IMAGE 1 (TEST): data:image/png;[...]
> REFTEST   IMAGE 2 (REFERENCE): data:image/png;[...same data URI as above...]

Right now my fuzzy annotation has `maxDifference=8;totalPixels=0-1`.  If I change it to have `maxDifference=0-8`, **then** the test starts being reported as passing. James, should I make that change? (i.e. should I use a 0-8 range for maxDifference)

I might be misremembering, but I think the "regular" reftest harness would treat my current annotation (without the maxDifference range) as meaning: "1 pixel is allowed to differ, but it's also fine if it doesn't differ.  If it does differ, then you should expect its maxDifference to be exactly 8".  Apparently that is not how WPT is interpreting my annotation though, because it treats no-pixels-differing as a failure, despite the `numPixels=0-1` in my syntax.
Revision 1 by
on 
Relevant snippet from the log:
> INFO	Allowed 0-1 pixels different, maximum difference per channel 8-8
> INFO	Found 0 pixels different, maximum difference per channel 0
> TEST_END: FAIL, expected PASS - Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> 
> TEST-UNEXPECTED-FAIL | /css/CSS2/lists/list-style-021.xht | Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> REFTEST   IMAGE 1 (TEST): data:image/png;[...]
> REFTEST   IMAGE 2 (REFERENCE): data:image/png;[...same data URI as above...]

Right now my fuzzy annotation has `maxDifference=8;totalPixels=0-1`.  If I change it to have `maxDifference=0-8`, **then** the test starts being reported as passing. James, should I make that change? (i.e. should I use a 0-8 range for maxDifference)

I might be misremembering, but I think the "regular" reftest harness would treat my current annotation (without the maxDifference range) as meaning: "1 pixel is allowed to differ, but it's also fine if it doesn't differ.  If it does differ, then you should expect its maxDifference to be exactly 8".  Apparently that is not how WPT is interpreting my annotation though, because it treats no-pixels-differing as a failure, despite the `totalPixels=0-1` in my syntax.
Revision 2 by
on 
Relevant snippet from the log:
> INFO	Allowed 0-1 pixels different, maximum difference per channel 8-8
> INFO	Found 0 pixels different, maximum difference per channel 0
> TEST_END: FAIL, expected PASS - Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> 
> TEST-UNEXPECTED-FAIL | /css/CSS2/lists/list-style-021.xht | Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
> REFTEST   IMAGE 1 (TEST): data:image/png;[...]
> REFTEST   IMAGE 2 (REFERENCE): data:image/png;[...same data URI as above...]

Right now my fuzzy annotation has `maxDifference=8;totalPixels=0-1`.  If I change it to have `maxDifference=0-8`, **then** the test starts being reported as passing. James, should I make that change? (i.e. should I use a 0-8 range for maxDifference)

I might be misremembering, but I think the "regular" reftest harness would treat my current annotation (without 0 included in the maxDifference range) as meaning: "1 pixel is allowed to differ, but it's also fine if it doesn't differ.  If it does differ, then you should expect its maxDifference to be exactly 8".  Apparently that is not how WPT is interpreting my annotation though, because it treats no-pixels-differing as a failure, despite the `totalPixels=0-1` in my syntax.

Back to Bug 1542002 Comment 4