940895 - Intermittent REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/839886-1.html | image comparison (==), max difference: 255, number of differing pixels: 8

Status: RESOLVED WORKSFORME

(Testing :: Reftest, defect)

Description

[Carsten Book [:Tomcat]]

Ubuntu VM 12.04 b2g-inbound opt test reftest on 2013-11-19 22:22:21 PST for push 18de3f6395e5

slave: tst-linux32-ec2-040

https://tbpl.mozilla.org/php/getParsedLog.php?id=30813086&tree=B2g-Inbound

REFTEST TEST-UNEXPECTED-FAIL | file:///builds/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/839886-1.html | image comparison (==), max difference: 255, number of differing pixels: 8

Comment 1

[Mats Palmgren (inactive)]

Attachment: Relevant portion of the log

Comment 2

[Mats Palmgren (inactive)]

I saved the data: urls locally in plain text files and then compared them -- they are identical!

I don't understand why the framework thinks there was difference here.  Looking at it in the
reftest-analyzer there's 1-2px difference around the edges of the two images, showed as
different shades of grey in the magnifier.  I don't see those grey pixels in the actual images
though.  And reftest-analyzer doesn't wrap any areas in red borders, AFAICT.

Does anyone understand what happened here?

Comment 3

[Legacy TBPL/Treeherder Robot]

Tomcat
https://tbpl.mozilla.org/php/getParsedLog.php?id=43423112&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound opt test reftest-no-accel on 2014-07-09 03:51:36
revision: ef24cd472cfb
slave: t-w864-ix-013

REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addLTR-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 472
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addRTL-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 472
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/839886-1.html | image comparison (==), max difference: 55, number of differing pixels: 338
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/box/flexbox-attributes-no-input-vertical.xhtml | image comparison (==), max difference: 55, number of differing pixels: 1640
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-01.xul | image comparison (==), max difference: 39, number of differing pixels: 255
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-05.xul | image comparison (==), max difference: 39, number of differing pixels: 300
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-21.xul | image comparison (==), max difference: 96, number of differing pixels: 255
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/370422-1.html | image comparison (==), max difference: 129, number of differing pixels: 2532
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/text-overflow/block-padding.html | image comparison (==), max difference: 39, number of differing pixels: 2670
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/dom/multipleappendwithinput.xhtml | image comparison (==), max difference: 55, number of differing pixels: 338
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-input-text-contents-at-2.html | image comparison (==), max difference: 55, number of differing pixels: 676
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-select-combobox-contents-at-1.html | image comparison (==), max difference: 129, number of differing pixels: 1506
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-select-combobox-contents-at-2.html | image comparison (==), max difference: 129, number of differing pixels: 1506
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/min-width-passes-1.html | image comparison (==), max difference: 62, number of differing pixels: 22906
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/forms/input/radio/label-dynamic.html | image comparison (==), max difference: 141, number of differing pixels: 149
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/forms/input/color/margin-padding-1.html | image comparison (==), max difference: 62, number of differing pixels: 522
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/position-sticky/padding-3.html | image comparison (==), max difference: 39, number of differing pixels: 525
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/text/449555-1.html | image comparison (==), max difference: 129, number of differing pixels: 538
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/xul/accesskey.xul | image comparison (==), max difference: 55, number of differing pixels: 112
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/spellcheck-hyphen-multiple-valid.html | image comparison (==), max difference: 55, number of differing pixels: 1220
Only displaying first 20 of 21 failures -
.

Comment 4

[Mats Palmgren (inactive)]

The failure in comment 3 is actually bug 970576.  I added a comment there.

Comment 5

[Legacy TBPL/Treeherder Robot]

:nigelb
https://tbpl.mozilla.org/php/getParsedLog.php?id=46089364&tree=Fx-Team
WINNT 6.2 fx-team pgo test reftest-no-accel on 2014-08-15 20:41:03
revision: 70b77e14d448
slave: t-w864-ix-052

REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addLTR-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 472
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addRTL-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 472
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/839886-1.html | image comparison (==), max difference: 55, number of differing pixels: 338
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-21.xul | image comparison (==), max difference: 96, number of differing pixels: 255
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/201293-1c.html | image comparison (==), max difference: 62, number of differing pixels: 4623
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/508816-1.xul | image comparison (==), max difference: 62, number of differing pixels: 2263
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/text-overflow/block-padding.html | image comparison (==), max difference: 39, number of differing pixels: 2670
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/dom/multipleappendwithinput.xhtml | image comparison (==), max difference: 55, number of differing pixels: 338
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-select-combobox-contents-at-1.html | image comparison (==), max difference: 129, number of differing pixels: 1506
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-select-combobox-contents-at-2.html | image comparison (==), max difference: 129, number of differing pixels: 1506
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/min-width-passes-1.html | image comparison (==), max difference: 62, number of differing pixels: 23434
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/xul/accesskey.xul | image comparison (==), max difference: 55, number of differing pixels: 112
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/spellcheck-hyphen-multiple-valid.html | image comparison (==), max difference: 55, number of differing pixels: 1220
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/spellcheck-url-valid.html | image comparison (==), max difference: 55, number of differing pixels: 1368

Comment 6

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-10-11T12:18:12
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-aurora&job_id=316863
repository: mozilla-aurora
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-009
buildname: WINNT 6.2 mozilla-aurora pgo test reftest
revision: e06d47be2a7b

REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addLTR-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 468
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/dynamicDirAuto-addRTL-Auto.html | image comparison (==), max difference: 55, number of differing pixels: 468
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bidi/dirAuto/839886-1.html | image comparison (==), max difference: 55, number of differing pixels: 324
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-01.xul | image comparison (==), max difference: 39, number of differing pixels: 255
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-02.xul | image comparison (==), max difference: 39, number of differing pixels: 300
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/192767-21.xul | image comparison (==), max difference: 96, number of differing pixels: 255
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/bugs/370422-1.html | image comparison (==), max difference: 129, number of differing pixels: 2532
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/text-overflow/block-padding.html | image comparison (==), max difference: 39, number of differing pixels: 2550
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/dom/multipleappendwithinput.xhtml | image comparison (==), max difference: 55, number of differing pixels: 324
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/threshold-select-combobox-contents-at-2.html | image comparison (==), max difference: 129, number of differing pixels: 1498
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/font-inflation/min-width-passes-1.html | image comparison (==), max difference: 62, number of differing pixels: 23636
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/spellcheck-hyphen-multiple-valid.html | image comparison (==), max difference: 55, number of differing pixels: 1220
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/spellcheck-url-valid.html | image comparison (==), max difference: 55, number of differing pixels: 1348
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/editor/reftests/642800.html | image comparison (==), max difference: 55, number of differing pixels: 468

Comment 7

[Ed Morley [:emorley]]

Inactive; closing (see bug 1180138).