952788 - Many reftests failing on OS X 10.8.5

Status: NEW

(Core :: Layout, enhancement, P5)

Description

[Ray Kiddy]

I am seeing rather a lot of reftests failing on the FF that I am building from newly checked out sources.

Should I file a bug for each of the UNEXPECTED-FAIL results that I see? I would not mind doing it, unless I am going to hear that there is some configuration issue I have that makes the results invalid.

Is the master branch supposed to be working?

I did a 'git pull' and then a 'mach build' and a 'mach reftest' and another 'mach reftest'. During the first reftest run I forgot to disable display sleep. That used to invalidate some tests, so I disabled display sleep and ran it again. Lots of issues:

$ grep UNEXPECTED-FAIL build_and_reftest_207a21dbe8152418b0f11f802a9faaad785b989b.txt
 0:08.57 REFTEST TEST-UNEXPECTED-FAIL | WARNING: USE_WIDGET_LAYERS disabled
 1:53.02 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-1.html | image comparison (==), max difference: 255, number of differing pixels: 301
 1:53.12 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-2.html | image comparison (==), max difference: 255, number of differing pixels: 301
 1:54.36 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/transforms-1.html | image comparison (==), max difference: 1, number of differing pixels: 25
 2:09.25 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/240933-2.html | image comparison (==), max difference: 255, number of differing pixels: 140
 2:13.59 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/301726-2.html | image comparison (!=)
 2:29.53 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/367247-s-auto.html | image comparison (!=)
 2:57.24 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/424074-1.xul | image comparison (!=)
 3:42.10 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/617242-1.html | image comparison (==), max difference: 1, number of differing pixels: 4
 4:03.38 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/933264-1.html | image comparison (==), max difference: 32, number of differing pixels: 462
 4:15.23 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-gradients/linear-percent.html | image comparison (==), max difference: 1, number of differing pixels: 11985
 4:45.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-overflow/xulscroll.html | image comparison (==), max difference: 255, number of differing pixels: 961
 4:55.20 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | image comparison (!=)
 5:09.10 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/canvas/test/reftest/webgl-color-alpha-test.html?colorVal=1.0&alphaVal=0.5&alpha&premult | image comparison (==), max difference: 255, number of differing pixels: 65536
 6:37.77 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/placeholder/placeholder-6-textarea.html | image comparison (==), max difference: 39, number of differing pixels: 33
 6:55.89 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/resize-background.html | image comparison (==), max difference: 101, number of differing pixels: 143
 6:56.23 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/ltr-scrollbar.html | image comparison (!=)
 6:56.31 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/in-ltr-doc-scrollbar.html | image comparison (!=)
 7:09.84 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/image-element/gradient-html-07a.html | image comparison (==), max difference: 2, number of differing pixels: 42003
 8:20.44 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.52 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.60 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.68 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.76 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.83 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirks.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:20.95 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.03 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.19 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.27 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.35 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.55 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.62 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.71 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:21.78 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 8:30.69 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-sanity.html | image comparison (==), max difference: 2, number of differing pixels: 4165
 8:30.80 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncbitmap-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
 8:31.58 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncdxgi-sanity.html | image comparison (==), max difference: 2, number of differing pixels: 4165
10:07.69 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/selection/pseudo-element-of-native-anonymous.html | image comparison (!=)
10:15.61 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/position-sticky/block-in-inline-2.html | image comparison (==), max difference: 1, number of differing pixels: 8
11:40.18 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/svg/radialGradient-basic-03.svg | image comparison (==), max difference: 3, number of differing pixels: 14981
17:54.38 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-shadow/text-shadow-on-selection-1.html | load failed: timed out waiting for reftest-wait to be removed
22:54.38 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-shadow/text-shadow-on-selection-2.html | load failed: timed out waiting for reftest-wait to be removed
22:55.36 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-svgglyphs/svg-glyph-objectgradient-zoom.svg | image comparison (==), max difference: 1, number of differing pixels: 7007
23:17.12 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/xul/grid/reftests/scrollable-columns.xul | image comparison (==), max difference: 128, number of differing pixels: 1090
28:18.12 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-load.html | load failed: timed out waiting for reftest-wait to be removed
33:18.12 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-create.html | load failed: timed out waiting for reftest-wait to be removed
38:18.12 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-number.html | load failed: timed out waiting for reftest-wait to be removed
43:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/button-load.html | load failed: timed out waiting for reftest-wait to be removed
48:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/button-create.html | load failed: timed out waiting for reftest-wait to be removed
53:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/textarea-load.html | load failed: timed out waiting for reftest-wait to be removed
58:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/textarea-create.html | load failed: timed out waiting for reftest-wait to be removed
63:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/select-load.html | load failed: timed out waiting for reftest-wait to be removed
68:18.11 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/select-create.html | load failed: timed out waiting for reftest-wait to be removed
72:58.42 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/autofocus-leaves-iframe.html | image comparison (==), max difference: 255, number of differing pixels: 1800
73:06.27 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 1852
73:06.61 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 922
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | WARNING: USE_WIDGET_LAYERS disabled
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-1.html | image comparison (==), max difference: 255, number of differing pixels: 301
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-2.html | image comparison (==), max difference: 255, number of differing pixels: 301
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/transforms-1.html | image comparison (==), max difference: 1, number of differing pixels: 25
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/240933-2.html | image comparison (==), max difference: 255, number of differing pixels: 140
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/301726-2.html | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/367247-s-auto.html | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/424074-1.xul | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/617242-1.html | image comparison (==), max difference: 1, number of differing pixels: 4
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/933264-1.html | image comparison (==), max difference: 32, number of differing pixels: 462
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-gradients/linear-percent.html | image comparison (==), max difference: 1, number of differing pixels: 11985
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-overflow/xulscroll.html | image comparison (==), max difference: 255, number of differing pixels: 961
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/canvas/test/reftest/webgl-color-alpha-test.html?colorVal=1.0&alphaVal=0.5&alpha&premult | image comparison (==), max difference: 255, number of differing pixels: 65536
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/placeholder/placeholder-6-textarea.html | image comparison (==), max difference: 39, number of differing pixels: 33
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/resize-background.html | image comparison (==), max difference: 101, number of differing pixels: 143
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/ltr-scrollbar.html | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/in-ltr-doc-scrollbar.html | image comparison (!=)
73:17.90 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/image-element/gradient-html-07a.html | image comparison (==), max difference: 2, number of differing pixels: 42003
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirks.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-sanity.html | image comparison (==), max difference: 2, number of differing pixels: 4165
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncbitmap-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncdxgi-sanity.html | image comparison (==), max difference: 2, number of differing pixels: 4165
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/selection/pseudo-element-of-native-anonymous.html | image comparison (!=)
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/position-sticky/block-in-inline-2.html | image comparison (==), max difference: 1, number of differing pixels: 8
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/svg/radialGradient-basic-03.svg | image comparison (==), max difference: 3, number of differing pixels: 14981
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-shadow/text-shadow-on-selection-1.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-shadow/text-shadow-on-selection-2.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-svgglyphs/svg-glyph-objectgradient-zoom.svg | image comparison (==), max difference: 1, number of differing pixels: 7007
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/xul/grid/reftests/scrollable-columns.xul | image comparison (==), max difference: 128, number of differing pixels: 1090
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-load.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-create.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/input-number.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/button-load.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/button-create.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/textarea-load.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/textarea-create.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/select-load.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/select-create.html | load failed: timed out waiting for reftest-wait to be removed
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/html/content/reftests/autofocus/autofocus-leaves-iframe.html | image comparison (==), max difference: 255, number of differing pixels: 1800
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 1852
73:17.91 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 922
 0:05.46 REFTEST TEST-UNEXPECTED-FAIL | WARNING: USE_WIDGET_LAYERS disabled
 1:35.65 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-1.html | image comparison (==), max difference: 255, number of differing pixels: 301
 1:35.73 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-2.html | image comparison (==), max difference: 255, number of differing pixels: 301
 1:36.92 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/transforms-1.html | image comparison (==), max difference: 1, number of differing pixels: 25
 1:47.82 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/240933-2.html | image comparison (==), max difference: 255, number of differing pixels: 140
 1:51.66 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/301726-2.html | image comparison (!=)
 1:59.67 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/367247-s-auto.html | image comparison (!=)
 2:16.42 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/424074-1.xul | image comparison (!=)
 2:51.54 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/617242-1.html | image comparison (==), max difference: 1, number of differing pixels: 4
 3:07.59 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/933264-1.html | image comparison (==), max difference: 21, number of differing pixels: 462
 3:16.17 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-gradients/linear-percent.html | image comparison (==), max difference: 1, number of differing pixels: 11985
 3:44.01 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-overflow/xulscroll.html | image comparison (==), max difference: 255, number of differing pixels: 961
 3:51.77 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | image comparison (!=)
 4:03.35 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/canvas/test/reftest/webgl-color-alpha-test.html?colorVal=1.0&alphaVal=0.5&alpha&premult | image comparison (==), max difference: 255, number of differing pixels: 65536
 5:16.67 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/placeholder/placeholder-6-textarea.html | image comparison (==), max difference: 39, number of differing pixels: 33
 5:23.36 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/resize-background.html | image comparison (==), max difference: 101, number of differing pixels: 143
 5:23.52 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/ltr-scrollbar.html | image comparison (!=)
 5:23.60 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/in-ltr-doc-scrollbar.html | image comparison (!=)
 5:30.95 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/image-element/gradient-html-07a.html | image comparison (==), max difference: 2, number of differing pixels: 42003
 6:22.73 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:22.81 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:22.89 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:22.98 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.06 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.14 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirks.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.26 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.35 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.42 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.50 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.58 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.66 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.78 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.86 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:23.94 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:24.02 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:24.09 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
 6:30.79 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
 6:30.86 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncbitmap-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 5
 6:31.64 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncdxgi-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
 7:59.37 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/position-sticky/block-in-inline-2.html | image comparison (==), max difference: 1, number of differing pixels: 8
 8:59.01 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/svg/radialGradient-basic-03.svg | image comparison (==), max difference: 3, number of differing pixels: 14981
 9:47.79 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-svgglyphs/svg-glyph-objectgradient-zoom.svg | image comparison (==), max difference: 1, number of differing pixels: 7007
10:05.76 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/xul/grid/reftests/scrollable-columns.xul | image comparison (==), max difference: 128, number of differing pixels: 1090
10:11.72 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 1852
10:12.06 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 922
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | WARNING: USE_WIDGET_LAYERS disabled
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-1.html | image comparison (==), max difference: 255, number of differing pixels: 301
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/scrollbar-clamping-2.html | image comparison (==), max difference: 255, number of differing pixels: 301
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/border-radius/transforms-1.html | image comparison (==), max difference: 1, number of differing pixels: 25
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/240933-2.html | image comparison (==), max difference: 255, number of differing pixels: 140
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/301726-2.html | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/367247-s-auto.html | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/424074-1.xul | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/617242-1.html | image comparison (==), max difference: 1, number of differing pixels: 4
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/bugs/933264-1.html | image comparison (==), max difference: 21, number of differing pixels: 462
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-gradients/linear-percent.html | image comparison (==), max difference: 1, number of differing pixels: 11985
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-overflow/xulscroll.html | image comparison (==), max difference: 255, number of differing pixels: 961
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/content/canvas/test/reftest/webgl-color-alpha-test.html?colorVal=1.0&alphaVal=0.5&alpha&premult | image comparison (==), max difference: 255, number of differing pixels: 65536
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/placeholder/placeholder-6-textarea.html | image comparison (==), max difference: 39, number of differing pixels: 33
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/resize-background.html | image comparison (==), max difference: 101, number of differing pixels: 143
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/ltr-scrollbar.html | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/forms/textarea/in-ltr-doc-scrollbar.html | image comparison (!=)
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/image-element/gradient-html-07a.html | image comparison (==), max difference: 2, number of differing pixels: 42003
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeight.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirks.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-2D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-1D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollSimpleMinHeightQuirks-3D.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/percent-overflow-sizing/hScrollAbsMinHeightQuirksD.html | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncbitmap-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 5
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/plugins/test/reftest/plugin-asyncdxgi-sanity.html | image comparison (==), max difference: 1, number of differing pixels: 4151
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/position-sticky/block-in-inline-2.html | image comparison (==), max difference: 1, number of differing pixels: 8
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/svg/radialGradient-basic-03.svg | image comparison (==), max difference: 3, number of differing pixels: 14981
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/reftests/text-svgglyphs/svg-glyph-objectgradient-zoom.svg | image comparison (==), max difference: 1, number of differing pixels: 7007
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/layout/xul/grid/reftests/scrollable-columns.xul | image comparison (==), max difference: 128, number of differing pixels: 1090
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 1852
10:21.47 REFTEST TEST-UNEXPECTED-FAIL | file:///Users/ray/Projects/mozilla/gecko-dev/dom/imptests/html/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs.html | image comparison (==), max difference: 128, number of differing pixels: 922

Comment 1

[Ray Kiddy]

Attachment: build_and_reftest_207a21dbe8152418b0f11f802a9faaad785b989b.txt.gz

build log and reftest run log

Comment 2

[Daniel Holbert [:dholbert]]

Attachment: cleaned up reftest log (just test failures & snapshots)

Here's a pruned version of your log, with basically just the test failures and snapshots.

You can feed this into http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml to see what the failures actually look like.

They largely all seem to be different things -- some of them are off-by-1px alignment issues, some are 1px-is-fuzzy issues, some of them are rounded-corners-not-quite-right issues.  And some of them look like legitimate test-failures (content that shows up in reference does not show up in testcase).

Unfortunately, it's probably not workable to cover all of these test-failures in a single bug (and to the extent that this is caused by local-platform-quirks, this may already be sort of covered by metabug 514067). But it may be worth filing bugs on specific failures that look particularly egregious, and on categories of similar-looking failures.

Comment 3

[Ray Kiddy]

I am not expecting to be able to manage these all in a single bug. I do not mind filing however many bugs it takes. I just wanted to find out if there was something I am doing wrong that would lead to me closing all the bugs.

Comment 4

[Mats Palmgren (inactive)]

Most of these failures appears to be caused by the OSX fade-away scrollbars.
I'm guessing your machine has different performance characteristics than the
testing machines, thus revealing timing issues in the tests.  All such
tests can probably be handled in the same bug, which would depend on
bug 952613 (we should force static scrollbars for these specific tests).

The off-by-one color errors could probably be handled as one bug as well.
Please have a look at the documentation of the "fuzzy" parameters in 
http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/README.txt
then modify the manifests (reftest.list) to make them pass.

Please file any new bugs as Priority:P5 and Severity:enhancement.

Comment 5

[Liz Henry (:lizzard) (relman/hg->git project)]

Bug 926292 may be related; see comment 27: https://bugzilla.mozilla.org/show_bug.cgi?id=926292#c27