Closed Bug 1473859 Opened 2 years ago Closed 6 months ago

Intermittent css-valuesandunits/unit-vh-vw-overflow-auto.html != css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 4

Categories

(Core :: CSS Parsing and Computation, defect, P5)

defect

Tracking

()

RESOLVED FIXED
mozilla69
Tracking Status
firefox68 --- fixed
firefox69 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: dholbert)

Details

(Keywords: intermittent-failure, Whiteboard: [TV-bf pass:pass][stockwell disabled])

Attachments

(2 files)

Filed by: cbrindusan [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=186807118&repo=autoland

https://queue.taskcluster.net/v1/task/KWQRHkdGTSSTgdhos3b_2w/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/KWQRHkdGTSSTgdhos3b_2w/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2018-07-06T10:56:07.426Z] 10:56:07     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto-ref.html
[task 2018-07-06T10:56:07.427Z] 10:56:07     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | 10 / 15 (66%)
[task 2018-07-06T10:56:07.494Z] 10:56:07     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto-ref.html | 10 / 15 (66%)
[task 2018-07-06T10:56:07.836Z] 10:56:07     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto-ref.html | image comparison, max difference: 1, number of differing pixels: 2

Over the last 7 days there are 48 failures on this bug. These happen on osx-10-10, linux64-ccov, linux64, linux32.

Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=235011045&repo=autoland&lineNumber=2627

task 2019-03-20T16:35:20.271Z] 16:35:20 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y.html | 13 / 16 (81%)
[task 2019-03-20T16:35:20.319Z] 16:35:20 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y-ref.html | 13 / 16 (81%)
[task 2019-03-20T16:35:20.377Z] 16:35:20 INFO - REFTEST TEST-KNOWN-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y-ref.html | image comparison, max difference: 214, number of differing pixels: 160
[task 2019-03-20T16:35:20.379Z] 16:35:20 INFO - REFTEST TEST-END | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-y-ref.html
[task 2019-03-20T16:35:20.391Z] 16:35:20 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html
[task 2019-03-20T16:35:20.536Z] 16:35:20 INFO - REFTEST TEST-UNEXPECTED-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 2

Flags: needinfo?(svoisen)
Keywords: leave-open
Whiteboard: [TV-bf pass:pass][stockwell disable-recommended] → [TV-bf pass:pass][stockwell disabled]
Pushed by apavel@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/215e4d951935
Disable css-valuesandunits/unit-vh-vw-overflow-auto-ref.html on linux for frequent failures r=jmaher

There have been 91 failures within the last 7 days:

  • 22 failures on Linux x32 debug
  • 13 failures on Linux shippable opt
  • 4 failures on Linux x64 CCov debug
  • 6 failures on Linux x64 shippable opt
  • 3 failures on OS X 10.10 debug
  • 9 failures on OS X 10.10 shippable
  • 34 failures on Linux x64 debug/asan/opt

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=237702077&repo=autoland&lineNumber=9063

Whiteboard: [TV-bf pass:pass][stockwell disabled] → [TV-bf pass:pass][stockwell disabled][stockwell needswork]

This bug has failed 73 times in the last 7 days. These happen on Linux and OSX platforms on debug, asan and opt build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=239065856&repo=autoland&lineNumber=6215

We have 92 occurrences of this bug in the past week, all on Linux platform opt, debug and asan

Recent log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241330371&repo=autoland&lineNumber=14282

Attachment #9053130 - Attachment description: Bug 1473859 - Disable css-valuesandunits/unit-vh-vw-overflow-auto-ref.html on linux for frequent failures → Bug 1473859 - Disable != unit-vh-vw-overflow-auto.html unit-vh-vw-overflow-scroll.html on linux for frequent failures.

There are 96 total failures in the last 7 days on linux and osx platforms: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-04-21&endday=2019-04-28&tree=trunk&bug=1473859

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=243105217&repo=autoland&lineNumber=8751

[task 2019-04-27T14:15:59.409Z] 14:15:59 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace
[task 2019-04-27T14:15:59.747Z] 14:15:59 INFO - REFTEST TEST-UNEXPECTED-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 2
[task 2019-04-27T14:15:59.747Z] 14:15:59 INFO - REFTEST IMAGE 1 (TEST): [...removing for brevity...]

Flags: needinfo?(svoisen)
Whiteboard: [TV-bf pass:pass][stockwell disabled][stockwell needswork] → [TV-bf pass:pass][stockwell disabled][stockwell needswork:owner]

Sean, could you please assign this to someone?

Flags: needinfo?(svoisen)
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3bc847f39bd7
Disable != unit-vh-vw-overflow-auto.html unit-vh-vw-overflow-scroll.html on linux for frequent failures. r=jmaher

In the last 7 days, there have been 65 failures.

Most of the failures are on linux64, linux32, linux32-shippable and linux64-shippable platforms.
Affected build types: debug, opt, asan.

An example of a recent log file: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245268251&repo=mozilla-central&lineNumber=40492

And the relevant part of the log:

task 2019-05-08T05:01:09.144Z] 05:01:09     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html
[task 2019-05-08T05:01:09.145Z] 05:01:09     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html | 14 / 16 (87%)
[task 2019-05-08T05:01:09.164Z] 05:01:09     INFO - ++DOMWINDOW == 76 (0x7fcad586c000) [pid = 9906] [serial = 84] [outer = 0x7fcad8ccdd40]
[task 2019-05-08T05:01:09.182Z] 05:01:09     INFO - ++DOCSHELL 0x7fcad588d000 == 10 [pid = 9906] [id = {e13d19a3-6a54-4c93-bc4d-22b7d065fea1}]
[task 2019-05-08T05:01:09.183Z] 05:01:09     INFO - ++DOMWINDOW == 77 (0x7fcad7f30980) [pid = 9906] [serial = 85] [outer = (nil)]
[task 2019-05-08T05:01:09.183Z] 05:01:09     INFO - ++DOCSHELL 0x7fcad588e000 == 11 [pid = 9906] [id = {9ea5a1c9-e08c-4a7b-927b-b989721357e1}]
[task 2019-05-08T05:01:09.184Z] 05:01:09     INFO - ++DOMWINDOW == 78 (0x7fcad7f30b60) [pid = 9906] [serial = 86] [outer = (nil)]
[task 2019-05-08T05:01:09.192Z] 05:01:09     INFO - ++DOMWINDOW == 79 (0x7fcad586f000) [pid = 9906] [serial = 87] [outer = 0x7fcad7f30b60]
[task 2019-05-08T05:01:09.208Z] 05:01:09     INFO - ++DOMWINDOW == 80 (0x7fcad5871800) [pid = 9906] [serial = 88] [outer = 0x7fcad7f30980]
[task 2019-05-08T05:01:09.276Z] 05:01:09     INFO - --DOMWINDOW == 79 (0x7fcad7c39000) [pid = 9906] [serial = 9] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-div-width-outer.html]
[task 2019-05-08T05:01:09.280Z] 05:01:09     INFO - --DOMWINDOW == 78 (0x7fcad7783000) [pid = 9906] [serial = 10] [outer = (nil)] [url = data:text/html;charset=UTF-8,%3C%21%2D%2DCLEAR%2D%2D%3E]
[task 2019-05-08T05:01:09.281Z] 05:01:09     INFO - --DOMWINDOW == 77 (0x7fcad7c38000) [pid = 9906] [serial = 11] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-iframe.html]
[task 2019-05-08T05:01:09.281Z] 05:01:09     INFO - --DOMWINDOW == 76 (0x7fcad778a000) [pid = 9906] [serial = 20] [outer = (nil)] [url = data:text/html;charset=UTF-8,%3C%21%2D%2DCLEAR%2D%2D%3E]
[task 2019-05-08T05:01:09.283Z] 05:01:09     INFO - --DOMWINDOW == 75 (0x7fcad7c38400) [pid = 9906] [serial = 21] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-ref2-root-fontsize.html]
[task 2019-05-08T05:01:09.285Z] 05:01:09     INFO - --DOMWINDOW == 74 (0x7fcad7da6c00) [pid = 9906] [serial = 17] [outer = (nil)] [url = data:text/html;charset=UTF-8,%3C%21%2D%2DCLEAR%2D%2D%3E]
[task 2019-05-08T05:01:09.287Z] 05:01:09     INFO - --DOMWINDOW == 73 (0x7fcad7da6400) [pid = 9906] [serial = 19] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-ref-root-fontsize.html]
[task 2019-05-08T05:01:09.287Z] 05:01:09     INFO - --DOMWINDOW == 72 (0x7fcad7c2dc00) [pid = 9906] [serial = 14] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-ref-iframe.html]
[task 2019-05-08T05:01:09.288Z] 05:01:09     INFO - --DOMWINDOW == 71 (0x7fcad7c39c00) [pid = 9906] [serial = 16] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-ref-iframe-inside.html]
[task 2019-05-08T05:01:09.292Z] 05:01:09     INFO - --DOMWINDOW == 70 (0x7fcad8c5a400) [pid = 9906] [serial = 18] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-root-fontsize.html]
[task 2019-05-08T05:01:09.292Z] 05:01:09     INFO - --DOMWINDOW == 69 (0x7fcad7da5400) [pid = 9906] [serial = 13] [outer = (nil)] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-rem-iframe-inside.html]
[task 2019-05-08T05:01:09.293Z] 05:01:09     INFO - --DOCSHELL 0x7fcad8061800 == 10 [pid = 9906] [id = {a32205ae-7764-4a97-a97d-66bd03752bf9}] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-x-ref-iframe.html]
[task 2019-05-08T05:01:09.293Z] 05:01:09     INFO - --DOCSHELL 0x7fcad8062800 == 9 [pid = 9906] [id = {b8e5cd8b-0454-4733-8083-62b7eb3f1b8d}] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-x-ref-iframe.html]
[task 2019-05-08T05:01:09.295Z] 05:01:09     INFO - --DOCSHELL 0x7fcad805c000 == 8 [pid = 9906] [id = {7a972b9e-2bcc-450a-a146-e40c655e38a4}] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-iframe.html]
[task 2019-05-08T05:01:09.297Z] 05:01:09     INFO - --DOCSHELL 0x7fcad805a800 == 7 [pid = 9906] [id = {be948154-55c0-4efb-89a5-5cb0806227c9}] [url = file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll-iframe.html]
[task 2019-05-08T05:01:09.441Z] 05:01:09     INFO - REFTEST TEST-UNEXPECTED-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 2
[task 2019-05-08T05:01:09.443Z] 05:01:09     INFO - REFTEST   IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAcVUlEQVR4nO3Xsc0oW3Je0QaUmYJhIMTQZA7EC0ohjMMHQlICI4sXsi4HOkezu/9vGctrHGx0OVXP8zx/AwAA+AfJAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHsDz/O3f/+2f//Yf//pPv/XnH3/JO5eZEQDAFXkAz/NfLrb/qe5cZkYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXHH2wP/47//tR6gH8f9zua3/rRkBAPB/OXugXkott2b0E2YEADDk7IF6KbXcmtFPmBEAwJCzB+ql1HJrRj9hRgAAQ84eqJdSy60Z/YQZAQAMOXugXkott2b0E2YEADDk7IF6KbXcmtFPmBEAwJCzB+ql1HJrRj9hRgAAQ84eqJdSy60Z/YQZAQAMOXugXkott2b0E2YEADDk7IGvLI9v94YDpP4Hb+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA5bbOxwg7+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA5bbOxwg7+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA5bbOxwg7+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3vHnH3/5LxfbP//4FzP6oTMCABhy9oDl9o6//vWvfxcz+pkzAgAYcvaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh74e5fbf4QX/Mz/Z284QMyomxEAwJCzB+qF1nJrRj9hRgAAQ84eqBday60Z/YQZAQAMOXugXmgtt2b0E2YEADDk7IF6obXcmtFPmBEAwJCzB+qF1nJrRj9hRgAAQ84eqBday60Z/YQZAQAMOXugXmgtt2b0E2YEADDk7IF6obXcmtFPmBEAwJCzB+qF1nJrRj9hRgAAQ84eWFg8/xHecIDU/+DtHCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACuyAN4LLdfYEYAAFfkATyW2y8wIwCAK/IAHsvtF5gRAMAVeQCP5fYLzAgA4Io8gMdy+wVmBABwRR7AY7n9AjMCALgiD+Cx3H6BGQEAXJEH8Fhuv8CMAACu+P0H//5v//y3//jXf/qH+/OPv/zdC99P8L//1//8u5jRN2cEAMAvv/+gWGz/U71wfmW5NaP3zwgAgF9+/4Hl9v3LrRm9f0YAAPzy+w8st+9fbs3o/TMCAOCX339guX3/cmtG758RAAC//P4Dy+37l1szev+MAAD45fcfWG7fv9ya0ftnBADAL7//wHL7/uXWjN4/IwAAfvn9B5bb9y+3ZvT+GQEA8MvvP/jzj78ki+2ff/xLvnB+Zbk1o/fPCACAX/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAMAAIAdeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAI/4PBFSjiA5FdpUAAAAASUVORK5CYII=
[task 2019-05-08T05:01:09.448Z] 05:01:09     INFO - REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAcYklEQVR4nO3Xwc0wW1ae0ZKcmYMhEGiG5AA3KIfQE66QTQLNiCsPrJ+Wz6Gfqu9dgzUrHT2qPdn7eZ7nLwAAAH8jeQAAALAjDwAAAHbkAQAAwI48AAAA2JEHAAAAO/IAAABgRx4AAADsyAN4nr/86z///V/+7Z/+7pd+/+1Peeey3//lH8wIAOBcHsDz/JeL7X+qO5eZEQDAFXkAj+X2C8wIAOCKPIDHcvsFZgQAcEUewGO5/QIzAgC4Ig/gsdx+gRkBAFyRB/BYbr/AjAAArsgDeCy3X2BGAABXnD3wv/7n//gR6kH8dy639b81IwAA/i9nD9RLqeXWjH7CjAAAhpw9UC+lllsz+gkzAgAYcvZAvZRabs3oJ8wIAGDI2QP1Umq5NaOfMCMAgCFnD9RLqeXWjH7CjAAAhpw9UC+lllsz+gkzAgAYcvZAvZRabs3oJ8wIAGDI2QP1Umq5NaOfMCMAgCFnD9RLqeXWjH7CjAAAhpw98JXl8e3ecIDU/+DtHCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7HCDv5wABALji7AHL7R0OkPdzgAAAXHH2gOX2DgfI+zlAAACuOHvAcnuHA+T9HCAAAFecPWC5vcMB8n4OEACAK84esNze4QB5PwcIAMAVZw9Ybu9wgLyfAwQA4IqzByy3dzhA3s8BAgBwxdkDlts7fv/tT//lYvv7b/9oRj90RgAAQ84esNze8ec///mvYkY/c0YAAEPOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA5bbOxwg7+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA5bbOxwg7+cAAQC44uwBy+0dDpD3c4AAAFxx9oDl9g4HyPs5QAAArjh7wHJ7hwPk/RwgAABXnD1gub3DAfJ+DhAAgCvOHrDc3uEAeT8HCADAFWcPWG7vcIC8nwMEAOCKswcst3c4QN7PAQIAcMXZA3/tcvu38IKf+f/tDQeIGXUzAgAYcvZAvdBabs3oJ8wIAGDI2QP1Qmu5NaOfMCMAgCFnD9QLreXWjH7CjAAAhpw9UC+0llsz+gkzAgAYcvZAvdBabs3oJ8wIAGDI2QP1Qmu5NaOfMCMAgCFnD9QLreXWjH7CjAAAhpw9UC+0llsz+gkzAgAYcvZAvdBabs3oJ8wIAGDI2QMLi+ffwhsOkPofvJ0DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHDF2QOW2zscIO/nAAEAuOLsAcvtHQ6Q93OAAABccfaA5fYOB8j7OUAAAK44e8Bye4cD5P0cIAAAV5w9YLm9wwHyfg4QAIArzh6w3N7hAHk/BwgAwBVnD1hu73CAvJ8DBADgirMHLLd3OEDezwECAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBV5AI/l9gvMCADgijyAx3L7BWYEAHBFHsBjuf0CMwIAuCIP4LHcfoEZAQBckQfwWG6/wIwAAK7IA3gst19gRgAAV+QBPJbbLzAjAIAr8gAey+0XmBEAwBW//uBf//nv//Jv//R3f3O///anv3rh+wn+/f/877/K/2tGv//LP5jRy2cEAMAffv1Bsdj+p3rh/Mpya0bvnxEAAH/49QeW2/cvt2b0/hkBAPCHX39guX3/cmtG758RAAB/+PUHltv3L7dm9P4ZAQDwh19/YLl9/3JrRu+fEQAAf/j1B5bb9y+3ZvT+GQEA8Idff2C5ff9ya0bvnxEAAH/49QeW2/cvt2b0/hkBAPCHX3/w+29/Shbb33/7x3zh/Mpya0bvnxEAAH/IAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAOzIAwAAgB15AAAAsCMPAAAAduQBAADAjjwAAADYkQcAAAA78gAAAGBHHgAAAIz4D+TYo47RNLWAAAAAAElFTkSuQmCC
[task 2019-05-08T05:01:09.453Z] 05:01:09     INFO - REFTEST INFO | Saved log: START file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html
[task 2019-05-08T05:01:09.455Z] 05:01:09     INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-05-08T05:01:09.456Z] 05:01:09     INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-05-08T05:01:09.457Z] 05:01:09     INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-05-08T05:01:09.459Z] 05:01:09     INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-05-08T05:01:09.460Z] 05:01:09     INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-05-08T05:01:09.462Z] 05:01:09     INFO - REFTEST INFO | Saved log: RecordResult fired

Joel, this is still failing on linux, could you please take a look at this?

Thank you!

Flags: needinfo?(jmaher)

I suspect this is a case where the fails line invalidates the skip-if...although what about android?

I would change from skip-if(gtkWidget) -> fuzzy-if(gtkWidget,0-1,0-2) and leave the rest alone

Flags: needinfo?(jmaher)
Pushed by dvarga@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/46287ed82b8f
Changed skip-if to fuzzy-if on linux disable r=jmaher
Whiteboard: [TV-bf pass:pass][stockwell disabled][stockwell needswork:owner] → [TV-bf pass:pass][stockwell disabled]

This bug failed 51 times in the last 7 days. Occurs on linux32, linux32-shippable, linux64, linux64-shippable on debug and asan build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=247162075&repo=mozilla-inbound&lineNumber=7877

comment 86 (acting on the suggestion in comment 82) didn't help, because the fails annotation still overrides everything else that comes before it on that line including the new fuzzy-if annotation. So I don't think that commit made a difference.

In fact, a fuzzy annotation doesn't really make sense here, because this is a "!=" test (i.e. these things should not look alike). So in comment 81, when we saw this...

REFTEST TEST-UNEXPECTED-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-auto.html != file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 2

...then really, that line of log meant:

  • We compared these two files, which are expected to render differently, and they did render differently!
  • And this didn't match our expectations, because we think this test will normally fail (i.e. we think they do normally render the same)
  • And in this case, even though we unexpectedly "passed" (rendering the files differently), we in fact rendered them basically-the-same with an inconsequential fuzzy difference. So, this "unexpected-pass" was really still the two files rendering basically-the-same (which should be considered a failure).

Over the last 7 days there are 32 failures on this bug. These happen on linux64, linux64-ccov, linux64-shippable

Here is the latest log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249544343&repo=mozilla-central&lineNumber=9469

The unit-vh-vw-overflow-auto-ref.html for the same test was fixed in bug 1325798.
Fixing the Summary here to help the robot classify these intermittent failures correctly.

Summary: Intermittent css-valuesandunits/unit-vh-vw-overflow-auto.html == css-valuesandunits/unit-vh-vw-overflow-auto-ref.html | image comparison, max difference: 1, number of differing pixels: 2 → Intermittent css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 4
Summary: Intermittent css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 4 → Intermittent css-valuesandunits/unit-vh-vw-overflow-auto.html != css-valuesandunits/unit-vh-vw-overflow-scroll.html | image comparison, max difference: 1, number of differing pixels: 4

I don't get why we use "fails !=" for this test; why don't we simply use "==" so we can use fuzzy normally?

Flags: needinfo?(dholbert)

(In reply to Mats Palmgren (:mats) from comment #98)

I don't get why we use "fails !=" for this test; why don't we simply use "==" so we can use fuzzy normally?

As I understand it, the test is a "!=" test -- the expectation is that it mismatches. If we changed it to "==", then we'd be inverting the expectation.

It looks like it did used to pass (i.e. the files used to render differently) before stylo shipped, and we added 'fails' as part of the stylo implementation:
https://searchfox.org/mozilla-central/diff/c729f26a66da9b4988320e147708a2e08daaee5d/layout/reftests/css-valuesandunits/reftest.list#15

So now it fails (i.e. the files render the same), which is purportedly a bug in stylo. But occasionally there is a few-pixel mismatch which makes the files seem to mismatch (pass), though the mismatch is inconsequential.

I suppose we could swap to "==", but we'd want to be sure to add a comment to indicate that the equality-expectation is actually wrong, with an explanation about what the original expectation was and why.

Flags: needinfo?(dholbert)

I'm guessing that bug 1393603 (in particular bug 1393603 comment 5) may have been the underlying stylo issue here that caused this test to fail.

Given that bug 1393603 is WONTFIX (and our behavior matches Chrome/Safari according to that bug), perhaps we should indeed update the test expectation to "==" rather than "!="? emilio, what do you think?

Flags: needinfo?(emilio)

Yes, that test should be changed to use == at this point. rs=me on that change I guess? :)

Flags: needinfo?(emilio)

OK, thanks. I'll make that change.

Flags: needinfo?(svoisen)
Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b5f993ff67c1
Change expectation for reftest unit-vh-vw-overflow-auto.html. rs=emilio

Hopefully this is now FIXED. Resolving as such, though feel free to reopen if we get more instances of intermittent failure associated with this test.

(Though really, a fresh bug would probably be better, unless there's a mistake in the commit in comment 103/104.)

Status: NEW → RESOLVED
Closed: 6 months ago
Keywords: leave-open
Resolution: --- → FIXED
Assignee: nobody → dvarga
Assignee: dvarga → dholbert
Target Milestone: --- → mozilla69
You need to log in before you can comment on or make changes to this bug.