1590371 - [wpt-sync] Sync PR 19825 - reftests: if images hashes don't match, try to compare pixels

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P4)

Description

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Sync web-platform-tests PR 19825 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/19825
Details from upstream follow.

Carlos Alberto Lopez Perez <[e-mail]> wrote:

reftests: if images hashes don't match, try to compare pixels

• When comparing the result of a reftest, checking if the hash of
  the expected result matches the hash of the test is not enough.
  An image can have different hash, but still be the very same
  image (same pixels).

• This patch implements checks for pixel equality between the two
  images when the hashes are different.

• It has been observed that in WebKit based browsers (Safari, WebKitGTK)
  this happens and many tests are beeing flagged incorrectly as failing.

Comment 1

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Pushed to try https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd4d59f4fe1618d39fb0a2a0f0d5edf643910079

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df43f2225d1f
[wpt PR 19825] - reftests: if images hashes don't match, try to compare pixels, a=testonly

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Result changes from PR not available.

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/df43f2225d1f