718329 - Add max tolerance value to fuzzy reftests

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Matt Woodrow (:mattwoodrow)]

Attachment: Add fuzzy-max option

In some cases we want to support differences in reftests greater than 2 (skia in particular). I feel that it's better to have this configurable rather than disabling the tests entirely.

Comment 1

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I actually think the maximum number of different pixels, and the maximum difference per pixel, should be mandatory parameters to 'fuzzy'/'fuzzy-if'.

Comment 2

[Matt Woodrow (:mattwoodrow)]

Attachment: Add max values to fuzzy/fuzzy-if instead

Alternative option that roc suggested.

Comment 3

[Matt Woodrow (:mattwoodrow)]

Attachment: Add max values to fuzzy/fuzzy-if instead v2

Updated tolerance values to match the failures seen on tinderbox, and updated for new fuzzy tests that have been added since the previous patch was posted.

dbaron: Do you mind if roc reviews this?

Comment 4

[Matt Woodrow (:mattwoodrow)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/05df0cddc299

Comment 5

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/05df0cddc299

Presume the other patch is now obsolete?

Comment 6

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

This patch should have updated the documentation.  This has now been reported twice; see bug 729784.