1626496 - Round down for nearest filtering in SWGL

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P1)

Description

[Lee Salzman [:lsalzman]]

The OpenGL spec mandates that nearest filtering for texture sampling must use the "Manhattan distance" to find the nearest pixel, which boils down to a floor or truncate (which are the same for positive numbers) on the texture coordinates.

Currently SWGL is using the default rounding mode of round-to-nearest-even (a.k.a. banker's rounding) which is clearly buggy with regards to specified OpenGL behavior...

The OpenGL spec however does mandate that round-to-nearest-even is used on the Z/layer coordinate for indexing into a texture array regardless of filtering, so we need to use different rounding modes for depending on the texture coordinate axis in question.

Comment 1

[Lee Salzman [:lsalzman]]

Attachment: Bug 1626496 - round texture coordinates down when nearest filtering in SWGL. r?jrmuizel

Comment 2

[Lee Salzman [:lsalzman]]

Attachment: Bug 1626496 - fix bit casting of vectors for GCC. r?jrmuizel

Depends on D69150

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e53a045094df
round texture coordinates down when nearest filtering in SWGL. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/929cb28d1a1f
fix bit casting of vectors for GCC. r=jrmuizel

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/e53a045094df
https://hg.mozilla.org/mozilla-central/rev/929cb28d1a1f