1860338 - very bad precision in calculations involving CSS units vw and vh

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect)

Description

[firefoxbugs]

Steps to reproduce:

Given this environment:
100vw == 1920px (regular FHD display)

These should be equivalent:
calc( 100vw * 9900 )
calc( 1920px * 9900 )

Actual results:

But they're not even close:
calc( 100vw * 9900 ) -> 1.78957e+7px
calc( 1920px * 9900 ) -> 1.9008e+7px

It gets worse:

Given this environment:
100vh == 412px
This should be zero:
calc( ( 100vh - 412px ) * 900000 )
But instead, we get:
-3.52904e+8px
By back calculating, we're basically being told that 100vh ~= 19.88px which is VERY FAR from 412px.

Expected results:

Obviously with floating point math comes a little bit of error, but one should not be getting off by more than 6% simply because of multiplying both sides of an equation by a 4-digit number or more than 95% off simply by multiplying the sides by a 6-digit number.

Now compare to cqw:
100vw == 100cqw == 1920px
calc(100cqw - 1920px) -> 0px (as expected)
calc(100vw - 1920px) -> 0px (as expected)
calc((100cqw - 1920px) * 9900) -> 0px (as expected)
calc((100vw - 1920px) * 9900) -> -1112300px
Not very close at all, I'd say.

With very large numbers, cqw does show some compromised precision due to rearranging operation:
calc((100cqw - 1920px) * 1e30 / 3) -> 3.86856e+25px
It seems to distribute the error into the parentheses before the subtraction, whereas calculating the parentheses first would give the correct answer (0px).
But compared to vw and vh, cqw and cqh are much more usable.

[[Someone is likely to say, "Who needs to calculate something with so many pixels anyway?" There are probably lots of reasons, but here's mine: Given that mod(), rem(), and round() are all currently unusable (totally broken on Firefox and not implemented on Chromium), developers need workarounds and alternatives to make page logic work. If I want to emphasize a potentially very small difference between two calculated dimensions (which can subsequently be pigeonholed into different values with clamp(), min() or max()), I'm going to multiply the difference by a really big number. Calculations on the units vw and vh are complete garbage even with small multipliers. Calculations with cqw and cqh are better but still less reliable because of doing operations out of order. ]]

Comment 1

[Jonathan Kew [:jfkthame]]

But they're not even close:
calc( 100vw * 9900 ) -> 1.78957e+7px
calc( 1920px * 9900 ) -> 1.9008e+7px

It kinda looks to me like the vw computation was done in appUnits (1/60 of a px), and clamped to a limit of nscoord_MAX (2^30 - 1):

nscoord_MAX = ((1 << 30) - 1) = 1073741823 appunits = 17895697.05px, or 1.78957e+7px

whereas the px computation stayed in px throughout and didn't get clamped.

(Note that even calc(200vw * 9900) still computes to 1.78957e+7px, whereas calc(3840px * 9900) is 3.8016e+7px, as expected. The vw version is not just losing accuracy, it's actually clamped.)

Comment 2

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1860338 - Don't clamp to appUnit range during viewport-unit calculations. r=#style

Comment 3

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1860338 - Add a couple of testcases that check for clamping of viewport-unit values. r=#style

Depends on D192791

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/38a28572ca4a
Don't clamp to appUnit range during viewport-unit calculations. r=firefox-style-system-reviewers,emilio
https://hg.mozilla.org/integration/autoland/rev/54ed30bf1e15
Add a couple of testcases that check for clamping of viewport-unit values. r=firefox-style-system-reviewers,emilio

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/42957 for changes under testing/web-platform/tests

Comment 6

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/38a28572ca4a
https://hg.mozilla.org/mozilla-central/rev/54ed30bf1e15

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 8

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot