Created attachment 643238 [details]
Consider the attached document. I'd expect it to print "20.5px" but instead it prints "20.4833px", one app unit off.
I get 20.5, for what it's worth. At least on a Mac.
Oh yeah, forgot to mention: this is on 32 bit Linux. (And I think it also happens on Windows, based on try server failures of a non-reduced version of this test.)
I think the issue is due to 0.411 not being exactly representable as a float. The computation is done with:
I assume different compilers are doing the floating point calculations differently, and because we're rounding down when finally converting to a nscoord and not rounding to nearest, we end up losing that one app unit.
Is it reasonable to change it to use NSToCoordRound here?
(In reply to Cameron McCormack (:heycam) from comment #4)
> Is it reasonable to change it to use NSToCoordRound here?
Created attachment 643525 [details] [diff] [review]
Comment on attachment 643525 [details] [diff] [review]
(What adds the funny number at the end of the file?)
Ah that's git-format-patch that inserts that (and the script I have to convert the git patch to an hg patch seems to retain it). It's the version of git that produced the patch.