2010486 - Imprecision in Imperial unit conversion

Status: UNCONFIRMED

(Firefox :: Address Bar, defect)

Description

[Justin Peter]

Typing "0.75lb to oz" in the address bar outputs "11.9999999999 oz". This should output "12 oz".

I'm not sure if this is classic floating point rounding, or if it is a result of the constant for ounce not being exactly 16x that of pound in https://searchfox.org/firefox-main/source/browser/components/urlbar/unitconverters/UnitConverterSimple.sys.mjs#146-149.

Comment 1

[Justin Peter]

Actually, per Wikipedia (and verified with a few sources) the pound is defined as exactly 0.45359237 kg, so the constant listed in the file above is incorrect past about the 5th decimal place.