One potential hole in the non-canonical nan-shield is the math functions. By and large, platforms seem to only produce the canonical nan. Apparently, though, some don't. For example, bug 584653 shows that, on OS X 10.5 x64 builds, asin(4) returns a non-canonical nan (really Apple?). Bug 584168 shows that canonicalizing all math functions across the board costs 1-2% on SS and V8, so I'd like to do it conditionally by platform with some #define like JS_MATH_NEEDS_CANONICALIZATION. Then, for platforms where we don't canonicalize, we should have a jsapi-test that pumps a bunch of numbers through all the math functions to verify no non-canonical nans pop out the other end.
I made comment 0 private as exposing info about asin may hurt nightly build users.
fwiw, bug 584653 was determined to be non-exploitable because it confused the type-tagging in a predictably-crashing way.
Right. Only half the bug remains: build a jsapi test that gives us some build-time assurance that our math libs are sane.
Summary: conditionally canonicalize nans coming out of math.h → write jsapi-test for math libraries and canonical nan
You need to log in before you can comment on or make changes to this bug.