963464 - [10.9] lcamtuf's canvas fuzzer: "Error: this application has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API. It will become a fatal error in an upcoming update."

Status: RESOLVED FIXED

(Core :: Graphics: Canvas2D, defect)

Description

[Chris Peterson [:cpeterson]]

STR:
1. On OS X 10.9 Mavericks, run Firefox from the terminal (so you can watch stderr).
2. Load http://lcamtuf.coredump.cx/canvas/
3. Check options "Return large integers (exploitable crash in Safari, OOM/DoS elsewhere)" and "Skip time-consuming operations (quicker, but may miss issues)"
4. Click "Begin tests"
5. Watch stderr

RESULT:
CoreGraphics will log hundreds of warnings about invalid numeric values that will be fatal errors in an upcoming OS X update. Firefox should probably filter these invalid numeric values so we don't trigger fatal CoreGraphics errors.

Dec 31 15:59:59 cpeterson firefox[82511] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API. This is a serious error and contributes to an overall degradation of system stability and reliability. This notice is a courtesy: please fix this problem. It will become a fatal error in an upcoming update.

Comment 1

[Steven Michaud [:smichaud] (Retired)]

I'm not sure how seriously to take Apple's warning -- probably not very.

But the error messages themselves are annoying.  So perhaps it's worth some trouble to get rid of them -- especially now that you've given us a reliable way to reproduce them.

This error message is printed (on OS X 10.9.X) by a CoreGraphics method named CGFloatIsValid().  This is undocumented, and only called (grep tells me) from the CoreGraphics framework itself.  Here are the names of all the methods that can call it:

CGPathAddLineToPoint()
CGPathAddQuadCurveToPoint()
CGPathAddCurveToPoint()
CGPathAddArc()
CGPathAddArcToPoint()
CGPathAddRelativeArc()

All of these methods are documented here:
https://developer.apple.com/library/mac/documentation/graphicsimaging/reference/CGPath/Reference/reference.html

With one exception (CGPathAddRoundedRect()) these are all the CGPathAdd... methods that take CGFloat parameters.  Whoever wanted to get rid of these messages would need to grep the Mozilla tree for the above-listed methods, then figure out the best way to sanitize their input.

Comment 2

[Steven Michaud [:smichaud] (Retired)]

All the calls to the above-listed methods are from a single file:
http://hg.mozilla.org/mozilla-central/annotate/045dd7740be7/gfx/2d/PathCG.cpp

Comment 3

[Steven Michaud [:smichaud] (Retired)]

This might be a good first bug.

Comment 4

[Steven Michaud [:smichaud] (Retired)]

By the way, I got the information in comment #1 using grep (in /System/Library/ and /usr/lib/) and an excellent disassembler -- the Hopper Disassembler (http://www.hopperapp.com/).

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Is this still an issue?

Comment 6

[Chris Peterson [:cpeterson]]

I can't reproduce these errors with 58.0a1 on macOS 10.13.