Took a quick look in gdb while doing triage. The code in question is: ``` static void ConstrainToCoordValues(float& aStart, float& aSize) { MOZ_ASSERT(aSize >= 0); ``` https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/layout/base/nsLayoutUtils.cpp#2245-2246 ...and we have `aStart = inf` and `aSize = -nan(0x400000)`. And NaN fails all comparisons, so it fails the >=0 comparison here. How do we get `inf` and `NaN`? Well, up one level, in RoundGfxRectToAppRect, we have a passed-in `aRect` with these components: ``` { x = 3.40282347e+38, y = 0, width = 6, height = 6 } ``` And we scale up that rect using a call to `ScaleRoundOut()`, which basically sets `x = x*60` and then sets `width = XMost()*60 - x`. Our `x` value is large enough that the 60x multiplication bumps it up to be infinity, and so XMost() is also infinity, so we end up with `x=inf` and `width=inf-inf=NaN`, and then those are the values we pass into this nsLayoutUtils API. More than likely, nothing terrible happens as a result. But we should probably be clamping and/or checking for NaN here somewhere. Probably nsLayoutUtils::RoundGfxRectToAppRect() should be prepared for having huge & possibly NaN/inf values in its rect variables (either from being passed-in or from creating them locally via the 60x scaling), and it should simply clamp those to something sensible like `(0,0,0,0)` perhaps.
Bug 1565065 Comment 2 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Took a quick look in gdb while doing triage. The code in question is: ``` static void ConstrainToCoordValues(float& aStart, float& aSize) { MOZ_ASSERT(aSize >= 0); ``` https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/layout/base/nsLayoutUtils.cpp#2245-2246 ...and we have `aStart = inf` and `aSize = -nan(0x400000)`. And NaN fails all comparisons, so it fails the >=0 comparison here. How do we get `inf` and `NaN`? Well, up one level, in RoundGfxRectToAppRect, we have a passed-in `aRect` with these components: ``` { x = 3.40282347e+38, y = 0, width = 6, height = 6 } ``` And we scale up that rect using a call to `ScaleRoundOut()`, which basically sets `x = x*60` and then sets `width = XMost()*60 - x`. Our `x` value is large enough that the 60x multiplication bumps it up to be infinity, and so XMost() is also infinity, so we end up with `x = inf` and `width = (inf - inf) = NaN`, and then those are the values we pass into this nsLayoutUtils API. More than likely, nothing terrible happens as a result. But we should probably be clamping and/or checking for NaN here somewhere. Probably nsLayoutUtils::RoundGfxRectToAppRect() should be prepared for having huge & possibly NaN/inf values in its rect variables (either from being passed-in or from creating them locally via the 60x scaling), and it should simply clamp those to something sensible like `(0,0,0,0)` perhaps.