Closed Bug 1287515 Opened 3 years ago Closed 3 years ago

int-overflow: 0xFFFFFFFFFFFFFFFF bytes requested in [@SkDashPathEffect::asPoints]

Categories

(Core :: Canvas: 2D, defect, critical)

50 Branch
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla50
Tracking Status
firefox50 --- fixed

People

(Reporter: tsmith, Assigned: lsalzman)

References

(Blocks 2 open bugs)

Details

(Keywords: csectype-intoverflow, testcase, Whiteboard: [gfx-noted])

Attachments

(3 files, 1 obsolete file)

Attached file log.txt
Repros on non ASan and non debug builds.

Log from an ASan build:
==20174==WARNING: AddressSanitizer failed to allocate 0xffffffffffffffff bytes
==20174==AddressSanitizer's allocator is terminating the process instead of returning 0
==20174==If you don't like this behavior set allocator_may_return_null=1
==20174==AddressSanitizer CHECK failed: /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:147 "((0)) != (0)" (0x0, 0x0)
    #0 0x4bb54d in __asan::AsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_rtl.cc:67:3
    #1 0x4bf3e1 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:159:5
    #2 0x4bdc53 in __sanitizer::ReportAllocatorCannotReturnNull() /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:147:3
    #3 0x41d238 in ReturnNullOrDie /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1317:5
    #4 0x41d238 in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_allocator.cc:359
    #5 0x4b2534 in malloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:53:10
    #6 0x4e06bd in moz_xmalloc /builds/slave/m-in-l64-asan-0000000000000000/build/src/memory/mozalloc/mozalloc.cpp:83:17
    #7 0x7fa080257000 in operator new[] /builds/slave/m-in-l64-asan-0000000000000000/build/src/obj-firefox/dist/include/mozilla/mozalloc.h:205:12
    #8 0x7fa080257000 in SkDashPathEffect::asPoints(SkPathEffect::PointData*, SkPath const&, SkStrokeRec const&, SkMatrix const&, SkRect const*) const /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/skia/skia/src/effects/SkDashPathEffect.cpp:264
    #9 0x7fa0804d46fa in SkDraw::drawPoints(SkCanvas::PointMode, unsigned long, SkPoint const*, SkPaint const&, bool) const /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/skia/skia/src/core/SkDraw.cpp:647:25
    #10 0x7fa08021ae8a in SkCanvas::onDrawPoints(SkCanvas::PointMode, unsigned long, SkPoint const*, SkPaint const&) /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:2082:9
    #11 0x7fa08022b7d7 in drawPoints /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:1902:5
    #12 0x7fa08022b7d7 in SkCanvas::drawLine(float, float, float, float, SkPaint const&) /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:2827
    #13 0x7fa078a2730e in mozilla::gfx::DrawTargetSkia::StrokeLine(mozilla::gfx::PointTyped<mozilla::gfx::UnknownUnits, float> const&, mozilla::gfx::PointTyped<mozilla::gfx::UnknownUnits, float> const&, mozilla::gfx::Pattern const&, mozilla::gfx::StrokeOptions const&, mozilla::gfx::DrawOptions const&) /builds/slave/m-in-l64-asan-0000000000000000/build/src/gfx/2d/DrawTargetSkia.cpp:577:3
    #14 0x7fa07b6486a4 in mozilla::dom::CanvasRenderingContext2D::StrokeRect(double, double, double, double) /builds/slave/m-in-l64-asan-0000000000000000/build/src/dom/canvas/CanvasRenderingContext2D.cpp:2872:5
...
see log.txt for full log.
Attached file test_case.html
Based on log, I think this was reproduced with asan build on linux.
Vincent, are you able to reproduce this?
Flags: needinfo?(vliu)
Whiteboard: [gfx-noted]
SkScalarFloorToInt(len2 / fIntervalLength) was overflowing the resulting int, causing it to go negative. When that int is cast to a size_t, further havoc ensues. Since this is a fast-path for dash rendering, we just want to bail out here when the conditions aren't sane.
Assignee: nobody → lsalzman
Status: NEW → ASSIGNED
Attachment #8772854 - Flags: review?(mchang)
Comment on attachment 8772854 [details] [diff] [review]
limit the number of points in SkDashPathEffect::asPoints

Review of attachment 8772854 [details] [diff] [review]:
-----------------------------------------------------------------

Can we also check in the test case here as a crash test please?
Attachment #8772854 - Flags: review?(mchang) → review+
Now with more crashtestiness...
Attachment #8772854 - Attachment is obsolete: true
Attachment #8772913 - Flags: review?(mchang)
Comment on attachment 8772913 [details] [diff] [review]
limit the number of points in SkDashPathEffect::asPoints

Review of attachment 8772913 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!
Attachment #8772913 - Flags: review?(mchang) → review+
Summary: OOM: 0xFFFFFFFFFFFFFFFF bytes requested in [@SkDashPathEffect::asPoints] → int-overflow: 0xFFFFFFFFFFFFFFFF bytes requested in [@SkDashPathEffect::asPoints]
Upstream Skia bug report: https://codereview.chromium.org/2165013002/
Clear ni since lee is working on this.
Flags: needinfo?(vliu)
Pushed by lsalzman@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4bc6b6fe87b6
limit the number of points in SkDashPathEffect::asPoints. r=mchang
Duplicate of this bug: 1286593
Blocks: 1289929, grizzly
Flags: in-testsuite+
You need to log in before you can comment on or make changes to this bug.