1439802 - UBSan divide by zero in [@ nsDisplayTransform::UntransformRect]

Status: RESOLVED WORKSFORME

(Core :: Web Painting, defect)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

Not sure if this should go in layout or graphics. I guess I'll start here.

Found in mozilla-central changeset: 404376:d0d3693d9bef. Built with -fsanitize=float-divide-by-zero,integer-divide-by-zero

objdir-ff-ubsan/dist/include/mozilla/gfx/Matrix.h:708:52: runtime error: division by zero
    #0 0x7f994b3f4039 in mozilla::gfx::Point4DTyped<mozilla::gfx::UnknownUnits, double> mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits>::ProjectPoint<double>(mozilla::gfx::PointTyped<mozilla::gfx::UnknownUnits, double> const&) const src/objdir-ff-ubsan/dist/include/mozilla/gfx/Matrix.h:708:52
    #1 0x7f994b3c89e7 in mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits>::ProjectRectBounds<double>(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&) const src/objdir-ff-ubsan/dist/include/mozilla/gfx/Matrix.h:744:17
    #2 0x7f994b3afe86 in nsDisplayTransform::UntransformRect(nsRect const&, nsRect const&, nsIFrame const*, nsRect*) src/layout/painting/nsDisplayList.cpp:9101:32
    #3 0x7f994af10e23 in mozilla::PresShell::MarkFramesInSubtreeApproximatelyVisible(nsIFrame*, nsRect const&, mozilla::Maybe<nsClassHashtable<nsUint64HashKey, mozilla::gfx::IntRegionTyped<mozilla::CSSPixel> > >&, bool) src/layout/base/PresShell.cpp:6025:15
    #4 0x7f994af10e54 in mozilla::PresShell::MarkFramesInSubtreeApproximatelyVisible(nsIFrame*, nsRect const&, mozilla::Maybe<nsClassHashtable<nsUint64HashKey, mozilla::gfx::IntRegionTyped<mozilla::CSSPixel> > >&, bool) src/layout/base/PresShell.cpp:6032:7
    #5 0x7f994af10e54 in mozilla::PresShell::MarkFramesInSubtreeApproximatelyVisible(nsIFrame*, nsRect const&, mozilla::Maybe<nsClassHashtable<nsUint64HashKey, mozilla::gfx::IntRegionTyped<mozilla::CSSPixel> > >&, bool) src/layout/base/PresShell.cpp:6032:7
    #6 0x7f994af10e54 in mozilla::PresShell::MarkFramesInSubtreeApproximatelyVisible(nsIFrame*, nsRect const&, mozilla::Maybe<nsClassHashtable<nsUint64HashKey, mozilla::gfx::IntRegionTyped<mozilla::CSSPixel> > >&, bool) src/layout/base/PresShell.cpp:6032:7
    #7 0x7f994af10e54 in mozilla::PresShell::MarkFramesInSubtreeApproximatelyVisible(nsIFrame*, nsRect const&, mozilla::Maybe<nsClassHashtable<nsUint64HashKey, mozilla::gfx::IntRegionTyped<mozilla::CSSPixel> > >&, bool) src/layout/base/PresShell.cpp:6032:7
    #8 0x7f994af111fb in mozilla::PresShell::RebuildApproximateFrameVisibility(nsRect*, bool) src/layout/base/PresShell.cpp:6068:3
    #9 0x7f994af112f3 in mozilla::PresShell::DoUpdateApproximateFrameVisibility(bool) src/layout/base/PresShell.cpp:6100:3
    #10 0x7f994af3fb16 in applyImpl<mozilla::PresShell, void (mozilla::PresShell::*)()> src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:1149:12
    #11 0x7f994af3fb16 in apply<mozilla::PresShell, void (mozilla::PresShell::*)()> src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:1155
    #12 0x7f994af3fb16 in mozilla::detail::RunnableMethodImpl<mozilla::PresShell*, void (mozilla::PresShell::*)(), true, (mozilla::RunnableKind)0>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:1200
    #13 0x7f99470270a4 in mozilla::SchedulerGroup::Runnable::Run() src/xpcom/threads/SchedulerGroup.cpp:413:25
    #14 0x7f9947043192 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1040:14
    #15 0x7f994705ef00 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:517:10
    #16 0x7f9947ae967b in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:97:21
    #17 0x7f9947a11389 in RunHandler src/ipc/chromium/src/base/message_loop.cc:319:3
    #18 0x7f9947a11389 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:299
    #19 0x7f994ab294f6 in nsBaseAppShell::Run() src/widget/nsBaseAppShell.cpp:157:27
    #20 0x7f994e964dc4 in XRE_RunAppShell() src/toolkit/xre/nsEmbedFunctions.cpp:892:22
    #21 0x7f9947a11389 in RunHandler src/ipc/chromium/src/base/message_loop.cc:319:3
    #22 0x7f9947a11389 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:299
    #23 0x7f994e9649f0 in XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/nsEmbedFunctions.cpp:718:34
    #24 0x42d23b in content_process_main(mozilla::Bootstrap*, int, char**) src/browser/app/../../ipc/contentproc/plugin-container.cpp:63:30
    #25 0x42d358 in main src/browser/app/nsBrowserApp.cpp:280:18
    #26 0x7f996d5f71c0 in __libc_start_main /build/glibc-itYbWN/glibc-2.26/csu/../csu/libc-start.c:308
    #27 0x407159 in _start (objdir-ff-ubsan/dist/bin/firefox+0x407159)

Comment 1

[Tyson Smith [:tsmith]]

This issue is no longer reproducible with the attached testcase.