1772640 - src/layout/generic/nsGfxScrollFrame.cpp:1299:23: runtime error: 5.85677e+09 is outside the range of representable values of type 'int'

Status: RESOLVED FIXED

(Core :: Layout: Scrolling and Overflow, defect, P3)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

This was found by enabling the float-cast-overflow check in UBSan and running existing tests. This type of issue can create inconsistencies across platforms, architectures and optimization levels.

Found with m-c 20220603-68727ef04ccf

To enable this check add the following to your mozconfig:

ac_add_options --enable-undefined-sanitizer="float-cast-overflow"

src/layout/generic/nsGfxScrollFrame.cpp:1299:23: runtime error: 5.85677e+09 is outside the range of representable values of type 'int'
    #0 0x7f3841032d70 in GetScrollableOverflowForPerspective(nsIFrame*, nsIFrame*, nsRect, nsPoint, nsRect&) src/layout/generic/nsGfxScrollFrame.cpp:1299:23
    #1 0x7f38410300cd in nsHTMLScrollFrame::AdjustForPerspective(nsRect&) src/layout/generic/nsGfxScrollFrame.cpp:1353:3
    #2 0x7f384102fc29 in nsHTMLScrollFrame::PlaceScrollArea(mozilla::ScrollReflowInput&, nsPoint const&) src/layout/generic/nsGfxScrollFrame.cpp:1074:3
    #3 0x7f3841033596 in nsHTMLScrollFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsGfxScrollFrame.cpp:1412:3
    #4 0x7f3840f89dd1 in nsBlockReflowContext::ReflowBlock(mozilla::LogicalRect const&, bool, nsCollapsingMargin&, int, bool, nsLineBox*, mozilla::ReflowInput&, nsReflowStatus&, mozilla::BlockReflowState&) src/layout/generic/nsBlockReflowContext.cpp:288:11
    #5 0x7f3840f83df2 in nsBlockFrame::ReflowBlockFrame(mozilla::BlockReflowState&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3901:11
    #6 0x7f3840f813b7 in nsBlockFrame::ReflowLine(mozilla::BlockReflowState&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3251:5
    #7 0x7f3840f79698 in nsBlockFrame::ReflowDirtyLines(mozilla::BlockReflowState&) src/layout/generic/nsBlockFrame.cpp:2778:7
    #8 0x7f3840f73e39 in nsBlockFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsBlockFrame.cpp:1410:3
    #9 0x7f3840fa27c7 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1005:14
    #10 0x7f3840fa12d9 in nsCanvasFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsCanvasFrame.cpp:791:7
    #11 0x7f3840fa27c7 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1005:14
    #12 0x7f384102d07d in nsHTMLScrollFrame::ReflowScrolledFrame(mozilla::ScrollReflowInput&, bool, bool, mozilla::ReflowOutput*) src/layout/generic/nsGfxScrollFrame.cpp:838:3
    #13 0x7f384102e398 in nsHTMLScrollFrame::ReflowContents(mozilla::ScrollReflowInput&, mozilla::ReflowOutput const&) src/layout/generic/nsGfxScrollFrame.cpp:974:3
    #14 0x7f3841033370 in nsHTMLScrollFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsGfxScrollFrame.cpp:1395:3
    #15 0x7f3840f66fc2 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, int, int, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1045:14
    #16 0x7f3840f66804 in mozilla::ViewportFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/ViewportFrame.cpp:374:7
    #17 0x7f3840d8fb93 in mozilla::PresShell::DoReflow(nsIFrame*, bool, mozilla::OverflowChangedTracker*) src/layout/base/PresShell.cpp:9618:11
    #18 0x7f3840da10b7 in mozilla::PresShell::ProcessReflowCommands(bool) src/layout/base/PresShell.cpp:9789:24
    #19 0x7f3840d9ff57 in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) src/layout/base/PresShell.cpp:4361:11
    #20 0x7f3840d9edc8 in mozilla::PresShell::FlushPendingNotifications(mozilla::ChangesToFlush) src/objdir-ff-ubsan/dist/include/mozilla/PresShell.h:1448:5
    #21 0x7f3840d9edc8 in mozilla::PresShell::DoFlushPendingNotifications(mozilla::FlushType) src/layout/base/PresShell.cpp:4155:3
    #22 0x7f383b5d60aa in mozilla::PresShell::FlushPendingNotifications(mozilla::FlushType) src/objdir-ff-ubsan/dist/include/mozilla/PresShell.h:1439:5
    #23 0x7f383db5b4c6 in mozilla::EventStateManager::FlushLayout(nsPresContext*) src/dom/events/EventStateManager.cpp:5928:16
    #24 0x7f383db55da6 in mozilla::EventStateManager::PreHandleEvent(nsPresContext*, mozilla::WidgetEvent*, nsIFrame*, nsIContent*, nsEventStatus*, nsIContent*) src/dom/events/EventStateManager.cpp:780:7
    #25 0x7f3840dbc022 in mozilla::PresShell::EventHandler::DispatchEvent(mozilla::EventStateManager*, mozilla::WidgetEvent*, bool, nsEventStatus*, nsIContent*) src/layout/base/PresShell.cpp:8228:39
    #26 0x7f3840db6b63 in mozilla::PresShell::EventHandler::HandleEventWithCurrentEventInfo(mozilla::WidgetEvent*, nsEventStatus*, bool, nsIContent*) src/layout/base/PresShell.cpp:8197:17
    #27 0x7f3840db5f6b in mozilla::PresShell::EventHandler::HandleEventUsingCoordinates(nsIFrame*, mozilla::WidgetGUIEvent*, nsEventStatus*, bool) src/layout/base/PresShell.cpp:7124:30
    #28 0x7f3840db4471 in mozilla::PresShell::EventHandler::HandleEvent(nsIFrame*, mozilla::WidgetGUIEvent*, bool, nsEventStatus*) src/layout/base/PresShell.cpp:6927:12
    #29 0x7f3840db302b in mozilla::PresShell::HandleEvent(nsIFrame*, mozilla::WidgetGUIEvent*, bool, nsEventStatus*) src/layout/base/PresShell.cpp:6870:23
    #30 0x7f38406ace72 in nsViewManager::DispatchEvent(mozilla::WidgetGUIEvent*, nsView*, nsEventStatus*) src/view/nsViewManager.cpp:685:18
    #31 0x7f38406aca75 in nsView::HandleEvent(mozilla::WidgetGUIEvent*, bool) src/view/nsView.cpp:1129:9
    #32 0x7f384072fe16 in mozilla::widget::PuppetWidget::DispatchEvent(mozilla::WidgetGUIEvent*, nsEventStatus&) src/widget/PuppetWidget.cpp:355:37
    #33 0x7f383a793941 in mozilla::layers::APZCCallbackHelper::DispatchWidgetEvent(mozilla::WidgetGUIEvent&) src/gfx/layers/apz/util/APZCCallbackHelper.cpp:502:21
    #34 0x7f383f9e8608 in mozilla::dom::BrowserChild::DispatchWidgetEventViaAPZ(mozilla::WidgetGUIEvent&) src/dom/ipc/BrowserChild.cpp:1786:10
    #35 0x7f383f9e8608 in mozilla::dom::BrowserChild::HandleRealMouseButtonEvent(mozilla::WidgetMouseEvent const&, mozilla::layers::ScrollableLayerGuid const&, unsigned long const&) src/dom/ipc/BrowserChild.cpp:1749:3
    #36 0x7f383f9ea6d8 in mozilla::dom::BrowserChild::RecvRealMouseButtonEvent(mozilla::WidgetMouseEvent const&, mozilla::layers::ScrollableLayerGuid const&, unsigned long const&) src/dom/ipc/BrowserChild.cpp:1716:3
    #37 0x7f383f9ea98c in mozilla::dom::BrowserChild::RecvSynthMouseMoveEvent(mozilla::WidgetMouseEvent const&, mozilla::layers::ScrollableLayerGuid const&, unsigned long const&) src/dom/ipc/BrowserChild.cpp:1681:8
    #38 0x7f383fb98e8d in mozilla::dom::PBrowserChild::OnMessageReceived(IPC::Message const&) src/objdir-ff-ubsan/ipc/ipdl/PBrowserChild.cpp:5645:80
    #39 0x7f383fc4277f in mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) src/objdir-ff-ubsan/ipc/ipdl/PContentChild.cpp:8474:32
    #40 0x7f383f9902e7 in mozilla::dom::ContentChild::OnMessageReceived(IPC::Message const&) src/dom/ipc/ContentChild.cpp:3669:25
    #41 0x7f38399c3ab8 in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) src/ipc/glue/MessageChannel.cpp:1781:25
    #42 0x7f38399c0c7b in mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message, mozilla::DefaultDelete<IPC::Message> >) src/ipc/glue/MessageChannel.cpp:1706:9
    #43 0x7f38399c178e in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::ipc::MessageChannel::MessageTask&) src/ipc/glue/MessageChannel.cpp:1506:3
    #44 0x7f38399c27b9 in mozilla::ipc::MessageChannel::MessageTask::Run() src/ipc/glue/MessageChannel.cpp:1604:14
    #45 0x7f38382c0aba in mozilla::RunnableTask::Run() src/xpcom/threads/TaskController.cpp:475:16
    #46 0x7f3838282911 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:788:26
    #47 0x7f383828018e in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:620:15
    #48 0x7f38382808fb in mozilla::TaskController::ProcessPendingMTTask(bool) src/xpcom/threads/TaskController.cpp:398:36
    #49 0x7f38382b25d1 in mozilla::TaskController::InitializeInternal()::$_0::operator()() const src/xpcom/threads/TaskController.cpp:124:37
    #50 0x7f38382b25d1 in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_0>::Run() src/objdir-ff-ubsan/dist/include/nsThreadUtils.h:531:5
    #51 0x7f383829ea93 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1180:16
    #52 0x7f38382a6dd4 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:465:10
    #53 0x7f38399cadf2 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:85:21
    #54 0x7f38399cc3d2 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:268:30
    #55 0x7f383983f411 in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:380:10
    #56 0x7f383983f411 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:373:3
    #57 0x7f383983f411 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:355:3
    #58 0x7f38407b5228 in nsBaseAppShell::Run() src/widget/nsBaseAppShell.cpp:137:27
    #59 0x7f38456ae747 in XRE_RunAppShell() src/toolkit/xre/nsEmbedFunctions.cpp:875:20
    #60 0x7f38399cc3b1 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:235:9
    #61 0x7f383983f411 in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:380:10
    #62 0x7f383983f411 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:373:3
    #63 0x7f383983f411 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:355:3
    #64 0x7f38456ad818 in XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/nsEmbedFunctions.cpp:734:34
    #65 0x7f38456c2840 in mozilla::BootstrapImpl::XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/Bootstrap.cpp:67:12
    #66 0x55a340ff5cc5 in content_process_main(mozilla::Bootstrap*, int, char**) src/browser/app/../../ipc/contentproc/plugin-container.cpp:57:28
    #67 0x55a340ff60d5 in main src/browser/app/nsBrowserApp.cpp:338:18
    #68 0x7f3862704c86 in __libc_start_main /build/glibc-uZu3wS/glibc-2.27/csu/../csu/libc-start.c:310
    #69 0x55a340f360a8 in _start (src/objdir-ff-ubsan/dist/bin/firefox+0xf80a8) (BuildId: 71e47150e678236118ddf6c7503875d09a3e9c8a)

Comment 1

[Hiroyuki Ikezoe (:hiro)]

The calculation here in question is overhang.left /= rightDelta. It looks to me we can just use NSCoordSaturatingNonnegativeMultiply rather than dividing by a very small double value.

Comment 2

[Tyson Smith [:tsmith]]

This issue is currently triggered while fuzzing with the 'float-cast-overflow' UBSan check enabled. This issue will need to be addressed before the check can be enabled by default.

If it requires too much effort to fix immediately please ni? me and let me know. If necessary it will be added to a suppression list. Thank you :)

Comment 3

[Hiroyuki Ikezoe (:hiro)]

Attachment: Bug 1772640 - Use NSCoordSaturatingMultiply to avoid overflowing nscoord. r?tnikkel

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5db9af7d249e
Use NSCoordSaturatingMultiply to avoid overflowing nscoord. r=tnikkel

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/34515 for changes under testing/web-platform/tests

Comment 6

[bszekely]

https://hg.mozilla.org/mozilla-central/rev/5db9af7d249e

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot