1419508 - UBSan: division by zero in [@ mozilla::ContainerState::CreateMaskLayer]

Status: RESOLVED FIXED

(Core :: Web Painting, defect, P3)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

This was found with a Firefox build built with -fsanitize=float-divide-by-zero,integer-divide-by-zero

/layout/painting/FrameLayerBuilder.cpp:6405:39: runtime error: division by zero
    #0 0x7f10dcdbcede in mozilla::ContainerState::CreateMaskLayer(mozilla::layers::Layer*, mozilla::DisplayItemClip const&, mozilla::Maybe<unsigned long> const&, unsigned int) /layout/painting/FrameLayerBuilder.cpp:6405:39
    #1 0x7f10dcdc49ad in mozilla::ContainerState::SetupMaskLayer(mozilla::layers::Layer*, mozilla::DisplayItemClip const&, unsigned int) /layout/painting/FrameLayerBuilder.cpp:6329:5
    #2 0x7f10dcdb5777 in void mozilla::ContainerState::FinishPaintedLayerData<mozilla::PaintedLayerDataNode::PopPaintedLayerData()::$_0>(mozilla::PaintedLayerData&, mozilla::PaintedLayerDataNode::PopPaintedLayerData()::$_0) /layout/painting/FrameLayerBuilder.cpp:3280:5
    #3 0x7f10dcdb4955 in mozilla::PaintedLayerDataNode::PopPaintedLayerData() /layout/painting/FrameLayerBuilder.cpp:2867:21
    #4 0x7f10dcdb46a7 in mozilla::PaintedLayerDataNode::PopAllPaintedLayerData() /layout/painting/FrameLayerBuilder.cpp:2877:5
    #5 0x7f10dcdb4467 in mozilla::PaintedLayerDataNode::Finish(bool) /layout/painting/FrameLayerBuilder.cpp:2831:3
    #6 0x7f10dcdb4646 in mozilla::PaintedLayerDataNode::FinishAllChildren(bool) /layout/painting/FrameLayerBuilder.cpp:2820:19
    #7 0x7f10dcdb445f in mozilla::PaintedLayerDataNode::Finish(bool) /layout/painting/FrameLayerBuilder.cpp:2829:3
    #8 0x7f10dcdb4646 in mozilla::PaintedLayerDataNode::FinishAllChildren(bool) /layout/painting/FrameLayerBuilder.cpp:2820:19
    #9 0x7f10dcdb445f in mozilla::PaintedLayerDataNode::Finish(bool) /layout/painting/FrameLayerBuilder.cpp:2829:3
    #10 0x7f10dcdb6808 in mozilla::PaintedLayerDataTree::Finish() /layout/painting/FrameLayerBuilder.cpp:2891:12
    #11 0x7f10dcdc9dae in mozilla::ContainerState::Finish(unsigned int*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, nsDisplayList*, bool*) /layout/painting/FrameLayerBuilder.cpp:5273:25
    #12 0x7f10dcdcb490 in mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder*, mozilla::layers::LayerManager*, nsIFrame*, nsDisplayItem*, nsDisplayList*, mozilla::ContainerLayerParameters const&, mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits> const*, unsigned int) /layout/painting/FrameLayerBuilder.cpp:5697:11
    #13 0x7f10dce5d262 in nsDisplayTransform::BuildLayer(nsDisplayListBuilder*, mozilla::layers::LayerManager*, mozilla::ContainerLayerParameters const&) /layout/painting/nsDisplayList.cpp:8476:5
    #14 0x7f10dcdc176c in mozilla::ContainerState::ProcessDisplayItems(nsDisplayList*) /layout/painting/FrameLayerBuilder.cpp:4281:38
    #15 0x7f10dcdcb34e in mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder*, mozilla::layers::LayerManager*, nsIFrame*, nsDisplayItem*, nsDisplayList*, mozilla::ContainerLayerParameters const&, mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits> const*, unsigned int) /layout/painting/FrameLayerBuilder.cpp:5683:11
    #16 0x7f10dce5d262 in nsDisplayTransform::BuildLayer(nsDisplayListBuilder*, mozilla::layers::LayerManager*, mozilla::ContainerLayerParameters const&) /layout/painting/nsDisplayList.cpp:8476:5
    #17 0x7f10dcdc176c in mozilla::ContainerState::ProcessDisplayItems(nsDisplayList*) /layout/painting/FrameLayerBuilder.cpp:4281:38
    #18 0x7f10dcdcb34e in mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder*, mozilla::layers::LayerManager*, nsIFrame*, nsDisplayItem*, nsDisplayList*, mozilla::ContainerLayerParameters const&, mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits> const*, unsigned int) /layout/painting/FrameLayerBuilder.cpp:5683:11
    #19 0x7f10dce51144 in nsDisplayOwnLayer::BuildLayer(nsDisplayListBuilder*, mozilla::layers::LayerManager*, mozilla::ContainerLayerParameters const&) /layout/painting/nsDisplayList.cpp:6854:5
    #20 0x7f10dcdc176c in mozilla::ContainerState::ProcessDisplayItems(nsDisplayList*) /layout/painting/FrameLayerBuilder.cpp:4281:38
    #21 0x7f10dcdcb34e in mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder*, mozilla::layers::LayerManager*, nsIFrame*, nsDisplayItem*, nsDisplayList*, mozilla::ContainerLayerParameters const&, mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits> const*, unsigned int) /layout/painting/FrameLayerBuilder.cpp:5683:11
    #22 0x7f10dce2a995 in nsDisplayList::PaintRoot(nsDisplayListBuilder*, gfxContext*, unsigned int) /layout/painting/nsDisplayList.cpp:2507:9
    #23 0x7f10dc735f86 in nsLayoutUtils::PaintFrame(gfxContext*, nsIFrame*, nsRegion const&, unsigned int, nsDisplayListBuilderMode, nsLayoutUtils::PaintFrameFlags) /layout/base/nsLayoutUtils.cpp:3944:12
    #24 0x7f10dc65bb92 in mozilla::PresShell::Paint(nsView*, nsRegion const&, unsigned int) /layout/base/PresShell.cpp:6512:5
    #25 0x7f10dbf4932b in nsViewManager::ProcessPendingUpdatesPaint(nsIWidget*) /view/nsViewManager.cpp:480:19
    #26 0x7f10dbf48a1d in nsViewManager::ProcessPendingUpdatesForView(nsView*, bool) /view/nsViewManager.cpp:412:33
    #27 0x7f10dbf4ac9b in nsViewManager::ProcessPendingUpdates() /view/nsViewManager.cpp:1102:5
    #28 0x7f10dc5da77e in nsRefreshDriver::Tick(long, mozilla::TimeStamp) /layout/base/nsRefreshDriver.cpp:2027:11
    #29 0x7f10dc5e5003 in mozilla::RefreshDriverTimer::TickRefreshDrivers(long, mozilla::TimeStamp, nsTArray<RefPtr<nsRefreshDriver> >&) /layout/base/nsRefreshDriver.cpp:306:7
    #30 0x7f10dc5e4d4c in mozilla::RefreshDriverTimer::Tick(long, mozilla::TimeStamp) /layout/base/nsRefreshDriver.cpp:328:5
    #31 0x7f10dc5e92ca in mozilla::VsyncRefreshDriverTimer::RunRefreshDrivers(mozilla::TimeStamp) /layout/base/nsRefreshDriver.cpp:769:5
    #32 0x7f10dc5e7d20 in mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::TickRefreshDriver(mozilla::TimeStamp) /layout/base/nsRefreshDriver.cpp:682:35
    #33 0x7f10dc5e31dc in mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::ParentProcessVsyncNotifier::Run() /layout/base/nsRefreshDriver.cpp:528:20
    #34 0x7f10d505cdb9 in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1037:14
    #35 0x7f10d5095ed1 in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:513:10
    #36 0x7f10d61c7e31 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:97:21
    #37 0x7f10d6049d50 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:299:3
    #38 0x7f10dbfd70a4 in nsBaseAppShell::Run() /widget/nsBaseAppShell.cpp:159:27
    #39 0x7f10e09268d9 in nsAppStartup::Run() /toolkit/components/startup/nsAppStartup.cpp:288:30
    #40 0x7f10e0aedafb in XREMain::XRE_mainRun() /toolkit/xre/nsAppRunner.cpp:4685:22
    #41 0x7f10e0aef95c in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /toolkit/xre/nsAppRunner.cpp:4847:8
    #42 0x7f10e0af0651 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /toolkit/xre/nsAppRunner.cpp:4942:21
    #43 0x518238 in do_main(int, char**, char**) /browser/app/nsBrowserApp.cpp:231:22
    #44 0x517aba in main /browser/app/nsBrowserApp.cpp:304:16
    #45 0x7f1109f781c0 in __libc_start_main /build/glibc-CxtIbX/glibc-2.26/csu/../csu/libc-start.c:308
    #46 0x420589 in _start (/objdir-ff-ubsan/dist/bin/firefox+0x420589)

Comment 1

[Xidorn Quan [:xidorn] UTC+10]

The corresponding line is https://searchfox.org/mozilla-central/rev/fb5422ff98ba43f3732debd9d1f4dcd3b3a920f6/layout/painting/FrameLayerBuilder.cpp#6405

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1419508 - Return early from CreateMaskLayer if there is no visible data -

Review commit: https://reviewboard.mozilla.org/r/215532/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/215532/

Comment 3

[Matt Woodrow (:mattwoodrow)]

Comment on attachment 8945323 [details]
Bug 1419508 - Return early from CreateMaskLayer if there is no visible data -

https://reviewboard.mozilla.org/r/215532/#review221270

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c5d8b1ed2722
Return early from CreateMaskLayer if there is no visible data - r=mattwoodrow

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/c5d8b1ed2722

Comment 6

[Julien Cristau [:jcristau]]

Should we get this in 59?

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8945323 [details]
Bug 1419508 - Return early from CreateMaskLayer if there is no visible data -

(In reply to Julien Cristau [:jcristau] from comment #6)
> Should we get this in 59?
Sure, it's a trivial patch, worth considering. Thanks for the suggestion.

Approval Request Comment
[Feature/Bug causing the regression]: Layout of sub-atomic (<1 pixel) elements
[User impact if declined]: Unlikely, but possible crashes on bad websites
[Is this code covered by automated tests?]: Not for the early return case
[Has the fix been verified in Nightly?]: Verified locally with PoC test case
[Needs manual test from QE? If yes, steps to reproduce]: Don't think it's worth the time
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: It's another early return from a function, when an element is too small to show up
[String changes made/needed]: None

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8945323 [details]
Bug 1419508 - Return early from CreateMaskLayer if there is no visible data -

Avoiding a potential crash sounds good, let's uplift for beta 10.

Comment 9

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/releases/mozilla-beta/rev/286bf1028f5b