1490704 - Assertion failure: aOutSlice.LeftRight() <= minSize.width, at /builds/worker/workspace/build/src/gfx/thebes/gfxBlur.cpp:494

Status: RESOLVED FIXED

(Core :: Graphics, defect, P3)

Description

[Jason Kratzer [:jkratzer]]

Attachment: testcase.html

Testcase found while fuzzing mozilla-central rev 703546ab6d0c.

Assertion failure: aOutSlice.LeftRight() <= minSize.width, at /builds/worker/workspace/build/src/gfx/thebes/gfxBlur.cpp:494

rax = 0x0000000000000000   rdx = 0x0000000000000000
rcx = 0x0000000000000b40   rbx = 0x0000000000000000
rsi = 0x00007f7ddf70b8b0   rdi = 0x00007f7ddf70a680
rbp = 0x00007ffc7ebcb940   rsp = 0x00007ffc7ebcb780
r8 = 0x00007f7ddf70b8b0    r9 = 0x00007f7de0883740
r10 = 0x00000000ffffffc3   r11 = 0x0000000000000000
r12 = 0x00007ffc7ebcb8a0   r13 = 0x00007f7dc56c6400
r14 = 0x00007ffc7ebcb9c0   r15 = 0x00007ffc7ebcb9d0
rip = 0x00007f7dced175ca
OS|Linux|0.0.0 Linux 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64
CPU|amd64|family 6 model 78 stepping 3|1
GPU|||
Crash|SIGSEGV /SEGV_MAPERR|0x0|0
0|0|libxul.so|gfxAlphaBoxBlur::BlurRectangle(gfxContext*, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&, mozilla::gfx::RectCornerRadii const*, mozilla::gfx::PointTyped<mozilla::gfx::UnknownUnits, double> const&, mozilla::gfx::Color const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&)|hg:hg.mozilla.org/mozilla-central:gfx/thebes/gfxBlur.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|495|0x1c
0|1|libxul.so|nsContextBoxBlur::BlurRectangle(gfxContext*, nsRect const&, int, mozilla::gfx::RectCornerRadii*, int, mozilla::gfx::Color const&, nsRect const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, double> const&)|hg:hg.mozilla.org/mozilla-central:layout/painting/nsCSSRendering.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|4922|0x24
0|2|libxul.so|nsCSSRendering::PaintBoxShadowOuter(nsPresContext*, gfxContext&, nsIFrame*, nsRect const&, nsRect const&, float)|hg:hg.mozilla.org/mozilla-central:layout/painting/nsCSSRendering.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|1868|0x5
0|3|libxul.so|nsDisplayBoxShadowOuter::Paint(nsDisplayListBuilder*, gfxContext*)|hg:hg.mozilla.org/mozilla-central:layout/painting/nsDisplayList.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|5624|0x19
0|4|libxul.so|mozilla::FrameLayerBuilder::PaintItems(std::vector<mozilla::AssignedDisplayItem, std::allocator<mozilla::AssignedDisplayItem> >&, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, gfxContext*, nsDisplayListBuilder*, nsPresContext*, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits> const&, float, float)|hg:hg.mozilla.org/mozilla-central:layout/painting/FrameLayerBuilder.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|7442|0x1a
0|5|libxul.so|mozilla::FrameLayerBuilder::DrawPaintedLayer(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*)|hg:hg.mozilla.org/mozilla-central:layout/painting/FrameLayerBuilder.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|7622|0x18
0|6|libxul.so|mozilla::layers::ClientPaintedLayer::RenderLayerWithReadback(mozilla::layers::ReadbackProcessor*)|hg:hg.mozilla.org/mozilla-central:gfx/layers/client/ClientPaintedLayer.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|166|0x2d
0|7|libxul.so|mozilla::layers::ClientContainerLayer::RenderLayer()|hg:hg.mozilla.org/mozilla-central:gfx/layers/client/ClientContainerLayer.h:703546ab6d0cb643028a1ab4fda997b38f38a2e6|58|0xd
0|8|libxul.so|mozilla::layers::ClientLayerManager::EndTransactionInternal(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags)|hg:hg.mozilla.org/mozilla-central:gfx/layers/client/ClientLayerManager.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|339|0xa
0|9|libxul.so|mozilla::layers::ClientLayerManager::EndTransaction(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags)|hg:hg.mozilla.org/mozilla-central:gfx/layers/client/ClientLayerManager.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|397|0x11
0|10|libxul.so|nsDisplayList::PaintRoot(nsDisplayListBuilder*, gfxContext*, unsigned int)|hg:hg.mozilla.org/mozilla-central:layout/painting/nsDisplayList.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2836|0x17
0|11|libxul.so|nsLayoutUtils::PaintFrame(gfxContext*, nsIFrame*, nsRegion const&, unsigned int, nsDisplayListBuilderMode, nsLayoutUtils::PaintFrameFlags)|hg:hg.mozilla.org/mozilla-central:layout/base/nsLayoutUtils.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|3832|0x5
0|12|libxul.so|mozilla::PresShell::Paint(nsView*, nsRegion const&, unsigned int)|hg:hg.mozilla.org/mozilla-central:layout/base/PresShell.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|6348|0x17
0|13|libxul.so|nsViewManager::ProcessPendingUpdatesPaint(nsIWidget*)|hg:hg.mozilla.org/mozilla-central:view/nsViewManager.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|480|0x28
0|14|libxul.so|nsViewManager::ProcessPendingUpdatesForView(nsView*, bool)|hg:hg.mozilla.org/mozilla-central:view/nsViewManager.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|412|0xd
0|15|libxul.so|nsViewManager::ProcessPendingUpdates()|hg:hg.mozilla.org/mozilla-central:view/nsViewManager.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|1102|0x11
0|16|libxul.so|nsRefreshDriver::Tick(mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2046|0x8
0|17|libxul.so|mozilla::RefreshDriverTimer::TickRefreshDrivers(mozilla::TimeStamp, nsTArray<RefPtr<nsRefreshDriver> >&)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|325|0x8
0|18|libxul.so|mozilla::RefreshDriverTimer::Tick(mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|318|0xc
0|19|libxul.so|mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::TickRefreshDriver(mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|756|0xc
0|20|libxul.so|mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::NotifyVsync(mozilla::TimeStamp)|hg:hg.mozilla.org/mozilla-central:layout/base/nsRefreshDriver.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|572|0xc
0|21|libxul.so|mozilla::layout::VsyncChild::RecvNotify(mozilla::TimeStamp const&)|hg:hg.mozilla.org/mozilla-central:layout/ipc/VsyncChild.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|78|0x9
0|22|libxul.so|mozilla::layout::PVsyncChild::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:2c36fa176485b987fd1c1ce548d1f34c4c8bfdea36ff5dd016400feb13d3c5c0c7f99d5a56d13733937c9483a48617af010c09f521533a5ce0fc1f74c50b86a2/ipc/ipdl/PVsyncChild.cpp:|167|0xc
0|23|libxul.so|mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2248|0x6
0|24|libxul.so|mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2175|0xb
0|25|libxul.so|mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2012|0xb
0|26|libxul.so|mozilla::ipc::MessageChannel::MessageTask::Run()|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|2045|0xc
0|27|libxul.so|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|1161|0x15
0|28|libxul.so|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|519|0x11
0|29|libxul.so|mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|125|0xd
0|30|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:703546ab6d0cb643028a1ab4fda997b38f38a2e6|325|0x17
0|31|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:703546ab6d0cb643028a1ab4fda997b38f38a2e6|318|0x8
0|32|libxul.so|nsBaseAppShell::Run()|hg:hg.mozilla.org/mozilla-central:widget/nsBaseAppShell.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|158|0xd
0|33|libxul.so|XRE_RunAppShell()|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|944|0x11
0|34|libxul.so|mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|269|0x5
0|35|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:703546ab6d0cb643028a1ab4fda997b38f38a2e6|325|0x17
0|36|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:703546ab6d0cb643028a1ab4fda997b38f38a2e6|318|0x8
0|37|libxul.so|XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|770|0x8
0|38|firefox-bin|content_process_main(mozilla::Bootstrap*, int, char**)|hg:hg.mozilla.org/mozilla-central:ipc/contentproc/plugin-container.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|50|0x14
0|39|firefox-bin|main|hg:hg.mozilla.org/mozilla-central:browser/app/nsBrowserApp.cpp:703546ab6d0cb643028a1ab4fda997b38f38a2e6|287|0x11
0|40|libc-2.27.so||||0x21b97
0|41|firefox-bin|MOZ_ReportAssertionFailure|hg:hg.mozilla.org/mozilla-central:mfbt/Assertions.h:703546ab6d0cb643028a1ab4fda997b38f38a2e6|164|0x5

Comment 1

[Ryan Hunt [:rhunt]]

Lee is this a similar issue to bug 1486810?

Comment 2

[Lee Salzman [:lsalzman]]

Attachment: check rect size in RectIsInt32Safe

Bug 1474722 introduced RectIsInt32Safe, but at the same time removed the checks of rect size against max texture size at the call site. This allows the width to overflow when it gets converted to IntRect. We need to validate the size will also fit into an int32_t.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/329adcc056b4
check rect size in RectIsInt32Safe. r=mattwoodrow

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/329adcc056b4

Comment 5

[Julien Cristau [:jcristau]]

Is this worth uplifting to beta64?

Comment 6

[Lee Salzman [:lsalzman]]

(In reply to Julien Cristau [:jcristau] from comment #5)
> Is this worth uplifting to beta64?

The only seeming effect this has is to trigger the assertion in debug builds.

In release builds, things are fine. The assertion is skipped, but because the rectangle then has a negative size, the box shadow code treats it as empty and returns rather than trying to process it. That is the desired outcome, preventing any overflows from happening.

So I don't see a pressing reason to uplift this. Riding the trains should be fine.

Comment 7

[Julien Cristau [:jcristau]]

Sounds good.  Thanks!