2013957 - VideoFrame::Format::ByteCount might overflow on 32-bit machines

Status: NEW

(Core :: Audio/Video: Web Codecs, defect, P1)

Description

[C.M.Chang[:chunmin]]

size_t on 32-bit machines are 32-bit.

For RGBA format with modest dimensions like 32768 × 32768, the VideoFrame::Format::ByteCount,

ByteCount = 32,768 × 32,768 × 4
         = 1,073,741,824 × 4
         = 4,294,967,296

the ByteCount: 4,294,967,296 > UINT32_MAX (4,294,967,295) may overflow. Hence we should validate the CheckedInt<size_t> bytes before returning its value.

Comment 1

[C.M.Chang[:chunmin]]

Attachment: Bug 2013957 - Add crashtest

Comment 2

[C.M.Chang[:chunmin]]

Attachment: Bug 2013957 - Fix CheckedInt overflow in VideoFrame::Format::ByteCount

Add isValid() check before returning CheckedInt value to detect overflow.
Change return type to Result<size_t, MediaResult> so callsites handle
the error gracefully instead of crashing.

Comment 3

[C.M.Chang[:chunmin]]

Attachment: Bug 2013957 - Add defensive overflow checks to buffer reader plane accessors

Check isValid() on CheckedInt before calling value() in DataU(), DataV(),
DataA(), and DataUV(). While not exploitable via JS API (protected by
ByteCount check in caller), YByteSize could theoretically overflow on
32-bit systems. This is a defensive fix for code correctness.

Comment 4

[C.M.Chang[:chunmin]]

Attachment: Bug 2013957 - Add defensive isValid() checks to ValidateVisibility

Adds isValid() checks before calling value() on CheckedInt in
ValidateVisibility. While overflow cannot occur mathematically
(INT32_MAX + INT32_MAX < UINT32_MAX), this follows the same defensive
pattern as the ByteCount and buffer reader fixes.