Open Bug 2031056 Opened 7 months ago Updated 1 month ago

WebCodecs H.264 VideoEncoder malformed AVCC SPS

Categories

(Core :: Audio/Video: Web Codecs, defect, P2)

Firefox 149
defect

Tracking

()

People

(Reporter: awathefox, Unassigned, NeedInfo)

References

Details

(Keywords: good-first-bug)

Attachments

(2 files, 1 external file)

Attached file test.html —

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0

Steps to reproduce:

  1. Download test.html
  2. Click 'Run repro'

Actual results:

On Firefox 149.0.2 it outputs:

Encoding one frame with codec=avc1.64001f...
Encoded chunk type: key
decoderConfig.description bytes: 42
decoderConfig.description (AVCC hex):
01 64 00 0d 03 01 00 1a 67 67 64 00 0d ac 2c ac 14 1f a1 00 00 03 00 01 00 00 03 00 3c 0d a0 88 45 38 01 00 05 68 68 ce 3c 30
SPS[0] (26 bytes):
67 67 64 00 0d ac 2c ac 14 1f a1 00 00 03 00 01 00 00 03 00 3c 0d a0 88 45 38
Done.

Obviously, the AVCC SPS header is malformed.

ffmpeg (mp4 as container) usually outputs following with described header:
[h264 @ 0x590979427f80] crop values invalid 0 159 21 0 / 352 32
Last message repeated 2 times
[h264 @ 0x590979427f80] SPS decoding failure, trying again after escaping the NAL
[h264 @ 0x590979427f80] crop values invalid 0 159 21 0 / 352 32
Last message repeated 2 times
[h264 @ 0x590979427f80] sps_id 1 out of range
[h264 @ 0x590979427f80] SPS decoding failure, trying again after escaping the NAL
[h264 @ 0x590979427f80] sps_id 1 out of range
[h264 @ 0x590979427f80] crop values invalid 0 159 21 0 / 352 32
Last message repeated 2 times
[h264 @ 0x590979427f80] SPS decoding failure, trying again after escaping the NAL
[h264 @ 0x590979427f80] crop values invalid 0 159 21 0 / 352 32
Last message repeated 2 times
[h264 @ 0x590979427f80] sps_id 1 out of range
[h264 @ 0x590979427f80] SPS decoding failure, trying again after escaping the NAL
[h264 @ 0x590979427f80] sps_id 1 out of range

It seems that there is a duplicated 0x67 NAL header & invalid reserved bits.

Expected results:

On Chromium 145.0.7632.117:
Encoding one frame with codec=avc1.64001f...
Encoded chunk type: key
decoderConfig.description bytes: 41
decoderConfig.description (AVCC hex):
01 64 00 14 ff e1 00 16 67 64 00 14 ac 2b 40 a0 fd 80 88 00 00 1f 40 00 07 53 00 78 e1 15 01 00 04 68 ee 3c b0 fd f8 f8 00
SPS[0] (22 bytes):
67 64 00 14 ac 2b 40 a0 fd 80 88 00 00 1f 40 00 07 53 00 78 e1 15
Done.

Which is seemingly valid (didn't cause errors on ffmpeg).

Please note that Firefox and Chromium selected different profile level, therefore the size of SPS description may vary.

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Web Codecs' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Audio/Video: Web Codecs
Product: Firefox → Core
  • Summary: WebCodecs H.264 VideoEncoder malformed AVCC SPS
  • Reporter: awathefox@gmail.com
  • Status: UNCONFIRMED
  • Product: Core
  • Component: Audio/Video: Web Codecs
  • Created: 2026-04-11 (2 days ago)
  • Version: Firefox 149
  • Platform/OS: Unspecified (reporter on Windows 10 x64)

Research Summary and Key Findings

The reporter demonstrates that Firefox 149.0.2's WebCodecs VideoEncoder produces a malformed AVC Decoder Configuration Record (AVCC) for H.264 encoding. The decoderConfig.description returned in the encoder output callback contains three specification violations:

Firefox AVCC hex output:

01 64 00 0d [03] [01] 00 1a [67 67] 64 00 0d ac 2c ac 14 1f a1 ... 01 00 05 [68 68] ce 3c 30
Byte offset Firefox value Correct value Violation
[4] (lengthSizeMinusOne) 0x03 0xFF Upper 6 reserved bits must be 0b111111; Firefox outputs 0b000000
[5] (numSPS) 0x01 0xE1 Upper 3 reserved bits must be 0b111; Firefox outputs 0b000
SPS NALU start 67 67 64... 67 64... NAL unit type byte 0x67 duplicated
PPS NALU start 68 68 ce... 68 ce... NAL unit type byte 0x68 duplicated

The ffmpeg decoder fails with SPS decoding failure and sps_id out of range errors on this output. Chromium 145 produces a correctly-formed AVCC for the same codec configuration. A repro test.html is attached to the bug.

alice0775@gmail.com (a known-accurate external QA contributor) was added to CC but has not yet commented.

No duplicate bugs found — this appears to be a unique report with no prior tracking.

Regression Timeline

Not a regression. This appears to be a long-standing implementation bug in AVCC construction that has gone unnoticed because Firefox's own AVCC parser (AVCCConfig::Parse) is lenient about reserved bits and silently accepts the duplicated NAL type byte.

Assessment

  • Suggested Severity: S2
  • Suggested Priority: P2

Assessment Reasoning

The WebCodecs VideoEncoder for H.264 produces structurally invalid AVCC output that causes any spec-compliant decoder (ffmpeg, hardware decoders, other browsers) to fail. This makes H.264 WebCodecs encoding entirely unusable for interoperability — any pipeline that stores or transmits the decoderConfig.description for later playback outside Firefox will produce decode failures. There is no workaround available to web developers without post-processing the output.

The bug affects all platforms where Firefox uses H264::WriteExtraData: Linux (FFmpegVideoEncoder via libx264) and Windows (WMFMediaDataEncoder via AnnexB::ExtractExtraDataForAVCC). macOS (AppleVTEncoder) uses a separate code path and is likely unaffected.

S2 is appropriate because this is a major WebCodecs API feature rendered non-functional for interoperability. P2 reflects that it should be addressed in a near-term release cycle.

Codebase Investigation

Root Cause

The bug is in H264::WriteExtraData in H264.cpp, and is triggered inconsistently by its callers:

// H264.cpp:1314–1335
void H264::WriteExtraData(..., const Span<const uint8_t> aSPS, const Span<const uint8_t> aPPS) {
  aDestExtraData->AppendElement(1);
  aDestExtraData->AppendElement(aProfile);
  aDestExtraData->AppendElement(aConstraints);
  aDestExtraData->AppendElement(aLevel);
  aDestExtraData->AppendElement(3);  // BUG 1: should be 0xFF (reserved 0b111111 | 3)
  aDestExtraData->AppendElement(1);  // BUG 2: should be 0xE1 (reserved 0b111 | 1); comment says "numPPS" but this is numSPS
  uint8_t c[2];
  mozilla::BigEndian::writeUint16(&c[0], aSPS.Length() + 1);
  aDestExtraData->AppendElements(c, 2);
  aDestExtraData->AppendElement((0x00 << 7) | (0x3 << 5) | H264_NAL_SPS);  // prepends 0x67
  aDestExtraData->AppendElements(aSPS.Elements(), aSPS.Length());           // BUG 3 (caller side): if aSPS already starts with 0x67, this duplicates it
  ...
}

Bug 1 and 2 are in WriteExtraData itself: the lengthSizeMinusOne and numSPS bytes have incorrect reserved bits.

Bug 3 is a caller/contract mismatch: WriteExtraData prepends the NAL type bytes (0x67 for SPS, 0x68 for PPS) and accounts for them in the length (Length() + 1). Two of the three callers incorrectly pass NAL data with the NAL type byte already included, producing duplicate NAL headers:

Caller Passes NAL type byte? Result
H264::CreateExtraData No (SPS RBSP only) Correct ✓
AnnexB::ExtractExtraDataForAVCC Yes (full NALU with 0x67) Duplicate NAL byte ✗
FFmpegVideoEncoder::GetExtraData Yes (full NALU with 0x67) Duplicate NAL byte ✗

The AnnexB::ExtractExtraDataForAVCC path is used on Windows (via WMFMediaDataEncoder → ParseH264Parameters → AnnexB::ExtractExtraDataForAVCC). The FFmpegVideoEncoder::GetExtraData path is used on Linux with libx264 in AVCC mode (annexb=0).

The existing gtest AnnexBExtractExtraDataForAVCC in TestByteStreams.cpp:912 does not catch this because AVCCConfig::Parse discards reserved bits and treats the duplicated NAL type byte as opaque NALU data (the first byte still reads as H264_NAL_SPS = 7).

Relevant Files Examined

Findings

The AVCC format (ISO 14496-15) specifies reserved bits of 0b111111 and 0b111 in the lengthSizeMinusOne and numSPS bytes respectively. WriteExtraData writes 0x03 and 0x01 instead of 0xFF and 0xE1. These are straightforward bugs.

The NAL type duplication stems from a contract inconsistency: WriteExtraData is designed to receive raw SPS/PPS RBSP data and prepends the NAL type bytes itself, but the two non-CreateExtraData callers pass full NAL units (type byte included) extracted from Annex B or AVCC-format bitstreams.

Suggested Fix

Fix 1 — H264::WriteExtraData reserved bits (H264.cpp:1322–1323):

// Before:
aDestExtraData->AppendElement(3);  // nalLENSize-1
aDestExtraData->AppendElement(1);  // numPPS
// After:
aDestExtraData->AppendElement(0xFF);  // reserved (6 bits) | nalLengthSizeMinusOne (= 3)
aDestExtraData->AppendElement(0xE1);  // reserved (3 bits) | numSPS (= 1)

Fix 2 — Caller AnnexB::ExtractExtraDataForAVCC (AnnexB.cpp:483):

// Before:
H264::WriteExtraData(avcc, sps[1], sps[2], sps[3], sps, pps);
// After (strip the NAL type byte at index 0, already prepended by WriteExtraData):
H264::WriteExtraData(avcc, sps[1], sps[2], sps[3], sps.From(1), pps.From(1));

Fix 3 — Caller FFmpegVideoEncoder::GetExtraData (FFmpegVideoEncoder.cpp:925):

// Before:
H264::WriteExtraData(extraData, spsData[1], spsData[2], spsData[3], spsData, ppsData);
// After:
H264::WriteExtraData(extraData, spsData[1], spsData[2], spsData[3], spsData.From(1), ppsData.From(1));

The existing test at TestByteStreams.cpp:912 should be updated to validate the raw bytes of the AVCC output (reserved bits and absence of duplicate NAL type bytes) rather than relying solely on AVCCConfig::Parse which is too lenient to catch these bugs.

Good First Bug Assessment

This is a reasonable candidate for a good first bug: the root cause is clearly identified, the fix is surgical (3 one-line changes plus a test update), and the scope is narrow. A developer familiar with C++ but new to the media codebase could apply it with good confidence. Consider adding the good-first-bug keyword after confirming the fix approach.

Severity: -- → S3
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(cchang)
Keywords: good-first-bug
Priority: -- → P2

I would like to resolve that good first bug, at least on Linux. Please assign me to it.

The strict minimum update that makes the attached page test.html run properly is the following:

`--- a/dom/media/platforms/ffmpeg/FFmpegVideoEncoder.cpp
+++ b/dom/media/platforms/ffmpeg/FFmpegVideoEncoder.cpp
@@ -922,8 +922,8 @@ FFmpegVideoEncoder<LIBAV_VER>::GetExtraData(AVPacket* aPacket) {

// Create extra data.
auto extraData = MakeRefPtr<MediaByteBuffer>();

  • H264::WriteExtraData(extraData, spsData[1], spsData[2], spsData[3], spsData,
  •                   ppsData);
    
  • H264::WriteExtraData(extraData, spsData[1], spsData[2], spsData[3], spsData.From(1),
  •                   ppsData.From(1));
    
    MOZ_ASSERT(extraData);
    return extraData.forget();
    }`

I actually prefer not to resolve the issue by committing that change. The code has inconsistencies, as Jim indicated, and one that relates to the heart of this bug is the way SPS and PPS data is packaged as arguments in calls to method H264::WriteExtraData. Sometimes it starts with its corresponding NAL type byte (0x67 or 0x68), and sometimes it starts without one. I prefer the presence of a NAL type byte for the sake of better cohesion. Such a byte is a tag that identifies the nature of the remaining bytes in the aggregate variable. Following that preference, method H264::WriteExtraData is simplified as shown below and there is no more incentive to verify the doubling of NAL type bytes, since that way the method no longer introduces neither of those bytes:

`--- a/dom/media/platforms/agnostic/bytestreams/H264.cpp
+++ b/dom/media/platforms/agnostic/bytestreams/H264.cpp
@@ -1353,15 +1353,13 @@ void H264::WriteExtraData(MediaByteBuffer* aDestExtraData,
aDestExtraData->AppendElement(3); // nalLENSize-1
aDestExtraData->AppendElement(1); // numPPS
uint8_t c[2];

  • mozilla::BigEndian::writeUint16(&c[0], aSPS.Length() + 1);
  • mozilla::BigEndian::writeUint16(&c[0], aSPS.Length());
    aDestExtraData->AppendElements(c, 2);
  • aDestExtraData->AppendElement((0x00 << 7) | (0x3 << 5) | H264_NAL_SPS);
    aDestExtraData->AppendElements(aSPS.Elements(), aSPS.Length());

    aDestExtraData->AppendElement(1); // numPPS

  • mozilla::BigEndian::writeUint16(&c[0], aPPS.Length() + 1);

  • mozilla::BigEndian::writeUint16(&c[0], aPPS.Length());
    aDestExtraData->AppendElements(c, 2);
  • aDestExtraData->AppendElement((0x00 << 7) | (0x3 << 5) | H264_NAL_PPS);
    aDestExtraData->AppendElements(aPPS.Elements(), aPPS.Length());
    }`

The errors related to the padding of nalLengthSizeMinusOne and numSPS data in method H264::WriteExtraData are easy to catch and should be resolved along with this issue.

See Also: → 2082867
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: