Perma macOS 15 AArch64 Shippable MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime | Expected: (GetKeyFrameCount(output)) >= (r.mInputKeyframes), actual: 3 vs 4
Categories
(Core :: Audio/Video, defect, P4)
Tracking
()
People
(Reporter: intermittent-bug-filer, Assigned: chunmin)
References
Details
(Keywords: intermittent-failure)
Attachments
(3 external files)
Filed by: sstanca [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=570310382&repo=mozilla-central&task=JO5onxcMQjOxkfusphBxZg.0
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JO5onxcMQjOxkfusphBxZg/runs/0/artifacts/public/logs/live_backing.log
[task 2026-06-03T01:06:59.562+00:00] 01:06:59 INFO - TEST-PASS | MediaDataEncoderTest.H264EncodeBatch4KAnnexBRecord | took 841ms
[task 2026-06-03T01:06:59.562+00:00] 01:06:59 INFO - TEST-START | MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime
[task 2026-06-03T01:06:59.993+00:00] 01:06:59 INFO - TEST-UNEXPECTED-FAIL | MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime | Expected: (GetKeyFrameCount(output)) >= (r.mInputKeyframes), actual: 3 vs 4
[task 2026-06-03T01:06:59.993+00:00] 01:06:59 INFO -
[task 2026-06-03T01:06:59.993+00:00] 01:06:59 INFO - ./../../../../../checkouts/gecko/dom/media/gtest/TestMediaDataEncoder.cpp:535
[task 2026-06-03T01:07:00.004+00:00] 01:07:00 INFO - TEST-UNEXPECTED-FAIL | MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime | expected PASS
[task 2026-06-03T01:07:00.004+00:00] 01:07:00 INFO - TEST-INFO took 442ms
[task 2026-06-03T01:07:00.004+00:00] 01:07:00 INFO - TEST-START | MediaDataEncoderTest.H264EncodeBatch4KAVCCRecord
[task 2026-06-03T01:07:00.845+00:00] 01:07:00 INFO - TEST-PASS | MediaDataEncoderTest.H264EncodeBatch4KAVCCRecord | took 841ms
[task 2026-06-03T01:07:00.845+00:00] 01:07:00 INFO - TEST-START | MediaDataEncoderTest.H264EncodeBatch4KAVCCRealtime
[task 2026-06-03T01:07:01.276+00:00] 01:07:01 INFO - TEST-UNEXPECTED-FAIL | MediaDataEncoderTest.H264EncodeBatch4KAVCCRealtime | Expected: (GetKeyFrameCount(output)) >= (r.mInputKeyframes), actual: 3 vs 4
[task 2026-06-03T01:07:01.276+00:00] 01:07:01 INFO -
[task 2026-06-03T01:07:01.276+00:00] 01:07:01 INFO - ./../../../../../checkouts/gecko/dom/media/gtest/TestMediaDataEncoder.cpp:535
[task 2026-06-03T01:07:01.287+00:00] 01:07:01 INFO - TEST-UNEXPECTED-FAIL | MediaDataEncoderTest.H264EncodeBatch4KAVCCRealtime | expected PASS
[task 2026-06-03T01:07:01.287+00:00] 01:07:01 INFO - TEST-INFO took 441ms
[task 2026-06-03T01:07:01.287+00:00] 01:07:01 INFO - TEST-START | MediaDataEncoderTest.H264EncodeAfterDrainAnnexBRecord
[task 2026-06-03T01:07:01.556+00:00] 01:07:01 INFO - TEST-PASS | MediaDataEncoderTest.H264EncodeAfterDrainAnnexBRecord | took 269ms
[task 2026-06-03T01:07:01.556+00:00] 01:07:01 INFO - TEST-START | MediaDataEncoderTest.H264EncodeAfterDrainAnnexBRealtime
Comment 1•5 months ago
|
||
As https://bugzilla.mozilla.org/show_bug.cgi?id=2044018#c1, here it looks like a simmilar case. Jim, do you know someone who could take care of this one? Thank you!
Comment 2•5 months ago
|
||
This is a perma-failing gtest on the macOS 15 AArch64 Shippable config. The test MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime (and its AVCC sibling) feeds the VideoToolbox H.264 encoder a 4K batch containing 4 forced input keyframes and asserts the encoder emits at least that many keyframes (EXPECT_GE(GetKeyFrameCount(output), r.mInputKeyframes) at TestMediaDataEncoder.cpp:535), but in Realtime mode at 4K it emits only 3.
The test already relaxes its output-length check for the Realtime+4K case (lines 529-531: "Realtime encoding may drop frames for large frame sizes" -> EXPECT_LE), but the keyframe-count assertion at line 535 was not relaxed the same way. So this is likely one of: (a) a dropped frame happened to carry a forced keyframe, making the strict >= invariant invalid for the lossy Realtime path -- a test fix; or (b) the VideoToolbox encoder on this OS/arch coalesces or ignores a forced keyframe request at 4K -- an AppleVTEncoder fix. Per the see-also bug 2044018 (a sibling perma-fail on the same new macOS 15 AArch64 config, resolved in Networking), a batch of tests appears to have been unhidden together on this worker config rather than a media-specific regression.
Setting S4/P4: this is a CI/test-infra failure with no user-facing playback impact. Taking this for investigation.
Needinfo'ing the test author, Chun-Min Chang (:chunmin), who added the keyframe-count assertion at TestMediaDataEncoder.cpp:535 and the Realtime+4K output-length relaxation just above it (bug 1986544 / bug 1987888): since the Realtime path can legitimately drop a frame that happens to carry a forced keyframe, should the >= keyframe-count invariant at line 535 be relaxed for the Realtime+4K case the same way the length check was (a test fix), or do you expect VideoToolbox to always honor every forced-keyframe request at 4K (which would point at AppleVTEncoder)?
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
Comment 6•4 months ago
|
||
This is the remaining issue for the macOS AArch64 GTest suite. Chun-Min, could you take a look?
| Comment hidden (Intermittent Failures Robot) |
| Assignee | ||
Comment 8•4 months ago
|
||
When the encoder drops a frame in real-time usage, a frame that was requested as
a keyframe is silently lost from the output, so fewer keyframes than requested
are produced -- a more consequential outcome than an ordinary dropped frame, yet
indistinguishable in the log. The drop callback did not identify which frame was
dropped.
Tag each submitted frame that requested a keyframe so the drop callback can
recognise that case, and note it in the existing drop log.
Updated•4 months ago
|
| Assignee | ||
Comment 9•4 months ago
|
||
In real-time usage the encoder may drop frames for large frame sizes. Submitting
a whole batch of frames at once can make it drop the frame that requested a
keyframe, so the output legitimately contains fewer keyframes than were
requested. Dropping frames in real-time mode is permitted by the WebCodecs
specification and matches other browsers, so requiring every requested keyframe
to survive is not a valid expectation on this path.
The output-length check already tolerated dropped frames for real-time 4K, but
the neighbouring keyframe-count check did not. Relax the keyframe-count check the
same way in the batch test, requiring only that some output was produced and that
the stream still starts with a keyframe. The single-frame tests submit with
enough slack that the requested-keyframe frame is not dropped, so they keep the
strict check.
This fixes the permafailing MediaDataEncoderTest.H264EncodeBatch4KAnnexBRealtime
and H264EncodeBatch4KAVCCRealtime gtests on macosx1500-aarch64-shippable/opt.
| Assignee | ||
Comment 10•4 months ago
|
||
The single-frame and batch H.264 encode tests duplicated the entire post-encode
validation. Keeping the two copies in sync is what let this bug happen: the
real-time 4K carve-out was applied to the frame-count check but not to the
neighbouring keyframe-count check, in either copy.
Move the shared validation into one helper so the expectations live in a single
place and a carve-out cannot be applied to one copy only. The batch caller
tolerates a dropped keyframe on the real-time 4K path; the single-frame caller
keeps the strict check. No behaviour change, and the helper also guards its
output accesses against an empty result.
| Comment hidden (Intermittent Failures Robot) |
Comment 12•4 months ago
|
||
Comment 13•4 months ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/0b01156d16e4
https://hg.mozilla.org/mozilla-central/rev/4343be2833c3
https://hg.mozilla.org/mozilla-central/rev/62077c37e2bd
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
Comment 20•2 months ago
|
||
| uplift | ||
Updated•2 months ago
|
| Comment hidden (Intermittent Failures Robot) |
Description
•