VP8TrackEncoder per-frame public durations do not sum to the encoded track length
Categories
(Core :: Audio/Video: Recording, defect, P3)
Tracking
()
People
(Reporter: chunmin, Unassigned)
References
Details
Attachments
(1 file)
VP8TrackEncoder (the VP8 video encoder that MediaRecorder uses to produce WebM) assigns every encoded frame a public duration in microseconds. The internal video timeline is measured in 90 kHz track ticks (the VIDEO_TRACK_RATE constant), so each frame boundary lands on an integer number of ticks and the encoder must convert those ticks to microseconds when it publishes a frame.
The conversion is done incorrectly: the encoder derives each frame's public duration from that frame's own tick delta and converts the delta to microseconds, then relies on the sum of those per-frame microsecond durations to describe the whole track. Because 90 kHz ticks do not divide evenly into microseconds, converting-then-summing per-frame deltas is not equal to converting the cumulative tick endpoint once. The published durations therefore lose up to a microsecond per boundary and no longer sum to the microsecond length of the encoded range.
This is a defect in shipping code (dom/media/encoder/VP8TrackEncoder.cpp); it is independent of, and survives, the fix for bug 2053103. Bug 2053103 concerns a raw TimeDuration-to-track-tick truncation that drops the first frame; this bug concerns how already-encoded frames compose their public microsecond durations.
Steps to reproduce:
- Apply bug 2053103's VP8 track-tick duration test patch (Phabricator D311895). It adds the shared
EncodeFrameDurationsandSumDurationstest helpers todom/media/gtest/TestVideoTrackEncoder.cpp. - Apply the attached patch on top of it. It adds the
VP8VideoTrackEncoder.PublicDurationsConserveCumulativeTrackTimelinegtest. - Build and run:
./mach gtest 'VP8VideoTrackEncoder.PublicDurationsConserveCumulativeTrackTimeline'.
The test emits two frames whose boundaries fall at cumulative 1 tick and cumulative 90 ticks of the 90 kHz track. It uses a 12 us first boundary (which converts to one 90 kHz tick: 12 * 90000 / 1000000 = 1) and a 1000 us end boundary (which converts to 90 ticks), so it is not blocked by bug 2053103's dropped-first-frame behaviour and isolates public-duration composition.
Actual results: the first published duration is 11 us (correct), but the second is 988 us and the total is 999 us:
FAIL - Expected equality of these values:
totalPublicEnd - firstPublicEnd
Which is: 989
durations[1]
Which is: 988
FAIL - Expected equality of these values:
totalPublicEnd
Which is: 1000
SumDurations(durations)
Which is: 999
Expected results: the published durations should conserve the microsecond value of the cumulative encoded tick range. Cumulative 1 tick is 11 us and cumulative 90 ticks is 1000 us, so the second duration should be 1000 - 11 = 989 us and the total should be 1000 us.
Technical details: in VP8TrackEncoder::ExtractEncodedData the per-frame duration is computed as a difference of two media::TimeUnit values that are still in track ticks:
media::TimeUnit totalDuration =
media::TimeUnit(mExtractedDuration.value(), mTrackRate); // cumulative ticks
media::TimeUnit duration = totalDuration - mExtractedDurationUs; // per-frame tick delta
mExtractedDurationUs = totalDuration;
...
return MakeRefPtr<EncodedFrame>(timestamp, duration.ToMicroseconds(), ...);
Both operands share the 90 kHz base, so the subtraction stays in tick space and duration.ToMicroseconds() converts the per-frame tick delta. For the two frames above that yields ToMicroseconds(1 tick) = 11 and ToMicroseconds(89 ticks) = 988, summing to 999, whereas ToMicroseconds(90 ticks) = 1000.
The conserving fix is to keep the running endpoint in microseconds and difference the microsecond values: publish totalDuration.ToMicroseconds() - previousEnd.ToMicroseconds() for each frame (the variable is already named mExtractedDurationUs, suggesting this was the original intent). Successive microsecond endpoints then telescope back to the exact cumulative microsecond length. The skipped-frame branch of Encode extends a frame by skippedDuration.ToMicroseconds() and inherits the same drift.
| Reporter | ||
Comment 1•4 months ago
|
||
gtest reproducing the defect. Applies on top of the VP8 track-tick duration test patch from bug 2053103 (Phabricator D311895), which provides the EncodeFrameDurations/SumDurations helpers. Currently FAILS: durations[1] is 988 (expected 989) and the total published duration is 999 us (expected 1000 us).
Description
•