Open Bug 2054562 Opened 4 months ago Updated 4 months ago

VP8TrackEncoder per-frame public durations do not sum to the encoded track length

Categories

(Core :: Audio/Video: Recording, defect, P3)

defect

Tracking

()

People

(Reporter: chunmin, Unassigned)

References

Details

Attachments

(1 file)

VP8TrackEncoder (the VP8 video encoder that MediaRecorder uses to produce WebM) assigns every encoded frame a public duration in microseconds. The internal video timeline is measured in 90 kHz track ticks (the VIDEO_TRACK_RATE constant), so each frame boundary lands on an integer number of ticks and the encoder must convert those ticks to microseconds when it publishes a frame.

The conversion is done incorrectly: the encoder derives each frame's public duration from that frame's own tick delta and converts the delta to microseconds, then relies on the sum of those per-frame microsecond durations to describe the whole track. Because 90 kHz ticks do not divide evenly into microseconds, converting-then-summing per-frame deltas is not equal to converting the cumulative tick endpoint once. The published durations therefore lose up to a microsecond per boundary and no longer sum to the microsecond length of the encoded range.

This is a defect in shipping code (dom/media/encoder/VP8TrackEncoder.cpp); it is independent of, and survives, the fix for bug 2053103. Bug 2053103 concerns a raw TimeDuration-to-track-tick truncation that drops the first frame; this bug concerns how already-encoded frames compose their public microsecond durations.

Steps to reproduce:

  1. Apply bug 2053103's VP8 track-tick duration test patch (Phabricator D311895). It adds the shared EncodeFrameDurations and SumDurations test helpers to dom/media/gtest/TestVideoTrackEncoder.cpp.
  2. Apply the attached patch on top of it. It adds the VP8VideoTrackEncoder.PublicDurationsConserveCumulativeTrackTimeline gtest.
  3. Build and run: ./mach gtest 'VP8VideoTrackEncoder.PublicDurationsConserveCumulativeTrackTimeline'.

The test emits two frames whose boundaries fall at cumulative 1 tick and cumulative 90 ticks of the 90 kHz track. It uses a 12 us first boundary (which converts to one 90 kHz tick: 12 * 90000 / 1000000 = 1) and a 1000 us end boundary (which converts to 90 ticks), so it is not blocked by bug 2053103's dropped-first-frame behaviour and isolates public-duration composition.

Actual results: the first published duration is 11 us (correct), but the second is 988 us and the total is 999 us:

FAIL  - Expected equality of these values:
  totalPublicEnd - firstPublicEnd
    Which is: 989
  durations[1]
    Which is: 988
FAIL  - Expected equality of these values:
  totalPublicEnd
    Which is: 1000
  SumDurations(durations)
    Which is: 999

Expected results: the published durations should conserve the microsecond value of the cumulative encoded tick range. Cumulative 1 tick is 11 us and cumulative 90 ticks is 1000 us, so the second duration should be 1000 - 11 = 989 us and the total should be 1000 us.

Technical details: in VP8TrackEncoder::ExtractEncodedData the per-frame duration is computed as a difference of two media::TimeUnit values that are still in track ticks:

media::TimeUnit totalDuration =
    media::TimeUnit(mExtractedDuration.value(), mTrackRate);  // cumulative ticks
media::TimeUnit duration = totalDuration - mExtractedDurationUs;  // per-frame tick delta
mExtractedDurationUs = totalDuration;
...
return MakeRefPtr<EncodedFrame>(timestamp, duration.ToMicroseconds(), ...);

Both operands share the 90 kHz base, so the subtraction stays in tick space and duration.ToMicroseconds() converts the per-frame tick delta. For the two frames above that yields ToMicroseconds(1 tick) = 11 and ToMicroseconds(89 ticks) = 988, summing to 999, whereas ToMicroseconds(90 ticks) = 1000.

The conserving fix is to keep the running endpoint in microseconds and difference the microsecond values: publish totalDuration.ToMicroseconds() - previousEnd.ToMicroseconds() for each frame (the variable is already named mExtractedDurationUs, suggesting this was the original intent). Successive microsecond endpoints then telescope back to the exact cumulative microsecond length. The skipped-frame branch of Encode extends a frame by skippedDuration.ToMicroseconds() and inherits the same drift.

gtest reproducing the defect. Applies on top of the VP8 track-tick duration test patch from bug 2053103 (Phabricator D311895), which provides the EncodeFrameDurations/SumDurations helpers. Currently FAILS: durations[1] is 988 (expected 989) and the total published duration is 999 us (expected 1000 us).

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: