1311877 - VideoPlaybackQuality not returning amount of frames displayed or dropped.

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Jean-Yves Avenard [:jya]]

Per spec:
VideoPlaybackQuality

https://wicg.github.io/media-playback-quality/#concepts

"Each HTMLVideoElement MUST maintain a total video frame count variable keeping track of the total number of frames that have been displayed and dropped. It MUST follow these rules:

    It is initialized to 0 when the element is created.
    It is reset to 0 when the media element load algorithm is invoked.
    It is incremented when a video frame is displayed.
    It is incremented when the dropped video frame count is incremented
"

however, as seen with about:media
Quality: 100% (total:7 dropped:0 corrupted:0)
Buffered ranges: [(0, 8.982666)]
Internal Data:
audio decoder: apple CoreMedia decoder
audio frames decoded: 1
audio state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:1 out:1 qs=0 pending:0 waiting:0 sid:4294967295
video decoder: apple software VT decoder
hardware video decoding: disabled
video frames decoded: 1 (skipped:0)
video state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:7 out:1 qs=6 pending:0 waiting:0 sid:4294967295


So we have totalVideoFrameCount = 7, yet the number of frames returned by the decoder is 1.
The decoder has buffered 7 frames in total.

Or implementation uses the number of frames *parsed* as totalVideoFrames count.

A parsed frames doesn't equal the number of frames displayed nor the number of frames dropped as the spec requires.

The frames haven't been dropped yet, they are just buffered by the decoder as it has some latency due to how the h264 decoder works.

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you for noticing this.

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1311877 - VideoPlaybackQuality.totalVideoFrameCount is presented+dropped -

totalVideoFrameCount was previously incorrectly set to the number of demuxed
frames.
According to the current W3C specs [1], it should instead be the total number of
frames that have been presented, plus frames that have been discarded.

Also added a check that discarded<=total in mochitest.

[1] https://wicg.github.io/media-playback-quality/#concepts

Review commit: https://reviewboard.mozilla.org/r/87482/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/87482/

Comment 3

[Jean-Yves Avenard [:jya]]

Comment on attachment 8803200 [details]
Bug 1311877 - VideoPlaybackQuality.totalVideoFrameCount is presented+dropped -

https://reviewboard.mozilla.org/r/87482/#review86444

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6ac2e1d8d2c8
VideoPlaybackQuality.totalVideoFrameCount is presented+dropped - r=jya

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/6ac2e1d8d2c8