1297971 - Telemetry to support background video decoder suspend: Report single-keyframe video durations

Status: NEW

(Core :: Audio/Video: Playback, defect, P3)

Description

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Follow-up to bug 1295831, and spawned from discussion starting at bug 1282012 comment 14.

About 60% of the VIDEO_INTER_KEYFRAME_MAX_MS numbers are '0', meaning we only found one keyframe in videos that have played for at least media.suspend-bkgnd-video.delay-ms (default 10 seconds), which is surprising.

Possibilities:
- Bug that causes over-reporting.
- People have modified the delay preference to a lower value (unlikely).
- Small looping videos that only have one keyframe but play for a long time.
- It's real!
- Other?

As a first step, it would seem prudent to stop reporting 0s, so we stop swamping otherwise-fine telemetry with uncertain data, at least until we get more data another way.

Then, a new probe could collect the actual duration of videos that only contain one keyframe, this would give a better sense of what's out there.

As part of that work, extra care should go into ensuring that we collect the 1-keyframe-only information correctly: At the moment, this is done by counting actually-decoded keyframes (i.e., at the decoder level); maybe instead, or in addition, we should collect keyframe information at the demuxer level (which should theoretically know all the keyframes in advance of any decoding).

Comment 1

[Tzuhao Kuo [:kaku]]

(In reply to Gerald Squelart [:gerald] from comment #0)
> Possibilities:
> - Bug that causes over-reporting.
> - People have modified the delay preference to a lower value (unlikely).
> - Small looping videos that only have one keyframe but play for a long time.
This sounds very possible!

Comment 2

[u480271]

(In reply to Gerald Squelart [:gerald] from comment #0)
> counting actually-decoded keyframes (i.e., at the decoder level); maybe
> instead, or in addition, we should collect keyframe information at the
> demuxer level (which should theoretically know all the keyframes in advance
> of any decoding).

I like the idea of collecting at the demuxer level. It was my understanding that all the keyframe data is known once we have metadata. (Or I misunderstood what I was told). Perhaps ping :jya on this?

Comment 3

[Tzuhao Kuo [:kaku]]

(In reply to Gerald Squelart [:gerald] from comment #0)
> Follow-up to bug 1295831, and spawned from discussion starting at bug
> 1282012 comment 14.
> 
> About 60% of the VIDEO_INTER_KEYFRAME_MAX_MS numbers are '0', meaning we
> only found one keyframe in videos that have played for at least
> media.suspend-bkgnd-video.delay-ms (default 10 seconds), which is surprising.
> 
> Possibilities:
> - Bug that causes over-reporting.
> - People have modified the delay preference to a lower value (unlikely).
> - Small looping videos that only have one keyframe but play for a long time.
Considering this case, we should only report once for one video discarding the looping, right?

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

De-assigning myself from A/V bugs/tasks that I most probably won't touch anymore, so that someone else may have a look.