The timestamps of chained-video.ogv are confusing to MediaDecoderStateMachine

NEW
Unassigned

Status

()

defect
5 years ago
4 years ago

People

(Reporter: jwwang, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Reporter

Description

5 years ago
Timestamps shown in the logs:
Decoder=7faecd789300 OnVideoDecoded [33333,66666] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [66666,99999] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [99999,133332] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [133332,166665] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [166665,199998] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [199998,233331] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [233331,266664] disc=0 dup=0 key=0

Decoder=7faecd789300 OnVideoDecoded [0,33333] disc=0 dup=0 key=1
Decoder=7faecd789300 OnVideoDecoded [33333,66666] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [66666,99999] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [99999,133332] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [133332,166665] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [166665,199998] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [199998,233331] disc=0 dup=0 key=0
Decoder=7faecd789300 OnVideoDecoded [233331,266664] disc=0 dup=0 key=0

It looks like file contains 2 streams and timestamps seem to go backward in the 2nd stream. It will confuse the state machine and drop the wrong frames.
This is correct for this test file. It does consist of two concatenated streams, each with their own timestamp origin. This is valid for Ogg and a common case in "internet radio" style streams.

If the statemachine needs monotonic timestamps it needs to adjust those coming from the decoder by adding the offset to the local segment, as we do with seeking.
Reporter

Comment 2

5 years ago
So if you tell an OggReader to seek to (266664+33333), it will give you [33333,66666] from the 2nd stream, right?

I wonder if we should adjust the timestamps in the state machine or in the OggReader. I've met a test case where [0,33333] from the 2nd stream was dropped due to frame skip or something. Therefore, the frames received by the state machine were:
[199998,233331]
[233331,266664]
[33333,66666]
[66666,99999]

There is no way for the state machine to adjust the timestamps if the 1st frame of the 2nd stream is missed and the 1st frame starts from a non-zero position.
Flags: needinfo?(giles)
I think you're right that the decoder should do the timestamp translation; it's the component which knows about the chain structure.

I fear however that we don't have _any_ support for this you can crib from. OggReader::SetChained() marks the stream as unseekable. Seeking manually in the chained test files we have doesn't seem to work at all in the audio ones. As such you're probably justified in filing a bug and disabling that file in the seek test.
Flags: needinfo?(giles)

Comment 4

5 years ago
Is the desire to enable seeking on chained video, or just make sure that chained ogg video plays correctly with seeking disabled?

(Chaining is usually used in live-streaming which isn't very seek-friendly anyway.) If the decoder rewrites the timestamps after chaining boundaries, seeking within time that has already been decoded should be possible since we know where the stream boundaries are, but seeking to buffered data ahead of what's been decoded could be quite difficult, since we don't know if there's another stream boundary coming.

Just rewriting the timestamps to make the player state machine happy sounds easier. :)
Yeah, I don't think seeking in chained streams is high-priority (we'd have to fix a lot more than just this to make it work).
Reporter

Comment 6

5 years ago
Seeking is not my concern. I am fine with seeking disabled for a chained-ogg file. My concern is the problem described in comment 0 where wrong timestampes cause the state machine to drop wrong frames. And the faster it decodes, the worse it drops frames since all frames in the 2nd stream and later will be considered out-of-date for their frame time has passed current playback position.
We don't support chained ogg video files. Our chaining support is deliberately limited to audio only, and if we've detected a chain refuse to seek. This is to reduce the complexity of supporting chaining.

I don't think it's worth fixing this bug. At most, we should change to abort seeking if we think we're in a chained file; if we encounter a page with a serialno we don't expect perhaps would be sufficient grounds to give up.
Component: Audio/Video → Audio/Video: Playback
You need to log in before you can comment on or make changes to this bug.