1063466 - [META] [RTSP] Enhance performance by reducing frame drop rate of RTSP streaming

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: RTSP, defect)

Description

[Ethan Tseng [:ethan]]

While debugging bug 1056187 (Frame drops observed during RTSP streaming), we found several possible
causes that could impair smoothing video playback.
This bug is filed to track the issues we found. Each of them can be treated as a single solution to
improve the performance of RTSP streaming.

Cause 1 : RTP playout buffer doesn't have a playout delay.
Solution: Implement a dynamically calculated playout delay according to network jitter.
          (We plan to fix this issue in bug 1056187 since it's the major bottleneck for now).

Cause 2 : Video and audio tracks have a significant frequency deviation (jitter) in IPC.
Solution: Implement a mechanism (such as a timer) to uniform the IPC timing variation between
          video and audio tracks.

Cause 3 : We only know the start-time of a video frame, but not the end-time.
Solution: Calculate the frame duration in RTSPSource. It might be helpful to decide the precise
          playout time of a video frame.

Cause 4 : RtspMediaResource doesn't skip to the I-frame (also called key frame).
Solution: Implement the mechanism to detect and skip to the I-frame in RTP assembler and 
          RtspMediaResource. This could reduce the number of frames dropped by audio/video sync.

Cause 5 : Too many memory copies in current implementation.
Solution: Reduce memory copies to utilize buffers more efficiently.

Cause 6 : RTSP doesn't transfer the state of decoder-state-machine from decoding to buffering state.
Solution: When the playout buffer is empty, RtspMediaResource should tell the decoder-state-machine
          to transfer to buffering state. We think this could enhance user experience.

Cause 7 : IPC between RtspControllerParent/Child adds unnecessary overhead to RTSP streaming.
Solution: Refactor the RTSP architecture to move all of RTSP codes to content process except for
          socket transport, and use UDP socket e10s.

Note that cause 2 and 7 are a conflict. If we cope with cause 7, cause 2 would not be an issue.

Comment 1

[BMO Automation]

Firefox OS is not being worked on