[RTSP] Live stream video performance tuning

RESOLVED WONTFIX

Status

Firefox OS
RTSP
RESOLVED WONTFIX
4 years ago
2 years ago

People

(Reporter: jhao, Unassigned, Mentored)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
After Bug 1080464 is fix, the live stream video lags much more severely than played on Linux VLC player.
There are two directions come up to my mind.
1. Apply playout-delay buffer on live stream.
- How to apply it
- A/V sync information might loss in our RtspTrackBuffer
- Fake timestamp: cheat the StateMachine this is not a live stream
2. If we don't apply playout-delay
- performance issue because the threading model
(Reporter)

Updated

3 years ago
Assignee: nobody → jhao
(Reporter)

Comment 2

3 years ago
Created attachment 8545097 [details] [diff] [review]
Add timestamp to real time stream

At first, we weren't sure if RTP timestamps of real time streams are reliable. However, it turns out doing well and significantly improves the performance. The difference is very obvious on the following stream rtsp://78.109.84.47/public/flux15

Benjamin said the reason may be that the variable remainingTime in MediaDecoderStateMachine::AdvanceFrame() is incorrect because previously timestamps are not given, and resulting in |ScheduleStateMachine(remainingTime / mPlaybackRate);| to schdule state machine too often.

Also, the change in https://bugzil.la/1080464#c5 is no longer needed because timestamps are nonzero now, so I removed them.

Hi Benjamin and Ethan, could you review this? Thanks.
Attachment #8545097 - Flags: review?(ettseng)
Attachment #8545097 - Flags: review?(bechen)

Updated

3 years ago
Attachment #8545097 - Attachment is patch: true
Comment on attachment 8545097 [details] [diff] [review]
Add timestamp to real time stream

Review of attachment 8545097 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/MediaDecoderStateMachine.cpp
@@ +3014,5 @@
>      NS_ASSERTION(currentFrame->mTime >= mStartTime, "Should have positive frame time");
>      // Filter out invalid frames by checking the frame time. FrameTime could be
>      // zero if it's a initial frame.
>      int64_t frameTime = currentFrame->mTime - mStartTime;
> +    if (frameTime > 0  || (frameTime == 0 && mPlayDuration == 0)) {

I think the IsRealTime() check should keep.

::: dom/media/RtspMediaResource.cpp
@@ -640,5 @@
>    meta->GetTimeStamp(&time);
>    meta->GetFrameType(&frameType);
> -  if (mRealTime) {
> -    time = 0;
> -  }

Since you add timestamp on each frames, we don't need to tell MDSM it is a realtime stream, so please set the |mRealTime| to false. And it should make the playout-delay works.
Attachment #8545097 - Flags: review?(bechen) → review-

Comment 4

3 years ago
Comment on attachment 8545097 [details] [diff] [review]
Add timestamp to real time stream

Cancel the request since Benjamin already reviewed it.
Attachment #8545097 - Flags: review?(ettseng)
(Reporter)

Updated

3 years ago
Depends on: 1119681
(Reporter)

Comment 5

3 years ago
Bug 1119681 is checked-in. It makes live streams playing smoothly except in one condition. If it's paused and played again, the connection will be re-established and the play-out delay will lose effect. The performance will be bad for several seconds and recover afterwards.
(Reporter)

Comment 6

2 years ago
Unassign myself since I'm not actively working on Firefox OS anymore.
Assignee: jhao → nobody

Comment 7

2 years ago
Close this bug because RTSP in FxOS is rarely used by users.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.