1237245 - Video Playback not Smooth. Video JANK and JUDDER.

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Audio/Video, defect, P2)

Description

[Allen Kuo]

Attachment: firefox systrace (firefox process and surfaceflinger)

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36

Steps to reproduce:

Android Firebox cannot provide smooth video playback.
If you try to playback a 30 fps video, you will see serious video JANK  & JUDDER problem.
After checking display performance via Android Systrace, the video output frame rate cannot always keep constant video frame rate

My test video:
https://youtu.be/jqROVzXZPJs



Actual results:

play the video via my YouTube URL:
https://youtu.be/jqROVzXZPJs

You see video playback JANK & JUDDER problem.
FF android 43, 44 are all confirmed with the issue.


Expected results:

The correct video playback should be always stable.
You can compare with PC version Firefox.

Comment 1

[Kevin Brosnan [Ex-Mozilla]]

What device? Which video format? Please check our Nightly build https://nightly.mozilla.org has at least one major video fix that I know of that is not on either of the release channels yet.

Comment 2

[Allen Kuo]

After installed Nightly version, I still can reproduce the issue.

I have done the test with devices:
1. ASUS ZenPadS 8.0
2. Nexus 9

It looks like Firefox uses the OpenGL to render video.
MediaCodec -> SurfaceTexture -> OpenGL -> SurfaceView

The frame rate is not stable. I cannot point out the root cause of the problem.
Maybe SurfaceTexture to OpenGL renders (OnDraw) cannot meet reasonable timing of Display VSYNC period.

Comment 3

[jonathanshafer3]

I just tested on my Galaxy S6 and compared the video in question to my computer and did notice that the movement of the video on the S6 running latest nightly is stuttering constantly when compared with the PC version which displays smooth video movements. (PC in question is a Dell XPS l501x with a NVIDIA 430M GPU)

Comment 4

[streamencode]

I am actually seeing a similar issue with Windows 7 SP1 and Firefox 44.
Firefox 44 on Windows 10 looks fine as well as previous Firefox 43 versions on Windows 7.

I've been testing with the Youtube video that was posted by Allen.
https://www.youtube.com/watch?v=jqROVzXZPJs

I have also seen this behavior on live streams using a HTML5/HLS player.

Comment 5

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to streamencode from comment #4)
> I am actually seeing a similar issue with Windows 7 SP1 and Firefox 44.
> Firefox 44 on Windows 10 looks fine as well as previous Firefox 43 versions
> on Windows 7.

You should file a separate bug in Core - Audio/Video: Playback

Comment 6

[John Lin [:jhlin][:jolin]]

There seems to be more than one factors that cause jank:
1. Non-uniform scheduling of |VideoSink::UpdateRenderedVideoFrames()|. In that function, the audio sink play position is used to calculate the delay of updating for the next video frame[1]. Unfortunately |clockTime| there could be inaccurate. On OS X, it's because cubeb backend (AudioUnit) doesn't update it frequent enough; On Android, they are sometimes 2~4ms off. Will need more time to figure out why.

2. SurfaceFlinger and Compositor use different vsync sources. I believe that's what the attached systrace screen shows. Bug 1075025 should be able to address this. Will start with this one.

[1] https://dxr.mozilla.org/mozilla-central/source/dom/media/mediasink/VideoSink.cpp#388

Comment 7

[John Lin [:jhlin][:jolin]]

On Android, the non-uniform scheduling problem seems quite small (2~4ms) and IMHO is not significant enough to cause video stutter.
The GL compositor, although sometimes relatively slow (~20ms to composite a video frame), should be good enough for 30fps video. One possible improvement is raising the compositor thread priority like we did in B2G.
Also tried playing test video in Chrome and AOSP browser on same phones (Sony Z3C, Nexus 5, and Nexus 6) and both stutter too, but other videos look fine in all browsers.

Comment 8

[John Lin [:jhlin][:jolin]]

(In reply to John Lin [:jolin][:jhlin] from comment #7)
> The GL compositor, although sometimes relatively slow (~20ms to composite a
> video frame), should be good enough for 30fps video. One possible
> improvement is raising the compositor thread priority like we did in B2G.
Forgot to mention that what systrace shows is that usually when composition time is long, the frequency of corresponding CPU core that runs compositor thread is low. Does this mean it's CPU bound?

Comment 9

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Is this an issue with H.264 or VP9 playback?

Comment 10

[John Lin [:jhlin][:jolin]]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #9)
> Is this an issue with H.264 or VP9 playback?

Youtube sends VP9 video for both Fennec and Chrome.
However I've also downloaded it as MP4/H.264, and play on my Z3C locally with bundled video player app also looks janky to me.

Comment 11

[Levente Sacal]

Hi,

Does the issue still reproduce?

Comment 12

[Mira Lobontiu (Android QA)]

Hello,

Due to the lack of additional info, I'll close this issue as Incomplete.
If the reporter (or anyone else) can provide more information, please feel free to reopen or comment on the issue, and we'll have a thorough look on it.

Thank you!