1589230 - Modern Macs start dropping all frames on the youtube playback test halfway through

Status: NEW

(Core :: Audio/Video: Playback, defect, P3)

Description

[Markus Stange [:mstange]]

Steps to reproduce:

1. Be on a somewhat modern Mac.
2. Go to https://yttest.prod.mozaws.net/2019/main.html?test_type=playbackperf-test.
3. Click on 90. PlaybackPerf.VP9.2160p60@1.5X.

Expected results:
The video should play all the way through and no frames should be dropped.

Actual results:
The first ~500 frames of the video are played correctly. After that, all remaining frames are dropped.

Example:

• 0 dropped decoded frames
• 373 dropped sink frames
• 0 dropped compositor frames
• 526 presented frames

Profile: https://perfht.ml/2IRoOrl (from Instruments)

Is this because the first ~500 frames have less complexity, so it's easier to keep up with playback during that time?
On the frame_worker_threads, I can clearly see the load increasing around the 6.5 second mark, so maybe that's when those threads start decoding more complex frames.

Comment 1

[Karl Tomlinson (:karlt)]

Thanks. The behavior seems reproducible on an oldish Linux box too.

These tests don't demonstrate the same problem:
22. PlaybackPerf.VP9.2160p60@1X
84. PlaybackPerf.VP9.2160p60@1.25X
89. PlaybackPerf.VP9.1440p60@1.5X
95. PlaybackPerf.VP9.1440p60@2X

The total number of frames reported in the test UI differs from run to run, even on the tests with no dropped frames.

In Google Chrome 78.0.3904.34, the tests do not run at all:

TestExecutor:  90:PlaybackPerf.VP9.2160p60@1.5X: Failed with "Test failure: The video could not be loaded, either because the server or network failed or because the format is not supported."

Comment 2

[Karl Tomlinson (:karlt)]

Where do you find the "dropped sink frames" statistic?

https://ytlr-cert.appspot.com/2019/main.html?test_type=playbackperf-test will run the tests in Chrome on Linux, but for 90. PlaybackPerf.VP9.2160p60@1.5X, dropped frames are reported from the beginning and there are big stutters in playback.

Comment 3

[Timothy Nikkel (:tnikkel)]

(In reply to Karl Tomlinson (:karlt) from comment #2)

Where do you find the "dropped sink frames" statistic?

Probably refering to these in bug 1576107, comment 2.

Comment 4

[Markus Stange [:mstange]]

Attachment: printf patch

I've been testing in a build with this patch applied and was watching the console output.

Comment 5

[Markus Stange [:mstange]]

I can confirm that Chrome also provides a very poor experience for those videos on my machine. So it looks like we're doing no worse than Chrome on this test.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Karl Tomlinson (:karlt) from comment #1)

In Google Chrome 78.0.3904.34, the tests do not run at all:

TestExecutor:  90:PlaybackPerf.VP9.2160p60@1.5X: Failed with "Test failure: The video could not be loaded, either because the server or network failed or because the format is not supported."

Note that the perftest team only sees this when running the tests against our copy of the testsuite, but Chrome afair works fine when using the original URL:

https://ytlr-cert.appspot.com/2019/main.html?test_type=playbackperf-test