It looks like the problem here is the fact that your display frame rate is not exactly 2x of the video frame rate, when using the audio clock as the reference clock. The display frame rate is ever-so-slightly slower than that. I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, [this post](https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-about-%5Bvideo-stuff%5D.html) asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment. If the display frame rate were exactly 2x of the video frame rate, we would see the following pattern across the entire video: update/still/update/still/update/still/... . However, since the display frame rate is slightly slower, at some point we've drifted enough that we need to skip a "still" frame, and end up with two consecutive "update" frames. .../update/still/**update**/**update**/still/update/still/... In the profile, this double-update (or "skipped still") happens, on average, every 17.2 seconds. It happens at the following profile timestamps: 14.6, 32.0, 49.0, 66.2, 83.4, 100.0, 117.1, 134.5, 151.4, 169.2, 186.4, 203.6, 220.9, 237.8 You can see it in the marker list when looking at the UpdateCompositedFrame markers: https://share.firefox.dev/2WIiHwp A still frame has "no change" at the end. Some numbers: [This video](https://www.youtube.com/watch?v=FYCjJHhxONc) has a frame rate of 29.97 fps, as evidenced by the video frame timestamps in the video file: frame 3 is at 33ms and frame 9044 at 301701ms. This gives (9044 - 3) * 1000 / (301701 - 33) = 29.97 fps. In your profile, I also see the following timestamps: ``` frame 3993 plays at system time 2968ms - 21.3ms = 2947ms during composition 9929 and has video timestamp 133166000µs frame 9049 plays at system time 171662ms - 2.3ms = 171660ms during composition 20031 and has video timestamp 301868000µs frame 11300 plays at system time 246782ms - 8.8ms = 246773ms during composition 24531 and has video timestamp ??? ``` Let's compute the video fps relative to the system clock: (11300 - 3993) * 1000 / (246773 - 2947) = 29.968092 fps Good, it looks like the system clock and the audio clock mostly agree. Let's compute the display fps relative to the system clock: (24531 - 9930) * 1000 / (246782 - 2968) = 59.88581 fps So that's the problem. If the display was refreshing at 59.940 fps, there would be no stuttering. But it is refreshing at 59.886 fps. And since we use the audio clock as the reference clock, that means that we expect a "skipped still" every now and then. How frequently, exactly? Well, let's compare the frame rates: 59.940 fps - 59.886 fps = 0.054 fps = 1 frame per 18.5 seconds. Aside: On my machine, I also see microstuttering on the video, on average every 16.4 seconds - but it's the reverse problem! My display refreshes at 60 fps, so slightly *faster* than the video. So instead of "skipped stills" I get "skipped updates", or "double stills".
Bug 1652540 Comment 10 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
It looks like the problem here is the fact that your display frame rate is not exactly 2x of the video frame rate, when using the audio clock as the reference clock. The display frame rate is ever-so-slightly slower than that. I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, [this post](https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-about-%5Bvideo-stuff%5D.html) asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment. If the display frame rate were exactly 2x of the video frame rate, we would see the following pattern across the entire video: update/still/update/still/update/still/... . However, since the display frame rate is slightly slower, at some point we've drifted enough that we need to skip a "still" frame, and end up with two consecutive "update" frames. .../update/still/**update**/**update**/still/update/still/... In the profile, this double-update (or "skipped still") happens, on average, every 17.2 seconds. It happens at the following profile timestamps: 14.6, 32.0, 49.0, 66.2, 83.4, 100.0, 117.1, 134.5, 151.4, 169.2, 186.4, 203.6, 220.9, 237.8 You can see it in the marker list when looking at the UpdateCompositedFrame markers: https://share.firefox.dev/2WIiHwp A still frame has "no change" at the end. Some numbers: [This video](https://www.youtube.com/watch?v=FYCjJHhxONc) has a frame rate of 29.97 fps, as evidenced by the video frame timestamps in the video file: frame 3 is at 33ms and frame 9044 at 301701ms. This gives (9044 - 3) * 1000 / (301701 - 33) = 29.97 fps. In your profile, I also see the following timestamps: ``` frame 3993 plays at system time 2968ms - 21.3ms = 2947ms during composition 9929 and has video timestamp 133166000µs frame 9049 plays at system time 171662ms - 2.3ms = 171660ms during composition 20031 and has video timestamp 301868000µs frame 11300 plays at system time 246782ms - 8.8ms = 246773ms during composition 24531 and has video timestamp ??? ``` Let's compute the video fps relative to the system clock: (11300 - 3993) * 1000 / (246773 - 2947) = 29.968092 fps Good, it looks like the system clock and the audio clock mostly agree. Let's compute the display fps relative to the system clock: (24531 - 9930) * 1000 / (246782 - 2968) = 59.88581 fps So that's the problem. If the display was refreshing at 59.940 fps, there would be no stuttering. But it is refreshing at 59.886 fps. And since we use the video frame timestamps as authoritative (in referenc to the audio clock), that means that we expect a "skipped still" every now and then. How frequently, exactly? Well, let's compare the frame rates: 59.940 fps - 59.886 fps = 0.054 fps = 1 frame per 18.5 seconds. Aside: On my machine, I also see microstuttering on the video, on average every 16.4 seconds - but it's the reverse problem! My display refreshes at 60 fps, so slightly *faster* than the video. So instead of "skipped stills" I get "skipped updates", or "double stills".
It looks like the problem here is the fact that your display frame rate is not exactly 2x of the video frame rate, when using the audio clock as the reference clock. The display frame rate is ever-so-slightly slower than that. I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, [this post](https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-about-%5Bvideo-stuff%5D.html) asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment. If the display frame rate were exactly 2x of the video frame rate, we would see the following pattern across the entire video: update/still/update/still/update/still/... . However, since the display frame rate is slightly slower, at some point we've drifted enough that we need to skip a "still" frame, and end up with two consecutive "update" frames. .../update/still/**update**/**update**/still/update/still/... In the profile, this double-update (or "skipped still") happens, on average, every 17.2 seconds. It happens at the following profile timestamps: 14.6, 32.0, 49.0, 66.2, 83.4, 100.0, 117.1, 134.5, 151.4, 169.2, 186.4, 203.6, 220.9, 237.8 You can see it in the marker list when looking at the UpdateCompositedFrame markers: https://share.firefox.dev/2WIiHwp A still frame has "no change" at the end. Some numbers: [This video](https://www.youtube.com/watch?v=FYCjJHhxONc) has a frame rate of 29.97 fps, as evidenced by the video frame timestamps in the video file: frame 3 is at 33ms and frame 9044 at 301701ms. This gives (9044 - 3) * 1000 / (301701 - 33) = 29.97 fps. In your profile, I also see the following timestamps: ``` frame 3993 plays at system time 2968ms - 21.3ms = 2947ms during composition 9929 and has video timestamp 133166000µs frame 9049 plays at system time 171662ms - 2.3ms = 171660ms during composition 20031 and has video timestamp 301868000µs frame 11300 plays at system time 246782ms - 8.8ms = 246773ms during composition 24531 and has video timestamp ??? ``` Let's compute the video fps relative to the system clock: (11300 - 3993) * 1000 / (246773 - 2947) = 29.968092 fps Good, it looks like the system clock and the audio clock mostly agree. Let's compute the display fps relative to the system clock: (24531 - 9930) * 1000 / (246782 - 2968) = 59.88581 fps So that's the problem. If the display was refreshing at 59.940 fps, there would be no stuttering. But it is refreshing at 59.886 fps. And since we use the video frame timestamps as authoritative (in reference to the audio clock), that means that we expect a "skipped still" every now and then. How frequently, exactly? Well, let's compare the frame rates: 59.940 fps - 59.886 fps = 0.054 fps = 1 frame per 18.5 seconds. Aside: On my machine, I also see microstuttering on the video, on average every 16.4 seconds - but it's the reverse problem! My display refreshes at 60 fps, so slightly *faster* than the video. So instead of "skipped stills" I get "skipped updates", or "double stills".