Bug 1579127 Comment 22 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

The way we used to calculate dropped frame in `ImageComposite` is, we would remember the last chosen image, and update the chosen image when (1) set new images or (2) any one is asking for a current image, no matter for painting it or just checking its information, eg. size.

For (1), if the timestamp of the eariest image from the new images, which video sink gave to compositor, is later than the timestamp of the chosen image, then we would check how many frames are in between those two images, and add this number to the dropped frame amount.

For (2), every time someone is asking for a current image, we would check the current time to see if the chosen frame's timestamp fall behind the current time. If so, we would update the chosen image until the image's timestamp exceed the current time.

---

Back to our issue, as the highest composited rate is 60hz, so in theory we are not able to render any video, which frame rate is higher than 60, without dropping any frame. So why we only see frame dropping on playback rate in 1.5x or higher? according to the theory, it should also happen on 1.25x or any playback rate higher than 1.

That is because our frame dropping calculation logic didn't reflect exactly how many frames we drop. As I mentioned above, we have two places where we would do the frame dropping calculation, but both operations are not restricted by being run 60 times per second, they can be run more frequently than that.

If the video sink sets images more frequently than 60 times per second, then we would get the frame drop in (1).
If asking for a current images more frequently than 60 times per second, then we would get the frame drop in (2).

And because of (2), we actually update the image index more frequently, which causes our frame dropping number is less than our expectation. However, those dropped frame are acutally not caused by overloaded machine, which is simply that we couldn't compositor image on such high rate.

Therefore, we should check the image update rate and the compositored rate to decide if the dropped frame is caused by slow machine or higher image update rate, and only report the former one to `FrameStatisticsData `.
The way we used to calculate dropped frame in `ImageComposite` is, we would remember the last chosen image, and update the chosen image when (1) set new images or (2) any one is asking for a current image, no matter for painting it or just checking its information, eg. size.

For (1), if the timestamp of the eariest image from the new images, which video sink gave to compositor, is later than the timestamp of the chosen image, then we would check how many frames are in between those two images, and add this number to the dropped frame amount.

For (2), every time someone is asking for a current image, we would check the current time to see if the chosen frame's timestamp fall behind the current time. If so, we would update the chosen image until the image's timestamp exceed the current time.

---

Back to our issue, as the highest composited rate is 60hz, so in theory we are not able to render any video, which frame rate is higher than 60, without dropping any frame. So why we only see frame dropping on playback rate in 1.5x or higher? according to the theory, it should also happen on 1.25x or any playback rate higher than 1.

That is because our frame dropping calculation logic didn't reflect exactly how many frames we drop. As I mentioned above, we have two places where we would do the frame dropping calculation, but both operations are not restricted by being run 60 times per second, they can be run more frequently than that.

If the video sink sets images more frequently than 60 times per second, then we would get the frame drop in (1).
If asking for a current images more frequently than 60 times per second, then we would get the frame drop in (2).

And because of (2), we actually update the image index more frequently, which results in our frame dropping number is less than our expectation. However, those dropped frame are acutally not caused by overloaded machine, which is simply that we couldn't compositor image on such high rate.

Therefore, we should check the image update rate and the compositored rate to decide if the dropped frame is caused by slow machine or higher image update rate, and only report the former one to `FrameStatisticsData `.
The way we used to calculate dropped frame in `ImageComposite` is, we would remember the last chosen image, and update the chosen image when (1) set new images or (2) any one is asking for a current image, no matter for painting it or just checking its information, eg. size.

For (1), if the timestamp of the eariest image from the new images, which video sink gave to compositor, is later than the timestamp of the chosen image, then we would check how many frames are in between those two images, and add this number to the dropped frame amount.

For (2), every time someone is asking for a current image, we would check the current time to see if the chosen frame's timestamp fall behind the current time. If so, we would update the chosen image until the image's timestamp exceed the current time.

---

Back to our issue, as the highest composited rate is 60hz, so in theory we are not able to render any video, which frame rate is higher than 60, without dropping any frame. So why we only see frame dropping on playback rate in 1.5x or higher? according to the theory, it should also happen on 1.25x or any playback rate higher than 1.

That is because our frame dropping calculation logic didn't reflect exactly how many frames we drop. As I mentioned above, we have two places where we would do the frame dropping calculation, but both operations are not restricted by being run 60 times per second, they can be run more frequently than that.

If the video sink sets images more frequently than 60 times per second, then we would get the frame drop in (1).
If asking for a current images more frequently than 60 times per second, then we would get the frame drop in (2).

And because of (2), we actually update the image index way more frequently, which results in our frame dropping number is less than our expectation. However, those dropped frame are acutally not caused by overloaded machine, which is simply that we couldn't compositor image on such high rate.

Therefore, we should check the image update rate and the compositored rate to decide if the dropped frame is caused by slow machine or higher image update rate, and only report the former one to `FrameStatisticsData `.

Back to Bug 1579127 Comment 22