707884 - requestAnimationFrame should be based on vsync

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect, P3)

Description

[Grant Galitz]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Build ID: 20111120135848

Steps to reproduce:

Utilized requestAnimationFrame and its variants.


Actual results:

When running different canvas demos, utilizing the requestAnimationFrame variants and MozBeforePaint (tested along with mozRequestAnimationFrame) fails to vertical sync upon execution with each callback firing.

In fact, tearing and frame rate swings become worse with the API usage in Firefox.


Expected results:

V-Sync like what WebKit and Google Chrome currently do.

Firefox can still keep its frame rate throttle, BUT it should also synchronize against the vertical sync rate properly (delay to the next v-sync event). Chrome seems to do this properly, as tearing is not visible upon many demos.

Comment 1

[Grant Galitz]

Technically the page linked doesn't do MozBeforePaint, but it does do the flavors of requestAnimationFrame. The GBC emulator I've talked to death about online attempts MozBeforePaint in addition to requestAnimationFrame (As mozbeforepaint is a bit more stable in its frame rate).

Comment 2

[Grant Galitz]

Huh, thought I had linked the page earlier.

Comment 3

[Grant Galitz]

Pertaining to bug 555834, Firefox can keep v-sync disabled outside of the requestAnimationFrame callbacks, as it appears webkit and google chrome only v-sync upon usage of this API. If the js developer doesn't program towards requestAnimationFrame, then don't bother.

Comment 4

[Mark Rejhon]

I have done some VSYNC tests on multiple web browsers.
Applicable observations on 2 desktops, 3 laptops, iPod, iPad, 2 Android, PlayBook:

- Chrome: VSYNC behavior on all 5 machines: Mac x 2, WindowsXP, Windows7, Windows8.
- Safari: VSYNC behavior on all current Apple platforms (Mac, iPhone 6, iPad 6) but doesn't VSYNC on Windows.
- IE10: VSYNC on Windows8 machine.
- Galaxy SII Android (4.0): VSYNC behavior.
- Old Android (2.3): Does not VSYNC.
- BlackBerry PlayBook: Does not VSYNC.
- Mains power made no differences in all tests, with the sole exception of the WindowsXP laptop which switched downwards to 50Hz mode in battery saver mode.  requestAnimationFrame operated 50 times per second in Chrome.  Also, iOS4 and lower platforms do not VSYNC.  iOS 6 does.  iOS 5 is untested.

I am impressed at how suddenly (in a 12 month time period) the majority of both desktop and mobile browsers are now choosing to VSYNC.  It is important that FireFox follows this path, I feel.

Also, 120Hz-native-refresh computer monitors are also gradually becoming cheaper, too.  Models such as Asus VG236H (120Hz), VG278H (144Hz), Samsung S23A700D (120Hz), Acer GD235Hz (120Hz), Benq XL2420T (120Hz).   On these displays, moving text much clearer in both Chrome(all) and Safari(Mac) at 120Hz - especially when scrolling.  Text doesn't blur as much when doing things like dragging around the window, scrolling, etc, and animations in many web games are much smoother.  I'd love to see this happen for FireFox.

Note, requestAnimationFrame runs at 120Hz (or 144Hz) on Chrome on these monitors.    If FireFox does not wish to do this, one possible interim compromise (if it meets FireFox needs) is a rAF that syncs to VSYNC only when it's near a 60Hz multiple.  (rAF every VSYNC at 60hz, rAF every 'other' VSYNC at 120Hz).  Not ideal, but would smooth out a lot of animations.

Comment 5

[Mark Rejhon]

It's also noteworthy you don't need precise VSYNC, as long as you've got compositing (all newer Windows and Mac operating systems use VSYNC'd window manager compositing).  The frame timings could be +/- 50% of the VSYNC, and the presentation will still be perfectly smooth.   Chrome's timer jitter during VSYNC is still approximately 1ms on a fast system.  

One possible method (that doesn't require busy loops) - one can even just poll the scan line register to predict when the next VSYNC will occur (on Windows, it's Direct3D "RasterStatus.ScanLine" - it's a vertical pixel row counter that continually increments through a refresh until it hits VSYNC (Direct3D "RasterStatus.InVBlank"), and schedule the requestAnimationFrame timer based on that.  The prediction will be accurate enough.

Comment 6

[Mark Rejhon]

Short Term: 
- FireFox needs to adopt a simple rAF VSYNC approach (sync to VSYNC only if it's ~60Hz refresh or less)

Long Term:
- Getting together a few Javascript videogame makers, home theater pioneers (including me), and 3D programmers (OpenGL/DirectX), to create a W3C working group for a standard method of controlling the framelimit of requestAnimationFrame.   I propose a setting such as 
- Framelimit needs to be controlled, to be able to uncap the framelimit for new 120Hz-native computer monitors.  Also, some CRT's can be tuned to 180Hz and even beyond (I've heard of some old CRT computer monitors tweaked to 240Hz!), and it is possible that future LCD's might even accept 240Hz native refresh rate.  From the tests so far, Chrome already works at these rates already.
- One possible idea, is to propose these settings, for a W3C workgroup recommendation for frame-rate capping:
window.maxFrameRate = 120;
window.mozMaxFrameRate = 120;     // Standard experimental prefix
window.webkitMaxFrameRate = 120;  // Standard experimental prefix
Whereupon that these framerates are simply maximums, and are automatically capped by VSYNC.
- There should be also a method to turn on/off synchronization to VSYNC, perhaps as a separate setting.  This would allow unbound framerates; which is sometimes useless and power-consuming, except for benchmarking or for input lag (displaying a more freshly-generated videogame frames at the next VSYNC).  Fast-twitch head-to-head competition videogames where milliseconds matter: Two people shooting at the same time.  The first person to hit the fire button first, wins the shooting match.  Even if the gamers don't notice the speedup, removing the VSYNC limit is well-known to lower input latency by a few milliseconds, which is important for competition online videogaming (where milliseconds matter much like an Olympic 100 meter sprint, even if humans can't normally 'feel' a millisecond).   Regardless, let software control enable/disable VSYNC, and let software allow operation with 120Hz monitors.

Comment 7

[Mark Rejhon]

Since it's quite important when web browsers become used for full-fledged full-screen video game applications, I did many hours of tests, and so far, it seems requestAnimationFrame is synchronized to VSYNC on most GPU-accelerated browsers on most current platforms.

- Chrome on all platforms I tested (Windows, Mac, Linux), version 19 and later (nVidia, ATI, and Intel GPU's)
- Safari 6 on all iOS 6+ platforms (iPhone, iPod, iPad, iPad Mini)
- Safari as part of OS X Lion platform and later
- Internet Explorer 9 and 10 on Windows 7
- Internet Explorer 10 on Windows 8
- Internet Explorer 10 on Surface (Windows 8 RT tablets)

Some observations about internal framelimiters:
- Chrome has an internal framelimit resembling 250, bound by the current display's VSYNC.  Its VSYNC will even vary on laptop power management (e.g. Lenovo reducing refresh rate), so it will even float down to 50Hz and stay synchronized, as long as the browser is in foreground and active.
- IE has a framelimit somewhere roughly a bit above 60.  So it will smoothly automatically skip a VSYNC when running at 120Hz.
- Safari framelimit is untested above 60, as I don't have a 120Hz test for Mac's
- No versions of FireFox or Opera synchronizes requestAnimationFrame to VSYNC.

The conclusion:
- All default browsers on 90%+ of the most current version of OS platform synchronizes requestAnimationFrame to VSYNC.  In this territory, FireFox is falling behind "the current trend"; so an answer is needed to this problem.

Comment 8

[Mark Rejhon]

Some interesting observation in jitter tests: I've tested some advanced jitter behaviours.  Due to letting compositing layer buffer the final frame, the timing of requestAnimationFrame doesn't have to be perfect synchronization to VSYNC -- as long as it's +/- a few milliseconds -- the compositing will cleanly bring it on time to VSYNC, even if frames are rendered at random jitters at +/- few milliseconds of VSYNC timing.   As long as it's within a predetermined jitter, and the frame render times are fast, the viewed synchronization is 'perfect' - perfect scroll movements occur in the VSYNC'd browsers (Chrome19+, IE9+, Safari6+) as long as the frames are generated fast enough.  On a Core i7 with a good GPU, most browser frames are rendered in less than 1 millisecond.

In Chrome tests, the jitter before animations look jumpy, varies:
-- 6ms of jitter: On slower GPU, older Intel HD graphics on a >2-year-old Macbook ... takes about 6ms of delay before animations starts looking jumpy.
-- 16ms of jitter: On fast i7 desktop with ATI Radeon 6870 ... takes about a full frame of delay (16ms) beforer animations starts looking jumpy.

So it's pretty clear there's quite some leeway in timing jitter; there can be lots of timer jitter through the entire subsystem; what's more important is the synchronization to VSYNC.  As long as the inaccuracies that build up before compositing (rendering delays, multithreading delays, etc) still allow compositing to completely finish /before/ the next VSYNC (16ms timing window at 60Hz), the inaccuracies does not matter -- it apparently has no effect in animation fluidity.

Comment 9

[Mark Rejhon]

...VSYNC is applicable to entire browser subsystem.  VSYNC'd timing of requestAnimationFrame, VSYNC'd timing of WebGL, VSYNC'd timing of compositing and actual display (including scrolling).  So there can be a lot of dependancies, in order to get that "perfect fluid motion look" in a modern web browser (Chrome19+, Safari6+, IE9+ -- let's try to add FireFox to this ability.)

Comment 10

[Kelsey Gilbert [:jgilbert]]

Getting vsync working should actually be easier than you make it out to be. Getting it to work with layers is mostly a game of figuring out how to request the right type of window-buffering with our various rendering backends.

WebGL's 'Present' function is called when the compositor is doing a pass in response to the canvas being dirty. Hooking up compositing to vsync gives us that for free. Currently rAF is timer-based, simply because we didn't/don't have vsync set up. Once that's set up, we can use it to queue rAF events.

Comment 11

[Mark Rejhon]

Right.  Figuring out the buffering can be trickier.  As you know, it is best to Let the compositing manager do that rather than always blindly enable excess extra layers at openGL or Canvas2D level because then you get two layers of double or triple buffering.   Yoy get the problem of triple buffering that now gets rebuffered by the compositing manager.  Worst case scenario is two compositing managers (application builtin compositing and OS level compositing).  That means three layers of either double or triple buffering.  You get input lag hell.  You probably already know that the ideal low-lag situation is to detect if the OS is using GPU accelerated compositing, and then treat that as the last two buffers.  Here, you can even flickery-draw (flickers under windows XP) but looks perfectly buffered under Windows 7 or Windows 8.  Enable application-internal compositing only if OS is not providing compositing.  In this case, you actually want to disable doublebuffering and triple buffering at all other layers where unnecessary, even the OpenGL layer.   From what I read, Chrome automatically does this.  Plan your buffer layers to avoid unnecessary lag.  I think you were referring to this, but this is a topic not everyone understands the subtleties of.

Comment 12

[minus54]

@Mark Rejhon: This v-sync issue could be related to the bug 792639 ?
I'm asking you because in both cases, Chrome and IE9 are unaffected.

Comment 13

[Mark Rejhon]

(In reply to minus54 from comment #12)
> @Mark Rejhon: This v-sync issue could be related to the bug 792639 ?
> I'm asking you because in both cases, Chrome and IE9 are unaffected.

@minus54, No, the VSYNC issue and the GPU clock issues are unrelated.  Though turning off VSYNC and letting framerate be unbound, could bring the problem out -- but this should be an opt-in setting for benchmarking purposes (much like chrome://flags ... and maybe a future Javascript method for browser-based FPS-shooter games to optionally turn off vsync).

@Jeff, I didn't realize you were the same person I replied to in the other thread, so I posted a redundant reply.  My apologies.  :-)

Comment 14

[Grant Galitz]

Yeah, rAF should be doing what web developers are expecting it to do... sync with the system's graphics buffering. If it's just an arbitrary timer, then it's encouraging web developers to fall back to setInterval and roll their own "tricks" of sort. I noticed google chrome triple buffers canvas now, as my gameboy emulator does not use rAF and relies on a setInterval, but chrome is produced no skipped frames. Pretty sure this is google's way of providing the capability rAF was supposed to be bringing to the table, but by force, and with an extra frame of lag.

Comment 15

[Mark Rejhon]

This is now fixed, due to #856427.
Change Status to "FIXED" in v24.
(with caveat -- see bottom of comments in #856427)

Related:
https://bugzilla.mozilla.org/show_bug.cgi?id=856427

A good website for testing rAF VSYNC is:
http://www.testufo.com
Change tests via using the selector at upper-right.
It works great on 120Hz computer monitors, too.

This is now solved in FireFox Version 24 beta:
http://news.softpedia.com/news/Firefox-Nightly-Adds-Support-for-Vsync-for-Smooth-Animations-360245.shtml

You are now W3C compliant.  Thank you.
http://www.w3.org/TR/animation-timing/
"The expectation is that the user agent will run tasks from the animation task source at at a regular interval matching the display's refresh rate. Running tasks at a lower rate can result in animations not appearing smooth."

Comment 16

[Avi Halachmi (:avih)]

(In reply to Mark Rejhon from comment #15)
> This is now fixed, due to #856427.

Windows is currently the only platform at which we actually synchronize to the vblank signal, so it's not FIXED generally.

On Linux and OS X - we don't synchronize rAF (or the refresh driver in general) with vsync.

On OS X we do block present on vblank (and iterate rAF at 60hz by default), so if the display is 60hz, as most(/all?) OS X displays are, then it appears to use vsync, but it would jitter badly on any non 60Hz Mac display, as rAF will still iterate at 60hz.

Note that the fact that we sync to vblank on windows still doesn't mean that it looks absolutely smooth. This is due to timers inaccuracies and other congestions at the pipe/event-queue, but probably mostly due to the fact that we don't block present on vblank. Hopefully bug 894128 will track the quest for absolute smoothness on windows.

Comment 17

[Mark Rejhon]

Thanks Avi.
Although not directly VSYNC related, this is a good tool for testing VSYNC accuracy.  Here's a new requestAnimationFrame() animation accuracy graph tool, which measures time deviation of the timing.  Compare this graph in IE10, Chrome, and FF24+

http://www.testufo.com/#test=animation-time-graph

Comment 18

[Grant Galitz]

Firefox nightly does seem to have smoother animation than stable on OS X. Wonder if nightly enabled triple buffering.

Comment 19

[Mason Chang [Inactive] [:mchang]]

Just a heads up, we plan to do this with Project silk, bug 987532.

Comment 20

[Jerry Shih[:jerry] (UTC+8) (inactive)]

I think we process the requestAnimationFrame at:
http://hg.mozilla.org/mozilla-central/annotate/f78e532e8a10/layout/base/nsRefreshDriver.cpp#l1128

With bug 987532, refresh driver's tick will be triggered at vsync. So requestAnimationFrame will also base on vsync.

Comment 22

[Mason Chang [Inactive] [:mchang]]

This should already be happening with Silk. Closing as WFM.