689418 - Tracking: Synchronize rendering to vblank

Status: RESOLVED DUPLICATE of bug 1071275

(Core :: Graphics, defect)

Description

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

It's easy to find 2d canvas and webgl demos and <video>s that exhibit significant tearing.  I can link a bad case that Andreas and I came across with webgl if it would help.  There are some other bugs that have been filed about tearing on scrolling, but I haven't noticed this as much.

With OpenGL, the right way to fix tearing appears to be setting the "swap interval" to one frame, which synchronizes the next glSwapBuffers() call to the next vblank.  The first glSwapBuffers() is queued asynchronously, but the next one might result in the CPU blocking on the GPU swap.  That would be bad, but if we block like that it means we're trying to draw at >60Hz, which is a bug in and of itself.  See http://www.opengl.org/wiki/Swap_Interval.  The right way to do this on mac is by setting NSOpenGLCPSwapInterval.  See dep bug.

I don't know how to set this up for D3D9/10.  I know in D3D10 we use a "swap chain" which may already be doing this for us.  Bas?  Has anyone been able to reproduce tearing with d3d9/10?

We need to do this on android too but I haven't seen tearing there yet.  Our rendering pipeline might already be preventing that, but I don't understand the details of the GL backend yet.  Vlad?

I've seen tearing with CPU compositing on gtk2, but I'm not sure I care that too much.  It's probably going to be ickier and more platform specific to fix that with CPU compositing.

I'll file a followup for desktop-x11.  Waiting for advice on d3d* and android.

Comment 1

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

I forgot to add: Eideticker (https://wiki.mozilla.org/Project_Eideticker) will have a tearing analysis and so will let us test for regressions here.

Comment 2

[Robert O'Callahan (:roc) (email my personal email if necessary)]

With D3D10 I don't see regular tearing, but I sometimes see a weird artifact while scrolling where the left half of the window is displaced vertically from the right half.

Comment 3

[Bas Schouten (:bas.schouten)]

On D3D9/D3D10 this is just a flag, we don't currently do it. The reason for this is that because we composite on the main thread, if we use a v-synced presentation interval we'll potentially (say rendering a frame takes exactly 21ms, continuously be blocking the main thread for 19ms waiting for the sync signal doing nothing useful.

Comment 4

[Andreas Gal :gal]

Android works the same way, and you are supposed to composite off the main thread. Can we do the same on windows?

Comment 5

[Bas Schouten (:bas.schouten)]

Well, that's certainly the desirable situation and the layers transaction system was actually designed with this in mind (to guarantee continuation of animations and such which are part of the layer tree even if the main thread is active in a transaction). But it's never been a high priority, it's not an easy task.

Comment 6

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to Bas Schouten (:bas) from comment #3)
> On D3D9/D3D10 this is just a flag, we don't currently do it. The reason for
> this is that because we composite on the main thread, if we use a v-synced
> presentation interval we'll potentially (say rendering a frame takes exactly
> 21ms, continuously be blocking the main thread for 19ms waiting for the sync
> signal doing nothing useful.

Does d3d9/10 have something like glSwapInterval() that will asynchronously queue the first swapped frame?  Do we use something equivalent to a double-buffered context with d3d9/10?  (I thought we did ...)

Comment 7

[Robert O'Callahan (:roc) (email my personal email if necessary)]

What's the simplest way to composite off the main thread?

Move the LayerManagerD3D10::Render() call out of EndTransaction() to a composition thread, and make BeginTransaction() and anything else that might interfere with the layer tree wait for the latest composition to finish?

Comment 8

[Emanuel Hoogeveen [:ehoogeveen]]

Regarding the Windows case:

Note that *if the DWM (Windows Vista/7's compositor) is doing its job right*, you shouldn't get any tearing on Windows Vista/7 (though this enters a race condition with OpenGl's internal VSyncing attempts from what I've heard). If it's not doing its job right there's not a lot you can do about it.

For Direct3D 9: If the DWM is disabled or unavailable (Windows XP and below), you have to handle syncing to VBlank yourself or by asking Direct3D 9 to do it with D3DPRESENT_INTERVAL_ONE (which makes Present calls block until VBlank). Note however that D3DPRESENT_INTERVAL_ONE manages to work better than D3DPRESENT_INTERVAL_DEFAULT only by reducing user mode thread granulity to 1ms per time slice, which supposedly causes problems with entering low power states. And even D3DPRESENT_INTERVAL_ONE isn't necessarily perfect as VBlank lasts ~0.5ms in my experience.

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

If we do want to composite off the main thread, ImageLayers would be a slight problem when used with gfxASurface Images, because we might (e.g. for animated images) modify the surface between layer transactions. We could just move SurfaceToTexture to the Validate phase and ensure it makes a copy.

The bigger problem would be CanvasLayers. We'd want to be able to overlap canvas drawing with composition, which means we'd probably have to introduce a copy during Validate.

Comment 10

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

For off-main-thread compositing I think we'd want to reuse our shadow layer machinery.  We're going to want this pretty soon for the native android frontend and some other projects.  I've been thinking a bit about it, was planning on posting to .platform but we could have the discussion here instead.

But I would still like to fix tearing where possible now, if we can do it without blocking the main thread.  Maybe windows isn't one of those places?

Comment 11

[Markus Stange [:mstange]]

OS X isn't one of those places either.

But at least on OS X the time we block on the main thread is very low if we don't swap at a higher rate than the refresh rate. Specifically, if more than one frame has elapsed since the last swap, the next swap will succeed immediately (without blocking on v-sync), probably due to double-buffering.

One way to ensure that the time between two swaps is more than 1/60 of a second would be to drive painting from the refresh driver, which bug 598482 is aiming at.

Comment 12

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Yes, that's fine; if we're painting faster than the monitor refresh rate, we're just wasting battery anyway so that's a separate bug.

Comment 13

[Markus Stange [:mstange]]

It's not just separate; fixing it may in fact be a prerequisite to this bug. That's what I was trying to say.

Comment 14

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Yep.  I'm not disagreeing :).

Comment 15

[Joe Drew (not getting mail)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #9)
> If we do want to composite off the main thread, ImageLayers would be a
> slight problem when used with gfxASurface Images, because we might (e.g. for
> animated images) modify the surface between layer transactions. We could
> just move SurfaceToTexture to the Validate phase and ensure it makes a copy.
> 
> The bigger problem would be CanvasLayers. We'd want to be able to overlap
> canvas drawing with composition, which means we'd probably have to introduce
> a copy during Validate.

I don't know if we can do BasicLayers *at all* of the main thread, too. Our Cairo isn't thread-safe in any way.

Comment 16

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Jeff and I briefly discussed re-enabling thread safety in cairo at the all hands.  He thinks it might be a palatable overhead, and pointed out that threadsafe is default in cairo so that's what FF runs with when using system cairo.  We could try just flipping the switch and firing off a talos bomb.  (Or do a telemetry or test pilot A/B study.)

Comment 17

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Note too that we need to solve this problem for Web Worker canvases.

Comment 18

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #10)
> For off-main-thread compositing I think we'd want to reuse our shadow layer
> machinery.  We're going to want this pretty soon for the native android
> frontend and some other projects.  I've been thinking a bit about it, was
> planning on posting to .platform but we could have the discussion here
> instead.

I'm not quite sure what you mean here.

Assuming we have a multiprocess setup, do you mean that the chrome process main thread, which currently does all the compositing, should block on vblank?

Or do you mean that, the chrome process main thread should combine the ShadowLayers from the content process with its own layers and then propagate that entire layer tree to another thread, which composites *those* ShadowLayers?

Comment 19

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

The "use cases" above are
 - with the new native android frontend, we'll have Gecko running web content on a non-main thread.  For responsiveness, we want another thread to composite.  Initially it's easiest for that thread to be main().
 - on windows (maybe) and android (maybe), the compositor might have to block on vblank to avoid tearing.  If so, we'll want to composite on a non-UI thread.  It appears that we don't need to do this on desktop-x11 and OS X.

So the set of general problems we face are
 - content thread and compositor thread will always be different
 - content thread and compositor threads might be in different processes
 - compositor thread and UI thread (processing input) might need to be different threads
 - compositor thread and UI thread might need to live in the same process to save on memory
 - compositor thread and UI thread might need to live in separate processes for stability

We already have a lot of machinery for maintaining an immutable semantic copy of the layer tree in another thread context through shadow layers.  My suggestion is to reuse that for cross-thread/same-process sharing too.  It was designed with that scenario in mind.

Comment 20

[Mark Rejhon]

I have done some VSYNC tests on multiple web browsers.
Applicable observations on 2 desktops, 3 laptops, iPod, iPad, 2 Android, PlayBook:

- Chrome: VSYNC behavior on all 5 machines: Mac x 2, WindowsXP, Windows7, Windows8.
- Safari: VSYNC behavior on all current Apple platforms (Mac, iPhone 6, iPad 6) but doesn't VSYNC on Windows.
- IE10: VSYNC on Windows8 machine.
- Galaxy SII Android (4.0): VSYNC behavior.
- Old Android (2.3): Does not VSYNC.
- BlackBerry PlayBook: Does not VSYNC.
- Mains power made no differences in all tests, with the sole exception of the WindowsXP laptop which switched downwards to 50Hz mode in battery saver mode.  requestAnimationFrame operated 50 times per second in Chrome.  Also, iOS4 and lower platforms do not VSYNC.  iOS 6 does.  iOS 5 is untested.

I am impressed at how suddenly (in a 12 month time period) the majority of both desktop and mobile browsers are now choosing to VSYNC.  It is important that FireFox follows this path, I feel.

Also, 120Hz-native-refresh computer monitors are also gradually becoming cheaper, too.  Models such as Asus VG236H (120Hz), VG278H (144Hz), Samsung S23A700D (120Hz), Acer GD235Hz (120Hz), Benq XL2420T (120Hz).   On these displays, moving text much clearer in both Chrome(all) and Safari(Mac) at 120Hz - especially when scrolling.  Text doesn't blur as much when doing things like dragging around the window, scrolling, etc, and animations in many web games are much smoother.  I'd love to see this happen for FireFox.

Comment 21

[Mark Rejhon]

Note, related topic.  requestAnimationFrame runs at 120Hz (or 144Hz) on Chrome on these monitors.    If FireFox does not wish to do this, one possible interim compromise (if it meets FireFox needs) is a rAF that syncs to VSYNC only when it's near a 60Hz multiple.  (rAF every VSYNC at 60hz, rAF every 'other' VSYNC at 120Hz).  Not ideal, but would smooth out a lot of animations.

It's also noteworthy you don't need precise VSYNC, as long as you've got compositing (all newer Windows and Mac operating systems use VSYNC'd window manager compositing).  The frame timings could be +/- 50% of the VSYNC, and the presentation will still be perfectly smooth. (until there's bigger deviations).  Multi-core devices with fast I/O (SSD, good GPU) are getting widespread and VSYNC is getting reliable (+/- less than 10%).   Chrome's timer jitter during VSYNC is still approximately 1ms on a fast system.  

One possible method (that doesn't require busy loops) - one can even just poll the scan line register to predict when the next VSYNC will occur (on Windows, it's Direct3D "RasterStatus.ScanLine" - it's a vertical pixel row counter that continually increments through a refresh until it hits VSYNC (Direct3D "RasterStatus.InVBlank"), and schedule the requestAnimationFrame timer based on that.  The prediction will be accurate enough.

Comment 22

[Bas Schouten (:bas.schouten)]

(In reply to Mark Rejhon from comment #21)
> One possible method (that doesn't require busy loops) - one can even just
> poll the scan line register to predict when the next VSYNC will occur (on
> Windows, it's Direct3D "RasterStatus.ScanLine" - it's a vertical pixel row
> counter that continually increments through a refresh until it hits VSYNC
> (Direct3D "RasterStatus.InVBlank"), and schedule the requestAnimationFrame
> timer based on that.  The prediction will be accurate enough.

On Vista+ IDXGIOutput::WaitForVBlank really sounds easier if you just want to wait for it :)

http://msdn.microsoft.com/en-us/library/windows/desktop/bb174559%28v=vs.85%29.aspx

Comment 23

[Mark Rejhon]

Bas, I know, but that's a blocking API.  You want to do a non-blocking API method.
Also, posted more info in the "requestAnimationFrame VSYNC bug" #707884
https://bugzilla.mozilla.org/show_bug.cgi?id=707884

Apparently, almost 90% of computer users (using machines 3 years old or newer) are now VSYNC'd.  I think it's time to raise the priority of this bug...

Comment 24

[Avi Halachmi (:avih)]

Following initial implementation for vsync support on windows (bug 856427), here are the vsync-related subjects I could think of, some of which already mentioned here and on related bugs (bug 555834, bug 707884, bug 689419, bug 792639, bug 748816, bug 647945).

1. Choose API
- Blocking/non-blocking (blocked thread?)?
- Different per platform? (on OS X we currently block main thread on glSwapBuffer AFAIK).
- Different per GFX backend?
- Bas' implementations: non-blocking getVblankInfo(out_lastVblankTimestamp, out_vblankInterval) seems a viable goal across the board (bug 856427).

2. How to handle high refresh rates? (i.e. 120hz, 144hz monitors, etc) and/or if we can't keep up?
- Drop vsync (rate only limited by system performance)? Should remove minimum timer duration? (is it 4ms now?)
- Decimate+latch to within a configurable acceptable range (e.g. latch to 30hz - 75hz?)
  - e.g. fixed 40hz of 120hz/3 is possibly better than frequent toggling between 40hz/60hz/120hz (I tried it, subjectively it is better).
  - How frequently to detect new constant target rate (5 frames? 100ms?)?
  - Easy to detect we're late, less easy to detect that we can increase rate (but possible).
- W3C working group for framelimit? (comment 6).
- (~Off topic) This subject might also be used to reduce CPU usage when on battery.

3. How does this interact with OMTC? what about WebGL?
- Hopefully no special treatment required.
- Possibly vsync timing comes from OMTC infrastructure?
- Even with perfect OMTC swap timing, there's still a value in accurate and synchronized refresh-driver timing.

4. Callback phase:
- Do we trigger callbacks just before vblank/bufferSwap? just after? some NN% before/after?

5. We currently have integer ms timers. Higher resolution could prevent some 1-2ms jitter (minor, but visible).
- IE10 desktop has perfect vsync (16.67ms with stddev of ~0.15ms, probably OMTC as well).
- Chrome desktop seems to have 1ms jitter.
- Opera desktop doesn't seem to have vsync support.

6. What do we do on platforms which don't support high-res timers? Can't sync to vblank?
- Keep current implementation (default 60hz timer, possibly change to 60/1.001 for 59.94Hz - per recommendation from MS, not explicitly synced to anything).

7. How does this relate to different drivers? (e.g. auto-vsync when can keep up by NVidia, possibly others).

Existing test cases:
- https://bug856427.bugzilla.mozilla.org/attachment.cgi?id=731656
- http://people.mozilla.org/~wmccloskey/example1.html
- https://bug731974.bugzilla.mozilla.org/attachment.cgi?id=637954

Comment 25

[Mark Rejhon]

Please do rAF at 120 Hertz.  I run the Blur Buster Blog and I oppose decimmation.  I am launching the Blur Busters Motion Tests and Mozilla is one of the currently "unsupported" browsers because it fails VSYNC tests.  Only Chrome passes the perfect vsync tests all the way to 240Hz (240fps@240Hz works in Chrome), IE10 runs perfect vsync to 105Hz (105fps@105Hz) then it decimates to half framerate artificially.  Safari VSYNC is tested on Mac's and iPad/iPhone to perfect 60fps@60Hz (higher refreshes untested on Apple).  I tested refresh rates in 1 Hertz increments.

Please, do not decimate at 120 Hz.  Make it run 120fps @ 120Hz when running on 120Hz systems.

Contact me at [e-mail] for a private VIP invite to VSYNC motion tests that I have developed (that Mozillla currently fails completely at, even at 60 Hz)

Thank you,
Blur Busters Blog
www.blurbusters.com

Comment 26

[Avi Halachmi (:avih)]

(In reply to Mark Rejhon from comment #25)
> Contact me at [e-mail] for a private VIP invite to VSYNC motion
> tests that I have developed (that Mozillla currently fails completely at,
> even at 60 Hz)

Note that I specifically mentioned configurable (though probably only via about:config). I.e. anyone could change this to higher limit than the default.

Could you please expand a bit more from your experience with high refresh rate monitors? e.g. at what rate does it not matter anymore? 100hz? 120hz? 144hz? etc. Also, if you have other interesting info you could share here, it would be appreciated.

Any chance you make this vsync motion test public? If not, please send me an invite to my bugzilla email.

Also, please test this on windows: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-d6bb269f5d02/try-win32/ It should support any refresh rate as long as your system is fast enough to keep up.

Thanks.

Comment 27

[Mark Rejhon]

The motion tests will launch publicly later this spring.
Email me [e-mail] to get the invite account for early access.

Yes, I'd like you to make it reportable and detectable by JavaScript accessing properties, so I know whenever framerate does not match refresh rate, and automatically prompt the user to raise their framelimit.  However, I implore you to make rAF run at fps=Hz by default.  So if it is only changeable via about:config, please it detectable (e.g. --moz-framelimit property or similiar) as a temporary stopgap until a W3C standardization occurs.  I plan to spearhead some standardization effort in framelimits.  

But please, seriously consider not doing it in a way that's not intuitive for 120 Hz monitor users.  If anyone is in Toronto, Canada then I offer the loan of one of my two 144 Hz monitors to one of the top-tier (official listed) Mozilla developers, for a period of month.  Consider this my plea offer.  

Amazon.com also has a sale on VG248QE's (LightBoost compatible 144Hz LCD's, 60% less motion blur than the best 60Hz LCD, and with an optional strobopic backlight mode that allows 92% less motion blur than a common 60HZ LCD) for only $280 each, in case anyone is interested in the world's clearest motion desktop LCD computer monitors.

-- In laboratory tests, you'd be surprised.  Although flicker fusion ends at approximately 75 Hz, there continues to be noticeable benefits on sample-and-hold displays far beyond 120 Hz.  The sample-and-hold effect causes motion blur even with instant pixel response, which is why CRT 60fps@60Hz still ends up having less motion blur than regular LCD 120fps@120Hz (non-stroboscopic).  120Hz LCD does have 50% less motion blur.  One good information is "Why Do Some OLED's Have Motion Blur?" at http://www.blurbusters.com/faq/oled-motion-blur .... As well as the scientific links halfway down http://www.blurbusters.com/references 

I'd say do not use a hard framelimit, but if you absolutely had to set one, set it to 250fps, to give breathing room to tomorrow's 240Hz displays (not likely to come out as native refresh rate on LCD while, but I already actually owned a 240 Hz CRT for testing.  It was a 21" CRT capable of well over 2048x1536@85Hz so it had the bandwidth; it successfully synced to 640x480 @ 240Hz.  However, high refresh rate benefit sample-and-hold displays (ala LCD) far more than they benefit flicker displays (ala CRT or LightBoost (2D, not 3D version)).

Bottom line:
- My popular motion tests will launch in public this spring
- Even if not yet W3C standardized, keep it easy
- Make it easy for 120fps @ 120Hz
.....Allow it by default like Chrome
.....If not by default, then via JavaScript control
.....If only via about:config, then make it detectable so I can warn the user. PLEASE no more difficult than this, at the absolute maximum.
- Hard frame limit no lower than 250 fps
- I offer the loan of a 144 HZ LCD to a local Mozilla developer to assist in this matter.

Comment 28

[Mark Rejhon]

P.S. Semi unrelated, but a recent rage among FPS-gaming users in the last while: 
If you have never heard about motion-blur-eliminating stroboscopic backlight, google "LightBoost"; a feature of several new 120 Hz LCD monitors.  A good article is the brand new TFT Central article, near the top four results.  It was found to outperform yesterday's "scanning backlight" (some were so weak that they ended up being a gimmick).

Comment 29

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to Bas Schouten (:bas.schouten) from comment #22)
> On Vista+ IDXGIOutput::WaitForVBlank really sounds easier if you just want
> to wait for it :)
> 
> http://msdn.microsoft.com/en-us/library/windows/desktop/bb174559%28v=vs.
> 85%29.aspx

This is kind of exactly what we want -- I'm a little hesitant about the patch that's posted here, because it still attempts to get lucky and hit vblank; it just uses more data to get lucky more often.  It's probably better than what we have now, but in an ideal world we'd spin up a small thread that does nothing but WaitForVBlank and then posts a message to the main thread to do the rAF callbacks.

Comment 30

[Katelyn Gadd (:kael)]

IMO, the best, cheapest way to lock to vsync on Windows is to do a synchronous Present on a worker thread (it's safe to use a D3D device from multiple threads as long as you do it correctly). In my experience most video drivers have an extremely efficient way to do this synchronous wait for vsync, and then you won't need high precision timers (or the battery use hit that comes with them).

To get the rAF callbacks to align with vsync, you can simply block the dispatch of those on whether or not vsync has hit since the last time you ran rAF, like a traditional double-buffering model. This will naturally align rAF callbacks with vsync when they are running fast enough while allowing you to run at framerates that aren't an integral factor of your refresh rate in performance-constrained scenarios (i.e. 45fps on a 60hz display) if you want to do that.

Comment 31

[Mark Rejhon]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #29)
> (In reply to Bas Schouten (:bas.schouten) from comment #22)
> > On Vista+ IDXGIOutput::WaitForVBlank really sounds easier if you just want
> > to wait for it :)
> > 
> > http://msdn.microsoft.com/en-us/library/windows/desktop/bb174559%28v=vs.
> > 85%29.aspx
> 
> This is kind of exactly what we want -- I'm a little hesitant about the
> patch that's posted here, because it still attempts to get lucky and hit
> vblank; it just uses more data to get lucky more often.  It's probably
> better than what we have now, but in an ideal world we'd spin up a small
> thread that does nothing but WaitForVBlank and then posts a message to the
> main thread to do the rAF callbacks.

You can also instead use IDXGIOutput::ScanLine ... (Also google "RasterStatus.ScanLine").  That's reading the scan line register; it works on GeForce's and Radeon's and many graphics adaptors.  I tested it on my computer; it does indeed give you a very accurate "number of scanlines until the vertical blanking is hit".  Make sure there's a backup if it returns invalid values.  That will give you a very accurate time estimate of when the next VBlank will occur; and allow you to schedule a timer (or allow you to sleep for a while before running a busy loop)

Also, if there are fluidity problems; one may want to test adjusting the phasing of the page flips relative to the blanking interval; e.g. flip early in the blanking interval, or flip late in the blanking interval, or even slightly before/after.

Yes -- graceful slowdown of framerate in performance-constrained situations.  The way Chrome does it is that rAF will always flip accurately (less than +/- 1 millisecond) if performance is fast and rAF doesn't spend too much time.  In my tests, I've found it is best to degrade slowly 60->59->58->57->etc, rather than suddenly 60->45->30.  e.g. Simply flip immediately when rAF returns, or on the next vblank if we've missed the previous one.   If the code in rAF takes a long time, it simply throttles the framerate but will quickly and easily snap back to flipping on vblank assuming rAF performs quickly enough (e.g. uses less than 10% of CPU or something). 

Also, I'm not sure how other browsers does it:
1. Executes rAF, then hold the buffer, then flip on vsync; or
2. Waits for vsync, execute rAF, flip immediately after rAF

I have a _suspicion_ that some browers do (1) and other browsers do (2).  The advantage of (1) is that it's more reliable; while the advantage of (2) is that there's less input lag.  Either way should work successfully if rAF is fairly quick (e.g. takes only a couple milliseconds to execute).

I did some intentional random delay's in Chrome rAF, and it proved resilent (animations played perfectly smooth) even though I added random delays from 1ms to 12ms into rAF for a 60fps@60Hz rAF; zero stutters occured even up to approximately ~80% of a frame length (1/60sec = 16.7ms), for a single Chrome window (and no background apps using CPU).   Safari on Mac was much more sensitive to excessive time being spent in rAF, it stuttered the moment I added about a 6ms delay (less than 50% of a 1/60sec delay) while Chrome on the same Mac was much more resilent in maintaining rAF VSYNC than Safari on the Mac.  Either way, it clearly shows that Chrome has a more robust rAF implementation it seems, or it integrates better with the graphics drivers.  

For Mozilla, the litmus test is that as long as rAF survives a 1/2 frame lateness, it should still accurately flip on VSYNC with absolutely no stutter.  A good test is to execute random delays within rAF (or do random graphics workloads) lasting up to (1ms to 8ms) for 60Hz rAF; if it stutters absolutely zero times, then you've done a sufficiently good rAF implementation.   If you manage to make animations play smooth all the way up to ~80-90% lateness (rAF's lasting 14 milliseconds), you've now become as good as Chrome's rAF VSYNC ability.

I spent hundreds of hours on the HTML5 programming as well as also testing dozens of web-based motion tests on desktop/mobile devices using Opera (Full/Mini), Safari, IE8, IE9, IE10; several versions of Chrome; iPad, iPhone, iPod, Win8 tablet.    I was surprised most browsers now does rAF VSYNC today (even on iPod Touches and Windows 8 Surface tablets!).   Only a few browers like Mozilla and Opera don't VSYNC during rAF yet; but you're on the right track working on this now.  Thanks!

I've now given username/password invites to one of you already (Vladimir); email [e-mail] if you want early VIP invites to the Blur Busters Motion Tests (going public later this spring).   I look forward to meeting some local Torontoians and demonstrating what I've done so far (and loaning a 144 Hz monitor, if necessary).

Comment 32

[Mark Rejhon]

Ooops, that's actually Microsoft.DirectX.Direct3D.RasterStatus.ScanLine
http://msdn.microsoft.com/en-us/library/windows/desktop/bb295962(v=vs.85).aspx

Comment 33

[Mark Rejhon]

(In reply to Avi Halachmi (:avih) from comment #26)
> Also, please test this on windows:
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/ahalachmi@mozilla.
> com-d6bb269f5d02/try-win32/ It should support any refresh rate as long as
> your system is fast enough to keep up.

I just had the opportunity to test the nightly (firefox-22.0a1.en-US.win32.installer.exe) ... It works MUCH better; it even detect 120Hz correctly and runs my motion tests better.  But still very jerky; but it's not flipping at the correct phasing.   

I've emailed the login/password to the Blur Busters Motion Tests to you;

Try tweaking:
- Executes rAF, then hold the buffer, then flip on vsync; or
- Waits for vsync, execute rAF, flip immediately after rAF
- Try reading RasterStatus.ScanLine, and calculating how long it is until the next blanking interval via formula: (1/Hz)*(ScanLine/ScreenHeight).  Set a timer for then, and flip the vsync then.  Try also adding small phasing offsets to the timer too (e.g. +1ms, +2ms).  

NOTE: ScreenHeight = pixel height of the current monitor the browser is on.  Different monitors can run at different refresh rates. Chrome seems to automatically use the slowest refresh rate if a window is overlapping multiple monitors running at different refresh rates.   Also note that monitors can run at fractional refresh rates (e.g. 59.9Hz for one monitor and 60.1Hz for a different monitor); bear this in mind.  Some OS calls (DisplayMode.RefreshRate) will only report integral refresh rates, try not to depend on those.

Comment 34

[Mark Rejhon]

Other ideas:
- Use a separate thread, wait for vblank using a loop
- Use a separate thread, wait for vblank using call

Additional note -- if using the RasterStatus.ScanLine method, the earlier formula I've given is actually incorrect; it is the time since the LAST vblank.  The correct formula for the time to the NEXT vblank is actually:

(1 / Hz) * (ScanLine / (ScreenHeight - ScanLine))

Basically:
(1 / Hz) = the length of a refresh
ScanLine = the scan line number (pixel row number) graphics is currently outputting. 
ScreenHeight = the vertical resolution 

(Hz/ScanLine/ScreenHeight of the screen you've chosen to synchronize VSYNC to; the other browsers tend to choose only one screen to synchronize to if the browser is spread over multiple screens during running multimonitor; sometimes all screens are synced to exactly the same refresh rate; other times not)  ... Mind you, I didn't account for the invisible scanlines (blanking interval lines) which would skew the calculations somewhat; but this only inserts an error of about ~5%.  May be sufficient.

Comment 35

[Mark Rejhon]

More testing done over the last while (nightly build).  Sometimes the motion tests run perfectly smooth at 120 Hz in the nightly build; if I resize the browser window to a specific size or minimize/maximize; but it is a little crapshoot.  Only 4% CPU, unless I resize the browser window; (then the animations freeze for 2 seconds; strange.  Usually it resumes within a fraction of a seciond).

P.S. I've now emailed early invites to the Blur Busters Motion Tests/benchmarks (launching this spring) to three people here now upon request; should be excellent test cases for this task.  If others who is programming Mozilla graphics needs to test; email me [e-mail]

Comment 36

[Avi Halachmi (:avih)]

Mar(In reply to Mark Rejhon from comment #35)
> More testing done over the last while (nightly build).  Sometimes the motion
> ...

Let's continue this discussion at bug 856427 - which contains the patch for the build you're testing.

This is a tracking bug, so preferably we keep it less noisy except for general ideas/summaries/related-bugs/etc. Once we have conclusions from the implementation of bug 856427, we can post them here.

Thanks.

Comment 37

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I have a proposal for OMTC vsync that I will propose on dev-platform.

Comment 38

[Mark Rejhon]

Some useful ongoing comments are still happening at:
https://bugzilla.mozilla.org/show_bug.cgi?id=856427
A new trybuild has been posted there.
My comments will now move over from there to here.

Also, some new links relevant to testing browser rendering to VSYNC:

1. My 60Hz,100Hz,120Hz browser testing analysis at:
http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/

2. Motion tests at:
http://www.testufo.com (check for stutters)
-- rAF() timing deviation http://www.testufo.com/#test=animation-time-graph
-- rAF() processing time http://www.testufo.com/#test=animation-time-graph&measure=rendering

3. 120fps experimental video test for 120Hz monitor users:
http://www.blurbusters.com/real-time-120fps-video (check for stutters)
(realtime 120fps H.264 high-framerate video playing back in <VIDEO> element).

Comment 39

[Mason Chang [Inactive] [:mchang]]

Resolving as a duplicate of silk, which aligns rendering to vsync.

Comment 40

[Daniel C]

(In reply to Mason Chang [:mchang] from comment #39)
> Resolving as a duplicate of silk, which aligns rendering to vsync.
> 
> *** This bug has been marked as a duplicate of bug 987532 ***

But that bug is in the Firefox OS Product and OS, whereas this is in Core with OS "All". This isn't a duplicate of that bug.

Comment 41

[Mason Chang [Inactive] [:mchang]]

We're porting it over to desktop as well, just the initial launch was for Firefox OS.