933953 - Investigate LuiC WebGL benchmark

Status: NEW

(Core :: Graphics, defect)

Description

[Chris Peterson [:cpeterson]]

http://luic.github.io/WebGL-Performance-Benchmark/

Google's LuiC WebGL bug: http://code.google.com/p/chromium/issues/detail?id=256729

This benchmark is part of Tom's Hardware Guide's Web Browser Grand Prix.

Comment 1

[Boris Zbarsky [:bzbarsky]]

This is a graphics thing.

Milan, note that we used to be faster than Chrome on this benchmark, but their bug claims they fixed that.  We should ideally unfix it.  ;)

Comment 2

[Milan Sreckovic [:milan] (needinfo for best results)]

We don't look good on my OS X, for sure.  I can't judge on how relevant the benchmark is from the "represents the real world" behaviour.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Fwiw, Windows performance is what tends to get benchmarked in Tom's Hardware stuff.

If this benchmark is not representative, then we should tell Tom's Hardware now, since we're the winners last time around and hence don't come off sounding like we're just sore losers!

Comment 4

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I haven't looked at it yet, but any benchmarks that attempt to measure how much they can draw for a given target framerate end up using setTimeout (or worse, setInterval).  We could easily be hurt by timer shenanigans that we worked around with requestAnimationFrame.

Comment 5

[Boris Zbarsky [:bzbarsky]]

I believe the Chrome folks are also unhappy about the setTimeout/setInterval usage in benchmarks like this, so we could coordinate with them to push back on Tom's Hardware or the benchmark authors or both if we think the benchmarks are bogus.

Comment 6

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I had a quick look.  They do use requestAnimationFrame, *but* the "cubes" benchmark defaults to a target FPS of 50.  Our rAF by default aims for 60.  Everything seems to work OK, but I wonder if they're getting some slop because of it.  The benchmark takes forever to run because it adds cubes one at a time until the framerate decreases :p  Regardless, some straightforward JS/SPS profiling should provide info here.  They use Three.js, so there could easily be lots of JS (not webgl) that's causing any perf issues.

(Also, there are some artifact issues on nightly with the cubes benchmark -- like the back buffer isn't being cleared?)

Comment 7

[Milan Sreckovic [:milan] (needinfo for best results)]

Attachment: Firefox result (pixel fill rate, z-buffer enabled)

Here's the graph from a Firefox run (OS X, Z-buffer).  Note the shape of the graph.

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

Attachment: Chrome result (pixel fill rate, z-buffer enabled)

Chrome result - note the shape of the graph.  It looks quite different with its step like nature than Firefox.  Any conclusions from that?

Comment 9

[Milan Sreckovic [:milan] (needinfo for best results)]

What about the pixel fill rate benchmark?  Why is our graph "smooth", Chrome's is "step like" and the benchmark description talks about us expecting to see a "step like" results?

Comment 10

[Kelsey Gilbert [:jgilbert]]

Ours is smooth because we don't base requestAnimationFrame on vsync.

Comment 11

[Kelsey Gilbert [:jgilbert]]

(In reply to Boris Zbarsky [:bz] from comment #1)
> This is a graphics thing.
> 
> Milan, note that we used to be faster than Chrome on this benchmark, but
> their bug claims they fixed that.  We should ideally unfix it.  ;)

In the Chromium bug linked, they only talk about resolving the canvas2d slowness.

Comment 12

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I was wondering what that "step-like" behaviour was that they were talking about in the description.

I like our graph more; it looks more like what you should expect as you increase the number of things drawn.  For Chrome, I think this is a symptom of their off-thread GL rendering and command batching.  vsync too would play a part, though I didn't think that they were actually vsync'd (just attempt to match vsync via timers)?  We could get the same stair-step behaviour with a change (I believe #if 0'd out) in the rAF code that causes us to schedule on vsync-interval quantums.

Which particular benchmark were we faster on?  Cubes?  Back when it was doing 2d canvas, or when it was webgl?  What are the numbers now?

Comment 13

[Boris Zbarsky [:bzbarsky]]

We used to be faster on Cubes, yes.  I don't know whether it was doing canvas or WebGL at the time.  See http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534-8.html for the benchmarks that got run and the scores back then...

Measuring now (on Windows) would be a good idea, yes.

Comment 14

[Guilherme Lima]

Results on the Cubes Benchmark on Windows 7 and Intel HD3000:

Firefox stabilized around 50fps with 770 cubes.
Chrome 31 stabilized around 50fps with 1480 cubes.

Firefox only used 1 core the whole time, while Chrome used more than 1 beyond a certain number of cubes. Before that I think it used another core everytime a GC (or CC?) was necessary.