Open Bug 933953 Opened 11 years ago Updated 2 years ago

Investigate LuiC WebGL benchmark


(Core :: Graphics, defect)





(Reporter: cpeterson, Unassigned)


(Blocks 1 open bug, )


(Keywords: perf)


(2 files)

Google's LuiC WebGL bug:

This benchmark is part of Tom's Hardware Guide's Web Browser Grand Prix.
Blocks: WBGP
This is a graphics thing.

Milan, note that we used to be faster than Chrome on this benchmark, but their bug claims they fixed that.  We should ideally unfix it.  ;)
Component: JavaScript Engine → Graphics
Flags: needinfo?(milan)
We don't look good on my OS X, for sure.  I can't judge on how relevant the benchmark is from the "represents the real world" behaviour.
Flags: needinfo?(milan)
Fwiw, Windows performance is what tends to get benchmarked in Tom's Hardware stuff.

If this benchmark is not representative, then we should tell Tom's Hardware now, since we're the winners last time around and hence don't come off sounding like we're just sore losers!
I haven't looked at it yet, but any benchmarks that attempt to measure how much they can draw for a given target framerate end up using setTimeout (or worse, setInterval).  We could easily be hurt by timer shenanigans that we worked around with requestAnimationFrame.
I believe the Chrome folks are also unhappy about the setTimeout/setInterval usage in benchmarks like this, so we could coordinate with them to push back on Tom's Hardware or the benchmark authors or both if we think the benchmarks are bogus.
I had a quick look.  They do use requestAnimationFrame, *but* the "cubes" benchmark defaults to a target FPS of 50.  Our rAF by default aims for 60.  Everything seems to work OK, but I wonder if they're getting some slop because of it.  The benchmark takes forever to run because it adds cubes one at a time until the framerate decreases :p  Regardless, some straightforward JS/SPS profiling should provide info here.  They use Three.js, so there could easily be lots of JS (not webgl) that's causing any perf issues.

(Also, there are some artifact issues on nightly with the cubes benchmark -- like the back buffer isn't being cleared?)
Here's the graph from a Firefox run (OS X, Z-buffer).  Note the shape of the graph.
Chrome result - note the shape of the graph.  It looks quite different with its step like nature than Firefox.  Any conclusions from that?
Attachment #826881 - Attachment description: Firefox result (with z-buffer) → Firefox result (pixel fill rate, z-buffer enabled)
Attachment #826883 - Attachment description: Chrome result (with Z-buffer) → Chrome result (pixel fill rate, z-buffer enabled)
What about the pixel fill rate benchmark?  Why is our graph "smooth", Chrome's is "step like" and the benchmark description talks about us expecting to see a "step like" results?
Ours is smooth because we don't base requestAnimationFrame on vsync.
(In reply to Boris Zbarsky [:bz] from comment #1)
> This is a graphics thing.
> Milan, note that we used to be faster than Chrome on this benchmark, but
> their bug claims they fixed that.  We should ideally unfix it.  ;)

In the Chromium bug linked, they only talk about resolving the canvas2d slowness.
I was wondering what that "step-like" behaviour was that they were talking about in the description.

I like our graph more; it looks more like what you should expect as you increase the number of things drawn.  For Chrome, I think this is a symptom of their off-thread GL rendering and command batching.  vsync too would play a part, though I didn't think that they were actually vsync'd (just attempt to match vsync via timers)?  We could get the same stair-step behaviour with a change (I believe #if 0'd out) in the rAF code that causes us to schedule on vsync-interval quantums.

Which particular benchmark were we faster on?  Cubes?  Back when it was doing 2d canvas, or when it was webgl?  What are the numbers now?
We used to be faster on Cubes, yes.  I don't know whether it was doing canvas or WebGL at the time.  See,3534-8.html for the benchmarks that got run and the scores back then...

Measuring now (on Windows) would be a good idea, yes.
Results on the Cubes Benchmark on Windows 7 and Intel HD3000:

Firefox stabilized around 50fps with 770 cubes.
Chrome 31 stabilized around 50fps with 1480 cubes.

Firefox only used 1 core the whole time, while Chrome used more than 1 beyond a certain number of cubes. Before that I think it used another core everytime a GC (or CC?) was necessary.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.