1028242 - [meta] Improve octane score on windows

Status: NEW

(Core :: JavaScript Engine: JIT, defect)

Description

[Hannes Verschore [:h4writer]]

On most platforms we are close to google chrome performance on octane, except for windows. This is for both the shell and browser. See:
http://arewefastyet.com/#machine=17

Google chrome is 25% higher than we are on octane (browser and shell).

To give an idea where we are at. Here the score of the subtests on octane between chrome and firefox browser:

>	        Chrome	Firefox	     Diff
> SplayLatency   26925     9208  -192.41%
> RegExp          5007     2444  -104.87%
> PdfJS          21931    15205   -44.24%
> Raytrace       62036    43350   -43.10%
> DeltaBlue      50772    36075   -40.74%
> EarleyBoyer    41404    30227   -36.98%
> Splay          18365    14599   -25.80%
> Gameboy        65940    52506   -25.59%
> Typescript     21399    17066   -25.39%
> Box2D          40974    33661   -21.73%
> Crypto         26473    24595    -7.64%
> CodeLoad       16655    15707    -6.04%
> Richards       29852    29134    -2.46%
> Mandreel       26014    26379     1.38%
> NavierStokes   26266    26813     2.04%
> MandreeLatency 21826    31661    31.06%
> Zlib           28807    47184    38.95%

Now we have good and bad subtests. But most are already known and getting already tackled. So this isn't a good summery to know where we need to look for specific Window things.
(e.g. SplayLatency is just a bad benchmark and we think benchmark is broken. See bug 958492)

>              Diff on OSX   Diff on Windows  Relative change
> Raytrace          25.97%           -43.10%          -69.07%
> RegExp           -57.41%          -104.87%          -47.46%
> Box2D              2.46%           -21.73%          -24.19%
> PdfJS            -20.51%           -44.24%          -23.73%
> Gameboy           -4.04%           -25.59%          -21.55%
> EarleyBoyer      -20.12%           -36.98%          -16.86%
> DeltaBlue        -26.65%           -40.74%          -14.09%
> Mandreel          13.78%             1.38%          -12.40%
> Zlib              51.23%            38.95%          -12.28%
> Splay            -16.01%           -25.80%           -9.79%
> Typescript       -15.77%           -25.39%           -9.62%
> CodeLoad           3.19%            -6.04%           -9.23%
> Crypto            -3.97%            -7.64%           -3.67%
> SplayLatency    -189.68%          -192.41%           -2.73%
> MandreeLatency    31.94%            31.06%           -0.88%
> NavierStokes      -0.60%             2.04%            2.64%
> Richards         -20.31%            -2.46%           17.85%

This portraits the relive change in performance difference over OSX (chrome - firefox) and Windows (chrome - firefox). Also sorted.

So this shows the biggest offender is Raytrace. Our score on OSX is really good, but for some reason it isn't on Windows. Also for some reason the regexp performance drops going to Windows. Box2D is a bit hard. Since it tends to be bimodal. So we are probably for some small time on a sweet spot in OSX, but we also have been bad on it before. Also for some reason our Zlib performance has dropped compared to chrome between OSX and Windows. Are we doing something something different on windows?

Comment 1

[Luke Wagner [:luke]]

(In reply to Hannes Verschore [:h4writer] from comment #0)
> Also for some reason our Zlib performance has dropped compared to chrome between OSX and Windows.

x64 has more efficient heap access (no bounds checks) than x86 (which branches on every access).

One thing that would be nice to see is if FF is relatively slower than Chrome just on Windows or on all x86 platforms.  That is, do you get a similar relative change when you run on x86 Linux/OSX?  (Note: to run FF as 32-bit on OSX, just start with 'arch -32 firefox ...'.

Comment 2

[Hannes Verschore [:h4writer]]

(In reply to Luke Wagner [:luke] from comment #1)
> (In reply to Hannes Verschore [:h4writer] from comment #0)
> > Also for some reason our Zlib performance has dropped compared to chrome between OSX and Windows.
> 
> x64 has more efficient heap access (no bounds checks) than x86 (which
> branches on every access).

OSX is tested on 32bit, windows too.

(In reply to Luke Wagner [:luke] from comment #1)
> One thing that would be nice to see is if FF is relatively slower than
> Chrome just on Windows or on all x86 platforms.  That is, do you get a
> similar relative change when you run on x86 Linux/OSX?  (Note: to run FF as
> 32-bit on OSX, just start with 'arch -32 firefox ...'.

On almost all platforms (except FFOS) we are a little bit slower than chrome. But really small. So that's on OSX, that is on fennec. That is on ARM...

It is only on Windows we have a difference of 25% between Chrome and Firefox

Comment 3

[Luke Wagner [:luke]]

(In reply to Hannes Verschore [:h4writer] from comment #2)
> It is only on Windows we have a difference of 25% between Chrome and Firefox

Oh wow, that is surprising then :)

Comment 4

[Hugh Nougher [:Hughman]]

I dont know how possible this idea is but I think it would be good to have a graph on AWFY showing this difference percent between platforms. The idea would be to make it obvious when there is a difference like this between platforms and when that started occurring. Just showing the difference to chrome on each platform in one graph is likely enough, understanding that the graph can pass zero.

I realise this graph would be contextual with chrome performance between platforms also playing a role but it would lessen the chance of this happening again given most people only would check one or two AWFY machine graphs out of the many available.