Last Comment Bug 511316 - TM: Current SunSpider Performance Comparison
: TM: Current SunSpider Performance Comparison
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: JavaScript Engine (show other bugs)
: unspecified
: x86 Mac OS X
: -- normal (vote)
: ---
Assigned To: general
:
: Jason Orendorff [:jorendorff]
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-08-18 18:37 PDT by Gregor Wagner
Modified: 2011-11-16 16:00 PST (History)
4 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description Gregor Wagner 2009-08-18 18:37:17 PDT
From = Tracemonkey
To   = WebKit


TEST                   COMPARISON            FROM                 TO         

=============================================================================

** TOTAL **:           1.72x as fast     748.5ms +/- 0.2%   435.3ms +/- 0.4%  

=============================================================================

  3d:                  2.00x as fast     113.2ms +/- 0.5%    56.6ms +/- 0.9%  
    cube:              1.84x as fast      30.2ms +/- 1.0%    16.4ms +/- 2.3%  
    morph:             -                  22.3ms +/- 1.5%    22.0ms +/- 2.2% 
    raytrace:          3.34x as fast      60.7ms +/- 0.6%    18.2ms +/- 1.7%  

  access:              2.32x as fast     110.9ms +/- 0.4%    47.7ms +/- 1.4%  
    binary-trees:      5.67x as fast      31.2ms +/- 1.0%     5.5ms +/- 9.2%  
    fannkuch:          2.67x as fast      49.7ms +/- 0.7%    18.6ms +/- 2.0%  
    nbody:             1.42x as fast      18.5ms +/- 2.0%    13.0ms +/- 0.0%  
    nsieve:            1.085x as fast     11.5ms +/- 3.3%    10.6ms +/- 3.5%  

  bitops:              1.171x as fast     30.1ms +/- 1.8%    25.7ms +/- 1.3%  
    3bit-bits-in-byte: *2.73x as slow*     1.5ms +/- 25.1%     4.1ms +/- 5.5%  
    bits-in-byte:      -                  7.3ms +/- 4.7%     7.1ms +/- 3.2% 
    bitwise-and:       *2.27x as slow*     2.2ms +/- 13.7%     5.0ms +/- 0.0% 
    nsieve-bits:       2.01x as fast      19.1ms +/- 1.2%     9.5ms +/- 4.0%  

  controlflow:         7.07x as fast      29.0ms +/- 0.0%     4.1ms +/- 5.5%  
    recursive:         7.07x as fast      29.0ms +/- 0.0%     4.1ms +/- 5.5%  

  crypto:              1.57x as fast      45.1ms +/- 1.2%    28.8ms +/- 2.0%  
    aes:               1.71x as fast      25.0ms +/- 2.3%    14.6ms +/- 2.5%  
    md5:               1.71x as fast      13.0ms +/- 0.0%     7.6ms +/- 4.9%  
    sha1:              1.076x as fast      7.1ms +/- 3.2%     6.6ms +/- 5.6%  

  date:                1.90x as fast     110.1ms +/- 0.6%    57.8ms +/- 0.5%  
    format-tofte:      2.02x as fast      52.6ms +/- 0.7%    26.0ms +/- 0.0%  
    format-xparb:      1.81x as fast      57.5ms +/- 0.7%    31.8ms +/- 0.9%  

  math:                *1.47x as slow*    25.9ms +/- 2.0%    38.0ms +/- 0.0%  
    cordic:            *1.43x as slow*     9.1ms +/- 2.5%    13.0ms +/- 0.0%  
    partial-sums:      *1.59x as slow*    11.3ms +/- 3.1%    18.0ms +/- 0.0%  
    spectral-norm:     *1.27x as slow*     5.5ms +/- 6.8%     7.0ms +/- 0.0%  

  regexp:              1.58x as fast      34.2ms +/- 0.9%    21.7ms +/- 1.6%  
    dna:               1.58x as fast      34.2ms +/- 0.9%    21.7ms +/- 1.6%  

  string:              1.61x as fast     250.0ms +/- 0.3%   154.9ms +/- 0.6%  
    base64:            *1.23x as slow*    14.3ms +/- 2.4%    17.6ms +/- 2.1%  
    fasta:             1.63x as fast      50.9ms +/- 0.4%    31.2ms +/- 1.0%  
    tagcloud:          2.34x as fast      80.9ms +/- 0.8%    34.5ms +/- 1.1%  
    unpack-code:       1.78x as fast      79.8ms +/- 0.4%    44.9ms +/- 0.5%  
    validate-input:    *1.108x as slow*   24.1ms +/- 0.9%    26.7ms +/- 1.8%
Comment 1 Gregor Wagner 2009-08-18 18:38:09 PDT
And v8:

--------------------------------------------
Total:                  466.6ms +/- 0.2%
--------------------------------------------

  3d:                    76.2ms +/- 0.6%
    cube:                20.6ms +/- 1.8%
    morph:               34.1ms +/- 0.7%
    raytrace:            21.5ms +/- 1.8%

  access:                33.1ms +/- 1.6%
    binary-trees:         3.0ms +/- 0.0%
    fannkuch:            11.7ms +/- 3.0%
    nbody:               14.7ms +/- 2.3%
    nsieve:               3.7ms +/- 9.3%

  bitops:                31.6ms +/- 2.2%
    3bit-bits-in-byte:    2.8ms +/- 10.8%
    bits-in-byte:         7.3ms +/- 4.7%
    bitwise-and:          8.0ms +/- 0.0%
    nsieve-bits:         13.5ms +/- 2.8%

  controlflow:            2.4ms +/- 15.4%
    recursive:            2.4ms +/- 15.4%

  crypto:                31.6ms +/- 1.6%
    aes:                 14.0ms +/- 0.0%
    md5:                  9.1ms +/- 2.5%
    sha1:                 8.5ms +/- 4.4%

  date:                  53.9ms +/- 0.4%
    format-tofte:        30.0ms +/- 0.0%
    format-xparb:        23.9ms +/- 0.9%

  math:                  38.8ms +/- 1.5%
    cordic:              15.5ms +/- 2.4%
    partial-sums:        16.4ms +/- 2.3%
    spectral-norm:        6.9ms +/- 3.3%

  regexp:                22.6ms +/- 1.6%
    dna:                 22.6ms +/- 1.6%

  string:               176.4ms +/- 0.5%
    base64:              17.4ms +/- 2.1%
    fasta:               24.0ms +/- 0.0%
    tagcloud:            45.0ms +/- 0.0%
    unpack-code:         63.2ms +/- 0.5%
    validate-input:      26.8ms +/- 2.5%
Comment 2 Robert Sayre 2009-08-19 19:13:39 PDT
run this with the -v8 tests in the SS harness
Comment 3 Gregor Wagner 2009-08-20 10:55:30 PDT
Results for the v8 SS v8 benchmark suite:

TM:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:             12852.6ms +/- 0.6%
--------------------------------------------

  v8:              12852.6ms +/- 0.6%
    crypto:          742.6ms +/- 2.0%
    deltablue:      6290.1ms +/- 1.4%
    earley-boyer:   1820.5ms +/- 0.8%
    raytrace:       1205.8ms +/- 1.1%
    regexp:          919.6ms +/- 0.4%
    richards:        692.5ms +/- 0.2%
    splay:          1181.5ms +/- 0.4%


V8:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:             2004.4ms +/- 0.0%
--------------------------------------------

  v8:              2004.4ms +/- 0.0%
    crypto:         209.9ms +/- 0.1%
    deltablue:      247.1ms +/- 0.2%
    earley-boyer:   203.3ms +/- 0.2%
    raytrace:        90.2ms +/- 0.3%
    regexp:         259.1ms +/- 0.1%
    richards:       266.1ms +/- 0.1%
    splay:          728.7ms +/- 0.1%


WebKit:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:             2771.8ms +/- 0.1%
--------------------------------------------

  v8:              2771.8ms +/- 0.1%
    crypto:         299.1ms +/- 0.2%
    deltablue:      437.0ms +/- 0.3%
    earley-boyer:   338.7ms +/- 0.4%
    raytrace:       143.0ms +/- 0.3%
    regexp:         236.3ms +/- 0.1%
    richards:       418.1ms +/- 0.3%
    splay:          899.6ms +/- 0.1%
Comment 4 Gregor Wagner 2009-08-21 13:56:59 PDT
Opcodes Executed in interpreter: jit enabled vs. not enabled

	          	-j	interp		int	
3d-cube			336203	7157216		4.70%	
3d-morph		432	6243916 	0.01%	
3d-raytrace		54709	4600015 	1.19%	
access-binary-trees	1556219	1556554 	99.98%	
access-fannkuch		2533	20525186	0.01%	
access-nbody		3750	5978379 	0.06%	
access-nsieve		276	6853577 	0.00%	
bitops-3bit-bits-	145	6277522 	0.00%	
bitops-bits-in-byte	140	10666272	0.00%	
bitops-bitwise-and	34	6600011 	0.00%	
bitops-nsieve-bits	204	12098457	0.00%	
controlflow-recursive	2630964	2630964 	100.00%	
crypto-aes		19741	5343957 	0.37%	
crypto-md5		32277	3190961 	1.01%	
crypto-sha1		5734	3372824 	0.17%	
date-format-tofte	647619	3073577 	21.07%	
date-format-xparb	11776	1014427 	1.16%	
math-cordic		465	10425150	0.00%	
math-partial-sums	1091	3841279 	0.03%	
math-spectral-norm	1349	4359144 	0.03%	
regexp-dna		380	464             81.90%	
string-base64		1106	2614005 	0.04%	
string-fasta		922	4252539 	0.02%	
string-tagcloud		610375	1290054 	47.31%	
string-unpack-code	330343	852072  	38.77%	
string-validate-input	935	1758101 	0.05%	

V8 benchmark suite
crypto		161136		385941200	0.04%		
deltablue	118333203	165843447	71.35%		
earley-boyer	72436057	78656566	92.09%		
raytrace	14877779	24523631	60.67%		
regexp		16822		3011528		0.56%			
richards	26361		267244563	0.01%			
splay		13955721	17723400	78.74%
Comment 5 Gregor Wagner 2009-08-21 14:07:20 PDT
I hope somehow more readable. Export from GDocs...

                        -j      interp   int
3d-cube               336203   7157216    4.70%   
3d-morph                 432   6243916    0.01%   
3d-raytrace            54709   4600015    1.19%   
access-binary-trees  1556219   1556554    99.98%   
access-fannkuch         2533   20525186   0.01%   
access-nbody            3750   5978379    0.06%   
access-nsieve            276   6853577    0.00%   
bitops-3bit-bits-        145   6277522    0.00%   
bitops-bits-in-byte      140   10666272   0.00%       
bitops-bitwise-and        34   6600011    0.00%   
bitops-nsieve-bits       204   12098457   0.00%   
controlflow-recursive 2630964  2630964    100.00%   
crypto-aes             19741   5343957    0.37%   
crypto-md5             32277   3190961    1.01%   
crypto-sha1             5734   3372824    0.17%   
date-format-tofte     647619   3073577    21.07%   
date-format-xparb      11776   1014427    1.16%   
math-cordic              465   10425150   0.00%   
math-partial-sums       1091   3841279    0.03%   
math-spectral-norm      1349   4359144    0.03%   
regexp-dna               380       464    81.90%   
string-base64           1106   2614005    0.04%   
string-fasta             922   4252539    0.02%   
string-tagcloud       610375   1290054    47.31%   
string-unpack-code    330343   852072     38.77%   
string-validate-input    935   1758101    0.05%   

V8 benchmark suite
crypto           161136   385941200    0.04%      
deltablue     118333203   165843447   71.35%      
earley-boyer   72436057   78656566    92.09%      
raytrace       14877779   24523631    60.67%      
regexp            16822   3011528      0.56%         
richards          26361   267244563    0.01%         
splay          13955721   17723400    78.74%
Comment 6 Gregor Wagner 2009-08-25 14:23:15 PDT
Numbers for v8 benchmarks without tracing. 
DeltaBlue is 2 seconds faster if I disable tracing.

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:             16380.2ms +/- 0.1%
--------------------------------------------

  v8:              16380.2ms +/- 0.1%
    crypto:         2348.2ms +/- 0.1%
    deltablue:      4026.1ms +/- 0.3%
    earley-boyer:   1670.6ms +/- 0.2%
    raytrace:       1377.2ms +/- 0.8%
    regexp:         1140.4ms +/- 0.1%
    richards:       4690.8ms +/- 0.3%
    splay:          1126.9ms +/- 0.2%
Comment 7 Ryan VanderMeulen [:RyanVM] 2011-11-16 16:00:29 PST
I'm going to go ahead and assume that AWFY fixed this :-)

Note You need to log in before you can comment on or make changes to this bug.