549504 - JM: Benchmark compilation time

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[David Mandelin [:dmandelin]]

We need to measure how much time we spend compiling, including relative to execution and relative to the speedup we are getting. JMFLAGS=profile does part of this; let's extend it to count the total time spent compiling and then run that through our benchmarks.

Comment 1

[David Mandelin [:dmandelin]]

SunSpider results:

                            bytecode size  native size       comp time
3d-cube.js                           4195        46359          626 us
3d-morph.js                           349         5956          120 us
3d-raytrace.js                       4878        49323          699 us
access-binary-trees.js                453         8153          151 us
access-fannkuch.js                    610         5150          151 us
access-nbody.js                      1465        15023          253 us
access-nsieve.js                      228         1949           74 us
bitops-3bit-bits-in-byte.js           176         1993           76 us
bitops-bits-in-byte.js                145         1316           69 us
bitops-bitwise-and.js                  53          987           54 us
bitops-nsieve-bits.js                 263         2927           88 us
controlflow-recursive.js              282         3893           93 us
crypto-aes.js                        4820        46822          679 us
crypto-md5.js                        3468        26489          441 us
crypto-sha1.js                       1196        12812          246 us
date-format-tofte.js                 1389        12616          214 us
date-format-xparb.js                 1614        17138          246 us
math-cordic.js                        498         4964          114 us
math-partial-sums.js                  442         5757          106 us
math-spectral-norm.js                 544         5146          118 us
regexp-dna.js                         341         5828          103 us
string-base64.js                     1398        16947          268 us
string-fasta.js                       721         8099          159 us
string-tagcloud.js                  48794       556535         7469 us
string-unpack-code.js                1287        13188          260 us
string-validate-input.js              800         6798          155 us
                                    80409       882168        13032 us

Total compilation time is 13ms, so I don't think we have to worry about that too much for now. string-tagcloud must call eval a lot. Total code size is under 1 MB, only about 300k if we leave out string-tagcloud.

Comment 2

[David Mandelin [:dmandelin]]

v8 (SunSpider's version) results:

                            bytecode size  native size       comp time

v8-crypto.js                        12218       135719         2763 us
v8-deltablue.js                      4997        57550          904 us
v8-earley-boyer.js                  30894       329377         4629 us
v8-raytrace.js                       7044        88591         1201 us
v8-regexp.js                        22137       208716         2970 us
v8-richards.js                       2534        32359          463 us
v8-splay.js                          1636        17704          386 us
                                    81460       870016        13316 us

Pretty similar to SS.

Comment 3

[Brendan Eich [:brendan]]

(In reply to comment #1)
...
>                                     80409       882168        13032 us

Is order-of-magnitude size increase expected? It's ok, just wondering what the competition uses, if we know.

> Total compilation time is 13ms, so I don't think we have to worry about that
> too much for now. string-tagcloud must call eval a lot. Total code size is
> under 1 MB, only about 300k if we leave out string-tagcloud.

Our eval caches scripts -- are you hitting that cache and caching JM code too?

/be