1190723 - BitcoinBench perf regressions on SHA, Scrypt, AddressChain, and AES tests

Status: NEW

(Core :: JavaScript Engine: JIT, defect, P3)

Description

[Chris Peterson [:cpeterson]]

Nightly 42 appears to have regressed on BitcoinBench across the board, but particularly on the SHA, Scrypt, AddressChain, and AES tests.

https://www.bitgo.com/bitcoinbenchmark/

Here is a Google Spreadsheet comparing the test results for different Firefox and Chrome versions (run on my MacBook Pro):

https://docs.google.com/spreadsheets/d/18vUP-yLhAxgUWE9HSUn-_iJcXnMH3AtQDgt3Xb5hm4E/

Browser             Score   Address Creation    Sign And Verify SHA Scrypt  AddressChain    AES
ESR 31              179     139                 147             248 280     155             149
Nightly 32          262     170                 279             411 266     221             231
Firefox 39          243     216                 306             319 313     236             132
Firefox 39          225     212                 210             311 321     217             133
Beta 40             242     202                 300             317 322     238             134
Aurora 41           243     198                 292             422 320     226             126
Nightly 42          234     196                 288             393 302     222             123
Chrome 35           184     157                 232             243 210     165             199
Chrome 44           230     160                 235             403 275     180             196
Chrome Canary 46    223     157                 168             424 310     175             201

Comment 1

[Jan de Mooij [:jandem]]

Is this better if you set JSGC_DISABLE_POISONING=1 on Nightly?

The difference is pretty small..

Comment 2

[Guilherme Lima]

I guess all the results are within noise, except for AES (231 -> 123 - from Fx 32 to Fx 39)

Comment 3

[Chris Peterson [:cpeterson]]

The AES test is the only one that isn't super noisy. I tried to bisect the AES regression between Firefox 32 to 39, but I wasn't able to isolate an exact changeset.

Between 32 and 39, there seem to be two regression points. On my MacBook Pro, AES seems to drop from ~170 to ~150 in this range of Nightly 34. Could this be a regression from NewString bug 1039551?

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=99f694d1b50c&tochange=35f3fa435d2c

The second drop is from ~150 to ~130, which seems to have happened in this range of Nightly 36. Could this be a regression from bytecode bug 1101905?

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=cef590a6f946&tochange=17de0f463944

Comment 4

[Tom S. (please needinfo tschuster)]

Firefox

Score 139
Address Creation 110
Sign And Verify 171
SHA 210
Scrypt 183
AddressChain 130
AES 75.0


Chrome

Address Creation 87.8
Sign And Verify 131
SHA 220
Scrypt 197
AddressChain 98.5
AES 126

Our AES is still much slower than Chrome, Sign and Verify is also slower. The rest is about equal or better. Somebody should investigate the AES test.