696405 - V8 regression for raytrace and regexp

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Gregor Wagner [:gwagner]]

I was looking at the v8 benchmark scores for a nightly build from end of 8/24 and today and saw regressions on raytrace and regexp on my MBP. The overall score is much better. 

curr:
Score: 5930
Richards: 7703
DeltaBlue: 8086
Crypto: 12119
RayTrace: 3499
EarleyBoyer: 7418
RegExp: 1593
Splay: 8263

8/24
Score: 5098
Richards: 7982
DeltaBlue: 5137
Crypto: 8494
RayTrace: 4030
EarleyBoyer: 4854
RegExp: 2092
Splay: 6283

Are these known regressions or should we investigate?

Comment 1

[Andrew McCreight [:mccr8]]

This brings up the question if we have any automated regression finding for individual V8 benchmarks.  Though I suppose most of the time you won't improve one and worsen another.

Comment 2

[Brian Hackett [Laid off!]]

There is a patch for raytrace in bug 638794 (needs a rewrite before it can land, though), and numbers for how that affects the benchmark in bug 642002.

I don't know what's going on with the regexp benchmark.  I thought it would be improved by bug 663138 but it wasn't, and now think this benchmark is pretty much totally determined by actual regexp performance (which makes sense, I guess).

Comment 3

[Chris Leary [:cdleary] (not checking bugmail)]

Attachment: x86 TM+JM on V8-RegExp

I've made some plots using AWFY data for Gregor's date range.

Comment 4

[Chris Leary [:cdleary] (not checking bugmail)]

Attachment: x86 JM+TI on V8-RegExp

Comment 5

[Chris Leary [:cdleary] (not checking bugmail)]

Attachment: x64 TM+JM on V8-RegExp

Comment 6

[Chris Leary [:cdleary] (not checking bugmail)]

There's no data on x64 JM+TI available through AWFY. It looks like the peak performance of TM (around Sept 25) was a few hundred points better than the peak performance of JM+TI (around Sept 25 as well).

Comment 7

[Chris Leary [:cdleary] (not checking bugmail)]

That aside, it does look like we took a regression around Oct 07 of ~50 points, which is when the lazy compilation landed. I should figure out why. Filed bug 696524.

Comment 8

[Gregor Wagner [:gwagner]]

(In reply to Gregor Wagner [:gwagner] from comment #0)

10/21
> RayTrace: 3499
> RegExp: 1593
> 
> 8/24
> RayTrace: 4030
> RegExp: 2092


With the current nightly it seems that it regressed even more:

12/12
RayTrace: 3290
RegExp: 1472