705313 - Regression in Kraken benchmark in todays (2011-11-25) Nightly build

Status: RESOLVED INVALID

(Core :: JavaScript Engine, defect)

Description

[blechri]

User Agent: Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:11.0a1) Gecko/20111125 Firefox/11.0a1
Build ID: 20111125033229



Actual results:

Regression in todays (2011-11-25) Nightly build (compared to 2011-11-23) in Kraken js benchmark. 4118.1ms +/- 0.4% ---> 4311.4ms +/- 0.5%
Main reason: desaturate: 293.7ms +/- 0.5% ---> 425.0ms +/- 6.2%
Full results: http://goo.gl/8lp72 (2011-11-23), http://goo.gl/wzEF7 (2011-11-25).
Restarted and reran the test with similar results. Before today the score has been below 4200 for the last tree weeks (at least).

Comment 1

[David Mandelin [:dmandelin]]

(In reply to blechri from comment #0)
> User Agent: Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:11.0a1)
> Gecko/20111125 Firefox/11.0a1
> Build ID: 20111125033229
> 
> 
> 
> Actual results:
> 
> Regression in todays (2011-11-25) Nightly build (compared to 2011-11-23) in
> Kraken js benchmark. 4118.1ms +/- 0.4% ---> 4311.4ms +/- 0.5%
> Main reason: desaturate: 293.7ms +/- 0.5% ---> 425.0ms +/- 6.2%
> Full results: http://goo.gl/8lp72 (2011-11-23), http://goo.gl/wzEF7
> (2011-11-25).
> Restarted and reran the test with similar results. Before today the score
> has been below 4200 for the last tree weeks (at least).

Would you mind retesting with a new nightly? I just tested 11-23 vs 11-28 and didn't get a regression for that pair, so I'm thinking it might have already been fixed.

Comment 2

[blechri]

For me it still stands. With 11-28 I get very similar results: 4312.6ms +/- 0.7%. Full results here: http://goo.gl/GDtrV
I will update to todays build (11-29) and try again.

Comment 3

[blechri]

Slightly better in todays build but not due to desaturation. 4291.9ms +/- 0.2%. Full results: http://goo.gl/tM0yu

Comment 4

[blechri]

Haven't had the time to update until now. There has been a change between the 12-01 and the 12-04 build where the problem with desaturate have been solved, it now again lies around 290ms, but the general score for kraken is still higher than it was before the regression with 4273,2ms for the 12-04 build (4312 for the 12-01 build).

Another thing that happened between 12-01 and 12-04 is that sunspider regressed from 241,3ms to 296,4ms but I'll file a separate bug for that if I can't find any already up.

Not much has changed in the 12-05 and 12-06 builds.

Comment 5

[blechri]

The sunspider regression was bug 707515

Comment 6

[Guilherme Lima]

Not valid anymore.