Closed Bug 498267 Opened 16 years ago Closed 14 years ago

TM: SFX and V8 are faster than TM on V8 test suite

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: gyuyoung, Assigned: gal)

References

Details

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ko; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 Build Identifier: I tested V8 test suite on both SFX and TM shell. The result has about 14 times difference. It seems we need to verify how to come the difference. I used sunspider test suite, which was included in webkit opensource. 1. Squirrelfish Extreme ============================================ RESULTS (means and 95% confidence intervals) -------------------------------------------- Total: 1946.4ms +/- 16.6% -------------------------------------------- v8: 1946.4ms +/- 16.6% crypto: 311.2ms +/- 14.3% deltablue: 526.8ms +/- 17.5% earley-boyer: 366.9ms +/- 16.4% raytrace: 302.7ms +/- 17.7% richards: 438.8ms +/- 17.6% 2. Tracemonkey ============================================ RESULTS (means and 95% confidence intervals) -------------------------------------------- Total: 28979.2ms +/- 16.1% -------------------------------------------- v8: 28979.2ms +/- 16.1% crypto: 4689.6ms +/- 17.3% deltablue: 7016.1ms +/- 16.5% earley-boyer: 2789.9ms +/- 15.1% raytrace: 3203.3ms +/- 15.9% richards: 11280.3ms +/- 16.5% SUT - OS : Ubuntu 8.04 - CPU : Intel Core2 Duo E7300 2.66 Ghz - Mem : 2GB - Source Ver. : SFX is based on 2009.06.14 (git://git.webkit.org/WebKit.git WebKit) , TMis based on 2009.06.14(http://hg.mozilla.org/mozilla-central) - Test : Run V8 test suite using each javascript shell Reproducible: Always
Summary: TM: On V8 test suite, SFX is 14 times faster than TM → TM: SFX is 14 times faster than TM on V8 test suite
Depends on: 498351
Depends on: 453730
3 things make us perform horribly on 3 of the 5 testcases: - instanceof (just fixed that) - arguments - call/callname into a call object on the stack We should schedule the last 2 items as high priority wanted. Also, the 14x doesn't reproduce for me, especially not on crypto. I think the reporter is running with JIT off (missing -j argument to the shell). But performance is bad and not good as it should. I will confirm the bug based on my own benchmarks.
Assignee: general → gal
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: TM: SFX is 14 times faster than TM on V8 test suite → TM: SFX and V8 are faster than TM on V8 test suite
Is it worth making JIT _on_ be the default for js.cpp ?
Yeah, I think it probably is; could use --disable-jit or -J to disable it? File that mother. :)
Yeah, he wouldn't be the first to fall into the trap. We could just flip the meaning of -j to disable jit. Thats a 1-liner and it almost makes sense "-j" == minus jit.
Except that it would break in this way everything that _currently_ has the JIT enabled...
Bug 498368 is filed.
It seems I didn't enable JIT of TM. When I use "-j", result of sunspider is about 3 times faster. So, I tested v8 test suite on TM after enabling JIT via "-j" option. If I still have mistakes, please let me know. The result is as below, - Tracemonkey with JIT ============================================ RESULTS (means and 95% confidence intervals) -------------------------------------------- Total: 22943.9ms +/- 0.6% -------------------------------------------- v8: 22943.9ms +/- 0.6% crypto: 1520.3ms +/- 1.4% deltablue: 13108.9ms +/- 1.1% earley-boyer: 3375.8ms +/- 0.4% raytrace: 3604.1ms +/- 0.8% richards: 1334.8ms +/- 0.4% - SFX with JIT ============================================ RESULTS (means and 95% confidence intervals) -------------------------------------------- Total: 1949.0ms +/- 16.6% -------------------------------------------- v8: 1949.0ms +/- 16.6% crypto: 304.1ms +/- 14.5% deltablue: 528.5ms +/- 17.6% earley-boyer: 370.6ms +/- 16.0% raytrace: 303.6ms +/- 16.9% richards: 442.2ms +/- 17.9% SUT - OS : Ubuntu 8.04 - CPU : Intel Core2 Duo E7300 2.66 Ghz - RAM : 2GB - Source Ver. : SFX is based on 2009.06.14 (git://git.webkit.org/WebKit.git WebKit) , TMis based on 2009.06.14(http://hg.mozilla.org/mozilla-central) - Test : Run V8 test suite using each javascript shell
Depends on: 504587
There are a lot of bugs tracking performance improvements for the individual v8 tests. Between those and the constant feedback at AWFY, this bug doesn't seem to have much use.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.