Last Comment Bug 785079 - Add Octane to arewefastyet.com
: Add Octane to arewefastyet.com
Status: RESOLVED FIXED
[js:t]
:
Product: Core
Classification: Components
Component: JavaScript Engine (show other bugs)
: Trunk
: x86 Mac OS X
: -- normal with 1 vote (vote)
: ---
Assigned To: general
:
Mentors:
http://octane-benchmark.googlecode.co...
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-23 07:56 PDT by Gary Kwong [:gkw] [:nth10sd]
Modified: 2013-01-18 14:52 PST (History)
16 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description Gary Kwong [:gkw] [:nth10sd] 2012-08-23 07:56:05 PDT
The Octane benchmark has been released to (possibly) replace v8 - it would be nice to have this implemented in arewefastyet.com

http://octane-benchmark.googlecode.com/svn/latest/index.html
Comment 1 Nicolas B. Pierron [:nbp] 2012-08-23 11:53:51 PDT
I would agree, but I would prefer if Mandreel benchmark can be either disabled or renamed.  We should not promote any proprietary software as a solution for the Web, and adding this test in benchmarks is like making advertisement for their product.

If we don't even want to keep the generated code either I would suggest to replace it by emscripten[1] generated code which is publicly available.

[1] https://github.com/kripken/emscripten/blob/master/LICENSE
Comment 2 Grant Galitz 2012-08-25 15:13:19 PDT
Octane does confirm what my own personal testing has shown about browsers running my JS GBC emulator though. I went through the copy they used for it and it seems to be a slightly out of date, but valid (modified) copy of https://github.com/grantgalitz/GameBoy-Online/blob/master/js/GameBoyCore.js . They're using demoscene demo ROM loaded in as a modified freeze state to test with. The test ROM in question uses scanline effects for the "3D" effect they mentioned. What this means is that the gfx JIT implemented has to render every scanline individually versus in batches. Firefox does run JS GBC slower than for what Chrome runs it as, probably due to regalloc issues in JM and lack of certain type optimizations that JSC/V8 perform.
Comment 3 Grant Galitz 2012-08-25 15:21:52 PDT
The initial load though is not what I think should be measured. Octane in my opinion does not run the GB-EMU test long enough for runtime performance to be measured over initial compilation time. This is what I'm afraid of: browsers trying to aim for initial compilation speed instead performance of the script running for hours on end.
Comment 4 Jim 2012-08-28 17:27:56 PDT
(In reply to Grant Galitz from comment #3)
> The initial load though is not what I think should be measured. Octane in my
> opinion does not run the GB-EMU test long enough for runtime performance to
> be measured over initial compilation time. This is what I'm afraid of:
> browsers trying to aim for initial compilation speed instead performance of
> the script running for hours on end.

Initial load time is important for the web. And it should compile in the background but ensure the page loads quick at first.
Comment 5 Grant Galitz 2012-08-29 07:12:10 PDT
(In reply to Jim from comment #4)
> (In reply to Grant Galitz from comment #3)
> > The initial load though is not what I think should be measured. Octane in my
> > opinion does not run the GB-EMU test long enough for runtime performance to
> > be measured over initial compilation time. This is what I'm afraid of:
> > browsers trying to aim for initial compilation speed instead performance of
> > the script running for hours on end.
> 
> Initial load time is important for the web. And it should compile in the
> background but ensure the page loads quick at first.

When I coded the JS GameBoy Color emulator, I specifically targeted runtime performance, not initialization speed. Things like the LFSR white noise cache and various ROM/RAM arrays have to be initialized on startup. That can take quite a bit of time and does not relay the performance of the interpreter loop itself. What I'm trying to say is that they should measure more of the gb-emu bench post-init, rather than a short period that includes the initial startup.
Comment 6 Grant Galitz 2012-08-29 07:17:38 PDT
JITs also have to warm themselves up into optimizing the dispatch table model used for the opcode/memory read/memory write/lcd scanline handling. We do someArray[someArrayIndex](this, arg1, arg2, ...) a lot in the code, as I purposefully avoided using switches as much as possible. A JIT will either:

a) Order the array lookups for most likely as checked first.

b) Create a dispatch table and use it if conditions are met (JITs might have a minimum call count before compiling it to a dispatch, as it can be a costly compilation.).

c) At worst, do a normal style lookup based on a search algo of choice.
Comment 7 Grant Galitz 2012-08-29 07:20:26 PDT
We use recursive dispatch calling in some places, so JITs MUST be able to handle that properly. We pass the "this" reference around a lot and access it as "parentObj" inside the called into function.
Comment 8 Nicholas Nethercote [:njn] 2012-09-10 20:29:22 PDT
BTW, I think we should show the old V8 tests separately from the new ones added to Octane.  See https://blog.mozilla.org/nnethercote/2012/08/24/octane-minus-v8/ for more.
Comment 9 Guilherme Lima 2013-01-18 14:50:41 PST
Octane is now part of AWFY!
Comment 10 Gary Kwong [:gkw] [:nth10sd] 2013-01-18 14:52:51 PST
Obviously fixed then. :) (or WFM but whatever.) Thanks!

Note You need to log in before you can comment on or make changes to this bug.