739016 - Performance on Julia language's math operations JIT benchmark

Status: UNCONFIRMED

(Core :: JavaScript Engine, defect)

Description

[timbugzilla]

Julia is a new JIT compiled language for mathematical/scientific programming, that is intended to be faster than Matlab etc, while offering more expressiveness and convenience.

http://julialang.org

As such, the developers have developed a small set of micro-benchmarks for numerical performance.

https://github.com/JuliaLang/​julia/blob/master/test/perf/​perf.js

Benchmarks are: recursive Fibonacci series, parsing for integers, Mandelbrot set generation, "quicksort", slow pi series, random matrix statistics, and random matrix multiplication.

V8 v3.6.6.11 performs quite well on this, beating C++ on one benchmark and Julia on another. Performance comparisons are on site main page:

http://julialang.org

Their dev-list is here:

https://groups.google.com/​forum/?fromgroups#!forum/​julia-dev


-----------------------------


Boris Zabarsky provided some analysis at the moz.dev.tech.js-engine thread:

https://groups.google.com/d/msg/mozilla.dev.tech.js-engine/Ct6JsfJEjdI/1Vz5LJxnkXYJ

	> Julia is a new JIT compiled language for mathematical programming, that is intended to be faster than Matlab etc, while offering more expressiveness and convenience.
	>
	> As such, the developers have developed a small set of micro-benchmarks for numerical performance.
	>
	> https://github.com/JuliaLang/​julia/blob/master/test/perf/​perf.js
	>
	> V8 v3.6.6.11 performs quite well on this, beating C++ on one benchmark and Julia on another.

	Huh.  When I run this test in V8 v3.7.3 I get an exception when calling
	set() in the rand_mat_stat test.

	In any case, a comparison of a reasonably recent Spidermonkey with -m -n
	with V8 on my hardware (excluding the rand_mat_stat test they throw on)
	shows:

		Spidermonkey:

		javascript,fib,0.185
		javascript,parse_int,0.366
		javascript,mandel,0.962
		javascript,quicksort,7.4
		javascript,pi_sum,33.8
		javascript,rand_mat_mul,11061

		V8:

		javascript,fib,0.113
		javascript,parse_int,0.498
		javascript,mandel,0.627
		javascript,quicksort,2.2
		javascript,pi_sum,34.5
		javascript,rand_mat_mul,9940

	Note that pi_sum (which we beat V8 on) is the one that V8 and Julia beat
	C++ on.

	On a side note, the "quicksort" test in JS is not actually using
	quicksort; it's using Array.prototype.sort with the callback
	function(a,b) { return a-b; }.  That's covered by bug 715181 and the
	bugs linked from the comments there.

	I looked at the C++ and Julia code for the "quicksort" test, and the C++
	and Julia and Python versions of this are both hand-coded naive
	quicksort.  This is covered by https://github.com/JuliaLang/​julia/issues/562

	Fwiw, recoding the quicksort in exactly the same way as the Julia or
	Python or C++ version makes the times look like this:

		Spidermonkey:
		   javascript,quicksort,1.08

		V8:
		   javascript,quicksort,1.84

	There are various other issues in the JS tests (missing var all over
	leading to global variable accesses when I think they meant to use
	function local variables, for example)....

	-Boris 

	-----------------------------------------

It seems that V8 has some performance advantages on this benchmark.