OUT-OF-DATE! Read http://shootout.alioth.debian.org/      

- all benchmarks - binary-trees chameneos-redux fannkuch fasta k-nucleotide mandelbrot meteor-contest n-body nsieve nsieve-bits partial-sums pidigits recursive regex-dna reverse-complement spectral-norm startup sum-file thread-ring - all languages - Ada 2005 GNAT BASIC FreeBASIC C CINT C GNU gcc C Intel C# Mono C++ GNU g++ C++ Intel CAL Clean D Digital Mars Eiffel SmartEiffel Erlang HiPE F# Mono Forth bigForth Forth GNU GForth Fortran G95 Fortran Intel Groovy Haskell GHC Icon Java 6 -client Java 6 -server Java 6 -Xint Java 6 -Xms64m Java GNU gcj JavaScript Rhino JavaScript SpiderMonkey Lisp SBCL Lua Lua LuaJIT Mercury Mozart/Oz Nice Oberon-2 OO2C OCaml Pascal Free Pascal Perl PHP Pike Prolog SWI Prolog YAP Python CPython Python IronPython Python Psyco Rebol Ruby 1.9 Ruby JRuby Ruby MRI Scala Scheme Chicken Scheme Ikarus Scheme PLT Smalltalk GNU Smalltalk Squeak Smalltalk VisualWorks SML MLton SML SML/NJ Tcl Zonnon Mono

 How big is the measured performance difference?

Each chart bar shows how many times more Time or how many times more Memory one unidentified ↓ binary-trees program used, compared to the benchmark program that used least Time or the program that used least Memory.

 binary-trees benchmark N=16

This table shows 5 measurements - CPU Time, Elapsed Time, Memory, Code and ≈ CPU Load.

Compare how much Memory the binary-trees programs used - sort Memory KB. Compare how much Code the benchmark programs used - sort Code B

Column × shows how many times more each program used compared to the benchmark program that used least.

		sort	sort	sort	sort
×	Program Source Code	CPU secs	Elapsed secs	Memory KB	Code B	≈ CPU Load
1.0	C GNU gcc #3	0.85		3,900	758
1.9	C GNU gcc #2	1.60		4,512	743
3.4	Java 6 -Xms64m #2	2.89		39,436	603
3.8	SML MLton #2	3.20		45,760	530
3.9	Haskell GHC	3.35		9,152	512
4.2	Erlang HiPE	3.58		18,096	441
4.4	Lisp SBCL #2	3.72		36,324	553
4.5	C GNU gcc	3.79		4,536	706
4.6	Pascal Free Pascal	3.92		4,224	769
4.7	Clean #3	3.98		16,868	539
4.8	C Intel	4.06		4,576	706
4.8	Eiffel SmartEiffel #2	4.11		11,652	630
5.1	D Digital Mars #2	4.34		4,724	705
5.3	C++ GNU g++ #2	4.47		6,996	541
5.6	Pascal Free Pascal #2	4.73		4,236	629
5.7	Scala #4	4.80		54,684	536
5.7	OCaml	4.83		8,732	469
5.7	C++ Intel #2	4.87		6,524	541
5.8	BASIC FreeBASIC #2	4.90		4,852	723
5.9	Fortran Intel	5.03		4,508	826
6.1	Ada 2005 GNAT	5.14		6,608	955
6.6	Mercury	5.59		28,944	910
7.0	Scala	5.92		58,656	529
7.0	#2	5.93		20,708	603
8.1	Java 6 -server #2	6.89		26,808	603
8.2	Java GNU gcj #2	6.95		24,032	603
8.7	Java 6 -client #2	7.36		26,136	603
8.8	Nice	7.45		23,984	485
8.9	CAL	7.58		27,308	736
9.6	Smalltalk VisualWorks	8.11		26,852	722
9.7	C# Mono	8.25		14,584	610
12	Scheme PLT #2	10.16		29,992	503
14	F# Mono	11.59		22,656	587
14	Oberon-2 OO2C	11.97		17,244	755
18	Forth GNU GForth	15.66		4,888	564
25	Java 6 -Xint #2	21.46		24,364	603
27	Mozart/Oz	23.10		27,252	479
31	D Digital Mars	26.26		17,596	511
40	Prolog YAP	34.28		20,252	719
47	Python Psyco #3	39.45		16,704	431
47	Fortran G95	39.69		10,736	826
56	Groovy #2	47.36		51,108	506
59	Smalltalk Squeak	49.76		31,384	722
64	Ruby 1.9 #2	54.41		17,864	409
65	Groovy	55.32		53,632	519
78	Forth bigForth	65.98		8,916	433
84	Python IronPython #3	71.41		84,772	418
93	Pike	78.47		16,568	520
97	Icon	82.17		15,860	526
110	Python CPython #3	93.41		16,116	418
166	Ruby JRuby #2	140.87		136,048	409
177	JavaScript SpiderMonkey	150.07		136,832	467
203	Ruby MRI #2	172.31		48,756	409
204	PHP #2	173.25		81,648	493
208	Tcl	176.07		30,592	540
220	JavaScript Rhino	186.94		81,940	467
282	Perl #2	239.36		47,936	541
300	Perl	254.18		37,604	481
587	C CINT	8 min		10,132	698
2,079	Rebol	29 min		78,484	490
6,422	Io	1h 30 min		109,996	400
	Prolog SWI	Failed			790
	Smalltalk GNU	Failed			729
"interesting alternative" programs
	Python IronPython	Failed			377
	C++ Intel #3	Failed			814
0.3	C++ GNU g++ #3	0.26	0.00	2,192	814
0.7	C++ GNU g++	0.60	0.00	1,860	778
1.2	C++ Intel	0.98	0.00	2,072	778
1.2	C Intel #3	1.05	0.00	3,940	758
1.3	Clean	1.08	0.00	8,748	538
1.7	OCaml #3	1.44	0.00	8,844	502
1.9	C Intel #2	1.58	0.00	4,556	743
2.3	OCaml #4	1.98	0.00	11,772	504
2.6	Lisaac #2	2.23	0.00	345,352	691
3.6	Haskell GHC #4	3.07	0.00	6,192	510
3.8	Scheme Ikarus	3.24	0.00	29,612	592
11	Scheme PLT	9.60	0.00	18,268	482
18	Python Psyco	14.99	0.00	16,692	389
23	Scheme Chicken	19.30	0.00	32,768	493
44	Lua LuaJIT #3	37.45	0.00	40,852	470
66	Lua #3	55.69	0.00	34,464	470
79	Python CPython	66.96	0.00	16,172	377
missing benchmark programs
	Lisaac	No program
	Lua	No program
	Lua LuaJIT	No program
	Scheme Chicken	No program
	Scheme Ikarus	No program
	SML SML/NJ	No program
	Zonnon Mono	No program

 binary-trees benchmark : Allocate and deallocate many many binary trees

diff program output N = 10 with this 1KB output file to check your program is correct before contributing.

Each program should

• define a tree node class and methods, a tree node record and procedures, or an algebraic data type and functions, or…
• allocate a binary tree to 'stretch' memory, check it exists, and deallocate it
• allocate a long-lived binary tree which will live-on while other trees are allocated and deallocated
• allocate, walk, and deallocate many bottom-up binary trees
  • allocate a tree
  • walk the tree nodes, checksum node items (and maybe deallocate the node)
  • deallocate the tree
• check that the long-lived binary tree still exists

Note: this is an adaptation of a benchmark for testing GC so we are interested in the whole tree being allocated before any nodes are GC'd - which probably excludes lazy evaluation.

Note: the left subtrees are heads of the right subtrees, keeping a depth counter in the accessors to avoid duplication is cheating!

Note: the tree should have tree-nodes all the way down, replacing the bottom nodes by some other value is not acceptable; and the bottom nodes should be at depth 0.

Note: these programs are being measured with the default initial heap size - the measurements may be very different with a larger initial heap size or GC tuning.

Please don't implement your own custom memory pool or free list.

The binary-trees benchmark is a simplistic adaptation of Hans Boehm's GCBench, which in turn was adapted from a benchmark by John Ellis and Pete Kovac.

Thanks to Christophe Troestler and Einar Karttunen for help with this benchmark.

  Home   Flawed   Fastest   License   Help