OpenChess

TESTING STOCKFISH DD = 640 Games.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 20.0 - 20.0 +8/=24/-8 50.00%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 21.5 - 18.5 +5/=33/-2 53.75%

Time Control= 2+2

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 24.0 - 16.0 +11/=26/-3 60.00%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 23.0 - 17.0 +9/=28/-3 57.50%

320 Games = http://www.mediafire.com/download/b1mykws89dm8l61/
Score using 6 cores= 177.0 – 143.0 = 55.31%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 20.0 - 20.0 +11/=18/-11 50.00%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 22.5 - 17.5 +14/=17/-9 56.25%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.0 - 22.0 +3/=30/-7 45.00%

Time Control = 2+2

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 21.5 - 18.5 +13/=17/-10 53.75%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.5 - 21.5 +4/=29/-7 46.25%

320 games= http://www.mediafire.com/view/2c5vg7j4p ... 0games.pgn
Score using 4 cores = 163.0 – 157.0 = 50.94%

Segmenting results by Time Control:

Fixed = 163.0 – 157.0 = 50.94%
Incremental = 177.0 – 143.0 = 55.31%

SUMMARY:

Global Score: 340.0 – 300.0 = 53.12%

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 53.44%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 54.06%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 54.37%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 50.62%

Average Elo Opponents = 3190.
Estimated ELO Performance = 3212

-----------------------------------------------

After the latest tests, this is the Estimated Elo Performance I have found in my computers with 4 and 6 cores under my testing conditions:

1.- Stockfish 111113SL Large Pages Ipman = 3220
2.- Stockfish DD = 3212
3.- Houdini 4.0 Pro B = 3210
4.- Stockfish 191113 64 SSE4.2x = 3204
5.- Houdini 3.0 Pro = 3172
6.- Komodo 6.0 = 3166
7.- Amitis 16-10-2013 = 3.160
8.- Stockfish 4= 3119
9.- Critter 1.6a = 3093
10.- Deep Rybka 4.1 = 3015

Regards from Barcelona,

Tom.

I would make 2 comments. I am not sure what "Estimated ELO Performance" means, as Elo itself is already an estimate (based on a finite sample). And pursuant to this latter point, I think it would be good to indicate (say with the final list) the expected error margins on the ratings from 640 games.

BB+ wrote:I would make 2 comments. I am not sure what "Estimated ELO Performance" means, as Elo itself is already an estimate (based on a finite sample). And pursuant to this latter point, I think it would be good to indicate (say with the final list) the expected error margins on the ratings from 640 games.

Thanks for your comments BB.

1.- Estimated Elo Performance means that the ELO you can see in my list is not very refined. There are different ways to calculate ELO and I us the simplest and less accurate one. I introduce the word 'estimated' to make clear this fact to the readers of my posts.

2.- This year 2013 I have produced and shared with the computer chess community -in other fora- more than 30,000 games. Each game lasts 8 minutes and 40 seconds on average and generates around 3 Billion nodes each engine per game. I am sure you know that the bigger the sample, the lower the error bar. The list posted comes from 30,000 games, rather than from 640. This is the reason I have no interest at all in error bars and therefore I don't calculate them.

Regards,

Tom.

TESTING 4 DIFFERENT STOCKFISH WITH HOUDINI AND KOMODO: 1200 Games Tournament.


Players:

Houdini 4.0 Standard, Contempt 0..
Komodo TCEC (World champion. )
Stockfish Rockwood (or Stockwood).
Stockfish Development (Latest version 041213)
Stockfish 111113 Ipman Large Pages (The current leader in my ranking).
Stockfish DD


First Leg: Fixed Time Control 4min. + 0 sec. = 600 Games.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

1 Stockfish 041213 9.0 - 11.0 11.0 - 9.0 11.5 - 8.5 9.5 - 10.5 12.0 - 8.0 53.0/100
2 StockWood 031213 11.0 - 9.0 10.0 - 10.0 12.0 - 8.0 9.5 - 10.5 10.0 - 10.0 52.5/100
3 Stockfish DD 64 9.0 - 11.0 10.0 - 10.0 10.0 - 10.0 11.5 - 8.5 11.0 - 9.0 51.5/100
4 Houdini 4 x64stCT0 8.5 - 11.5 8.0 - 12.0 10.0 - 10.0 12.0 - 8.0 13.0 - 7.0 51.5/100
5 Stockfish 111113SL 10.5 - 9.5 10.5 - 9.5 8.5 - 11.5 8.0 - 12.0 13.5 - 6.5 51.0/100
6 Komodo TCEC 64-bit 8.0 - 12.0 10.0 - 10.0 9.0 - 11.0 7.0 - 13.0 6.5 - 13.5 40.5/100

300 Games= http://www.mediafire.com/view/16ambtv5a ... 0Games.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

1 Houdini 4 x64xCT0 8.5 - 11.5 10.5 - 9.5 12.5 - 7.5 12.0 - 8.0 10.5 - 9.5 54.0/100
2 Stockfish DD 64 11.5 - 8.5 8.5 - 11.5 8.5 - 11.5 11.5 - 8.5 11.0 - 9.0 51.0/100
3 StockWood 031213 9.5 - 10.5 11.5 - 8.5 11.0 - 9.0 7.5 - 12.5 10.5 - 9.5 50.0/100
4 Stockfish 041213 x 7.5 - 12.5 11.5 - 8.5 9.0 - 11.0 10.0 - 10.0 11.0 - 9.0 49.0/100
5 Komodo TCEC 64-bit 8.0 - 12.0 8.5 - 11.5 12.5 - 7.5 10.0 - 10.0 9.5 - 10.5 48.5/100
6 Stockfish 111113SL 9.5 - 10.5 9.0 - 11.0 9.5 - 10.5 9.0 - 11.0 10.5 - 9.5 47.5/100

300 Games= http://www.mediafire.com/view/zp8uhhp8n ... 0Games.pgn

STANDINGS AFTER 600 GAMES ( One half of this Tournament):

1.- Houdini 4x64xCT0 105.5/200
2.- Stockwood 031213 102.5/200
3.- Stockfish DD 102.5/200
4.- Stockfish 041213 102.0/200
5.- Stockfish 111113SL 98.5/200
6.- Komodo TCEC 64 89.0/200


Second leg: Incremental Time Control 2min. + 2 sec. = 600 Games.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 2+2.

201312Sixstarstourn_2+2 2013


1 Houdini 4 x64_st_X6_CT0 11.5 - 8.5 10.0 - 10.0 9.5 - 10.5 12.0 - 8.0 12.5 - 7.5 55.5/100
2 Stockfish 041213 8.5 - 11.5 11.0 - 9.0 9.5 - 10.5 10.5 - 9.5 12.5 - 7.5 52.0/100
3 StockWood 031213 10.0 - 10.0 9.0 - 11.0 9.5 - 10.5 10.5 - 9.5 12.5 - 7.5 51.5/100
4 Stockfish DD 64 10.5 - 9.5 10.5 - 9.5 10.5 - 9.5 9.0 - 11.0 10.5 - 9.5 51.0/100
5 Stockfish 111113SL 8.0 - 12.0 9.5 - 10.5 9.5 - 10.5 11.0 - 9.0 12.0 - 8.0 50.0/100
6 Komodo TCEC 64-bitx6NOB 7.5 - 12.5 7.5 - 12.5 7.5 - 12.5 9.5 - 10.5 8.0 - 12.0 40.0/100


300 Games=
http://www.mediafire.com/download/30x5n7aznfs9kdi/

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 2+2

1 Houdini 4 x64xCT0 12.5 - 7.5 11.0 - 9.0 11.5 - 8.5 12.0 - 8.0 12.5 - 7.5 59.5/100
2 StockWood 031213 7.5 - 12.5 10.5 - 9.5 10.0 - 10.0 10.5 - 9.5 11.0 - 9.0 49.5/100
3 Stockfish 0412134.2x 9.0 - 11.0 9.5 - 10.5 9.5 - 10.5 10.0 - 10.0 11.5 - 8.5 49.5/100 4 Stockfish 111113SL 8.5 - 11.5 10.0 - 10.0 10.5 - 9.5 10.0 - 10.0 9.0 - 11.0 48.0/100 5 Komodo TCEC 64-bitx4 8.0 - 12.0 9.5 - 10.5 10.0 - 10.0 10.0 - 10.0 10.5 - 9.5 48.0/100
6 Stockfish DD 64 SSE4.2x 7.5 - 12.5 9.0 - 11.0 8.5 - 11.5 11.0 - 9.0 9.5 - 10.5 45.5/100

300 Games= http://www.mediafire.com/view/7jlczukc2 ... 0Games.pgn

STANDINGS FOR INCREMENTAL TIME CONTROL GAMES ( Second half of this Tournament):

1.- Houdini 4x64xCT0 115.0/200
2.- Stockfish 041213 101.5/200
3.- Stockwood 031213 101.0/200
4.- Stockfish 111113SL 98.0/200
5.- Stockfish DD 96.5/200
6.- Komodo TCEC 64 88.0/200


FINAL STANDINGS AFTER 1.200 GAMES

1.- Houdini 4x64xCT0 220.5/400 55.125%
2.- Stockfish 041213 203.5/400 50.875%
3.- Stockwood 031213 203.5/400 50.875%
4.- Stockfish DD 199.0/400 49.750%
5.- Stockfish 111113SL 196.5/400 49.125%
6.- Komodo TCEC 64 177.0/400 44.250%

Although the number of games is not big enough, my preliminary conclussions are:

- What an impressive win for Houdini 4.0!. In my computers Standard version with Contempt ‘0’ is stronger than Pro version with Contempt 1.
- We are lucky to see the fast improvement pace in computer chess: The previous leader of my ranking -Stockfish 111113SL Ipman’s compile with Large Pages- has been surpassed by all other Stockfish in this test.
- Stockwood has proven to be very strong. I will follow his further development.
- The performance of Komodo TCEC –current TCEC world champion- has been disappointing in this tournament.
- The efficient and creative Stockfish team has still work ahead to reach Houdini 4.0 Standard Contempt ‘0’ level. Go, Stockfish!.

Regards from Barcelona.

Tom.

480 Games Tournament

Players:
Stockfish 071213 - Bench: 8322172 - Timestamp: 1386415938
Houdini 4.0 Standard Contempt 0
Stockwood 061213 (Stockfish Rockwood 061213)

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

1 Stockfish 071213 64 SSE4.2X6 19.5 - 20.5 22.5 - 17.5 42.0/80
2 Stockwood 061213 64 SSE4.2X6 20.5 - 19.5 19.5 - 20.5 40.0/80
3 Houdini 4 x64_st_X6_CT0 17.5 - 22.5 20.5 - 19.5 38.0/80

120 Games = http://www.mediafire.com/view/272gobx2b ... 6cores.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0


1 Stockfish 071213 64 SSE4.2x 21.5 - 18.5 21.5 - 18.5 43.0/80
2 Houdini 4 x64xCT0 18.5 - 21.5 23.0 - 17.0 41.5/80
3 Stockwood 061213 64 SSE4.2x 18.5 - 21.5 17.0 - 23.0 35.5/80

120 Games= http://www.mediafire.com/view/rajwe503b ... _4%2B0.pgn

Summary after 240 Games at Fixed Time Control:

1 Stockfish 071213 64 SSE4.2x 85.0/160
2 Houdini 4 x64xCT0 79.5/160
3 Stockwood 061213 64 SSE4.2x 75.5/160

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 2+2.

1 Stockwood 061213 64 SSE4.2X6 22.0 - 18.020.5 - 19.5** 42.5/80
2 Houdini 4 x64_st_X6_CT0 18.0 - 22.021.5 - 18.5 ** 39.5/80
3 Stockfish 071213 64 SSE4.2X6 19.5 - 20.518.5 - 21.5 ** 38.0/80

120 Games = http://www.mediafire.com/view/jug8411kf ... 6cores.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 2+2

1 Stockwood 061213 64 SSE4.2x 21.0 - 19.021.0 - 19.0** 42.0/80
2 Houdini 4 x64xCT0 19.0 - 21.021.5 - 18.5 ** 40.5/80
3 Stockfish 071213 64 SSE4.2x 19.0 - 21.018.5 - 21.5 ** 37.5/80

120 Games= http://www.mediafire.com/view/y6c8xmd2o ... _2%2B2.pgn

Summary after 240 Games at Incremental Time Control:

1 Stockwood 061213 64 SSE4.2x 84.5/160
2 Houdini 4 x64xCT0 80.0/160
3 Stockfish 071213 64 SSE4.2x 75.5/160

GLOBAL SCORE AFTER 480 GAMES:

1 Stockfish 071213 64 SSE4.2x 160.5/320
2 Stockwood 061213 64 SSE4.2x 160.0/320
3 Houdini 4 x64xCT0 159.5/320

Extreme equality between these three engines.

Regards,

Tom.