Designing an analysis friendly Stockfish?

[Uly]

Hannibal of Rybka forum has provided compiles that implement these changes!:

http://dl.dropbox.com/u/11904592/stockfish_201_PH_C.zip

And, wow, they do work, not only does Stockfish present the expected behavior, the analysis and score seem stable now, and the granularity is gone!

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

Analysis starts normally.

10/09 0:00 +0.09 1.e4 Nc6 2.Nc3 Nf6 3.Nf3 d5 4.exd5 Nxd5 5.Bc4 (20.425) 217
10/09 0:00 +0.09 1.d4 Nf6 2.Nc3 Nc6 3.d5 Nb4 4.e4 Ng4 5.Nf3 (24.212) 220
11/08 0:00 +0.19 1.d4 Nf6 2.Nc3 e6 3.Nf3 d5 4.Be3 Nc6 (50.212) 356
11/09 0:00 +0.19 1.e4 Nc6 2.Nc3 Nf6 3.Nf3 d5 4.exd5 Nxd5 5.Bc4 (61.446) 391
12/09 0:00 +0.21 1.e4 Nc6 2.Nf3 Nf6 3.Nc3 d5 4.exd5 Nxd5 5.Bc4 (95.063) 465
13/11 0:00 +0.27 1.e4 Nc6 2.d4 d5 3.exd5 Qxd5 4.Nf3 Qe4+ 5.Be3 Nf6 6.Nc3 (174.351) 557
14/10 0:00 +0.27 1.e4 e5 2.Nc3 Nf6 3.Nf3 Nc6 4.d4 exd4 5.Nxd4 Bb4 (344.756) 648
15/09 0:00 +0.27 1.e4 e5 2.Nf3 Nf6 3.Nxe5 Qe7 4.Nf3 Nxe4 5.Be2 (564.697) 681
16/09 0:01 +0.27 1.e4 e5 2.Nf3 Nf6 3.Nxe5 Qe7 4.Nf3 Nxe4 5.Be2 (830.597) 708
17/09 0:02 +0.35++ 1.e4 e5 2.Nf3 Nf6 3.Nxe5 Qe7 4.Nf3 Nxe4 5.Be2 (1.460.112) 718
17/09 0:02 +0.35 1.e4 e5 2.Nf3 Nf6 3.Nxe5 Qe7 4.Nf3 Nxe4 5.Be2 (1.460.122) 718
18/13 0:03 +0.38 1.e4 e5 2.Nc3 Nc6 3.Nf3 Nf6 4.Bc4 Bc5 5.O-O O-O 6.d3 Bd6 7.Bg5 (2.556.664) 733

Restarting analysis

10/15 0:00 +0.38 1.e4 Nf6 2.e5 Nd5 3.Nf3 d6 4.c4 Nb6 5.Nc3 Nc6 6.exd6 exd6 7.d4 Bf5 8.Be2 (10.308) 644
11/13 0:00 +0.38 1.e4 Nf6 2.Nc3 d5 3.exd5 Nxd5 4.Nf3 Nxc3 5.bxc3 Nc6 6.d4 Bf5 7.d5 (16.949) 1059
12/13 0:00 +0.38 1.e4 Nf6 2.Nc3 d5 3.exd5 Nxd5 4.Nf3 Nxc3 5.bxc3 Nc6 6.d4 Bf5 7.d5 (27.340) 854
13/13 0:00 +0.38 1.e4 Nf6 2.Nc3 d5 3.exd5 Nxd5 4.Nf3 Nxc3 5.bxc3 Nc6 6.d4 Bf5 7.d5 (46.823) 743
14/15 0:00 +0.38 1.e4 e5 2.Nc3 Nf6 3.Nf3 Nc6 4.Bc4 Bb4 5.O-O d6 6.Nd5 O-O 7.Nxb4 Nxb4 8.d3 (122.944) 783
15/15 0:00 +0.39 1.e4 e5 2.Nc3 Nf6 3.Nf3 Nc6 4.Bc4 Bb4 5.O-O d6 6.Nd5 O-O 7.Nxb4 Nxb4 8.d3 (148.750) 791
16/14 0:00 +0.41 1.e4 e5 2.Nc3 Nf6 3.Nf3 Bb4 4.Bc4 O-O 5.O-O Nc6 6.d3 a6 7.Bg5 b5 (414.408) 803
17/14 0:00 +0.38 1.e4 e5 2.Nf3 Nf6 3.Nc3 Bb4 4.Bc4 Bxc3 5.dxc3 d6 6.O-O O-O 7.Re1 Be6 (669.697) 793
18/18 0:01 +0.38 1.e4 e5 2.Nf3 Nf6 3.Nc3 Bb4 4.Bc4 d6 5.O-O O-O 6.d3 Nc6 7.Be3 Bd7 8.d4 Bxc3 9.bxc3 exd4 (995.824) 796

Forcing move.

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

10/14 0:00 +0.41 1...Nf6 2.Nc3 e5 3.Nf3 Nc6 4.Bc4 Bc5 5.O-O O-O 6.d3 d6 7.Na4 Bb4 8.c3 (7.903) 493
10/14 0:00 +0.40 1...Nc6 2.Nf3 e5 3.Nc3 Nf6 4.Bc4 Bc5 5.O-O O-O 6.d3 d6 7.Na4 Bb4 8.c3 (9.531) 595
10/14 0:00 +0.40 1...e5 2.Nc3 Nf6 3.Nf3 Nc6 4.Bc4 Bc5 5.O-O O-O 6.d3 d6 7.Na4 Bb4 8.c3 (12.167) 392
11/14 0:00 +0.37 1...e5 2.Nc3 Nf6 3.Nf3 Nc6 4.Bc4 Bc5 5.O-O O-O 6.d3 d6 7.Na4 Bb4 8.c3 (19.076) 615
12/09 0:00 +0.37 1...e5 2.Nc3 Nf6 3.Nf3 Bb4 4.Nxe5 O-O 5.Be2 Qe7 (26.792) 570
12/15 0:00 +0.36 1...Nf6 2.Nc3 Nc6 3.Nf3 d5 4.exd5 Nxd5 5.Bc4 Nxc3 6.bxc3 e5 7.O-O Bd6 8.d4 e4 (30.620) 651
13/14 0:00 +0.37 1...Nf6 2.e5 Nd5 3.Nf3 d6 4.c4 Nb6 5.Nc3 Nc6 6.exd6 exd6 7.d4 Bf5 8.Be2 (45.533) 722
13/09 0:00 +0.37 1...e5 2.Nc3 Nf6 3.Nf3 Bb4 4.Nxe5 O-O 5.Be2 Qe7 (48.463) 621
14/17 0:00 +0.33 1...e5 2.Bc4 Nf6 3.Nc3 Bb4 4.Nf3 d6 5.O-O O-O 6.d3 Nc6 7.Be3 Bd7 8.d4 Bxc3 9.bxc3 exd4 (117.786) 755
15/10 0:00 +0.25 1...e5 2.Nf3 Nf6 3.Nxe5 Nc6 4.Nxc6 dxc6 5.e5 Bg4 6.Be2 (239.211) 805
16/10 0:00 +0.37-- 1...e5 2.Nf3 Nf6 3.Nxe5 Nc6 4.Nxc6 dxc6 5.e5 Bg4 6.Be2 (295.344) 787
16/10 0:00 +0.37 1...e5 2.Nf3 Nf6 3.Nxe5 Nc6 4.Nxc6 dxc6 5.e5 Bg4 6.Be2 (365.389) 834
17/10 0:00 +0.37 1...e5 2.Nf3 Nf6 3.Nxe5 Qe7 4.Nf3 Nxe4 5.Be2 Nc6 6.d3 (554.778) 825
18/17 0:02 +0.44 1...e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bd6 5.d4 Nxd4 6.Nxd4 exd4 7.Qxd4 Qe7 8.Nc3 c6 9.Be2 Be5 (1.617.494) 778

(Here one wonders where that 0.25 came from, but the behavior at depth 17 is as expected)

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Back to the root.

10/16 0:00 +0.44 1.e4 Nf6 2.e5 Nd5 3.Nf3 d6 4.c4 Nb6 5.Nc3 c5 6.exd6 Nc6 7.dxe7 Bxe7 8.Be2 O-O (6.298) 203
11/16 0:00 +0.44 1.e4 Nf6 2.e5 Nd5 3.Nf3 d6 4.c4 Nb6 5.Nc3 c5 6.exd6 Nc6 7.dxe7 Bxe7 8.Be2 O-O (8.301) 267
12/12 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.d4 exd4 4.Nxd4 Nf6 5.Nc3 Bb4 6.Nxc6 dxc6 (17.617) 374
13/16 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Ng5 d5 5.exd5 Na5 6.Bb5+ c6 7.dxc6 bxc6 8.Be2 h6 (31.176) 494
14/16 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Ng5 d5 5.exd5 Na5 6.Bb5+ c6 7.dxc6 bxc6 8.Be2 h6 (57.540) 612
15/16 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Ng5 d5 5.exd5 Na5 6.Bb5+ c6 7.dxc6 bxc6 8.Be2 h6 (76.885) 698
16/09 0:00 +0.38 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Nxe4 5.Qe2 (216.248) 769
17/15 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bd6 5.Bxc6 bxc6 6.d4 Ba6 7.dxe5 Bxf1 8.exd6 (508.435) 756
18/17 0:00 +0.44 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Ng5 d5 5.exd5 Na5 6.Bb5+ c6 7.dxc6 bxc6 8.Qf3 h6 9.Ne4 (765.583) 777
19/14 0:02 +0.43 1.e4 e5 2.Nf3 Nc6 3.Bc4 Be7 4.O-O Nf6 5.Nc3 O-O 6.d4 Nxe4 7.Nxe4 d5 (1.518.660) 759

Excellent! One wonders how is it possible that the clear improvement in behavior (the new code is killing 4 birds with 2 stones!) is costing Stockfish 133 elo on games.

But indeed, here's a report by Moz that indeed something more has changed.

"There are some scary differences between this and the default SF. I'm not sure what to make of them but analyzing the following position in multipv mode will give you an idea of what I'm referring to (this is from my WBCCC game with ralunger so please refrain from making comments about the game itself)"

6k1/2r2pb1/p1NR2p1/8/Pp5r/8/1PP2R1P/1K6 w - - 0 33

For some reason, the PH version has a *lot* of trouble in this position - Na5 and Nb8 should be the top 2 moves in multipv mode but it's all over the place. The analysis seems to confirm Marco's test results -- performance is much worse.

Use with caution."

And indeed:

Default Stockfish at MultiPV=4:

15 0:10 -0.28 1.Na5 Be5 2.Rxa6 Bxh2 3.Nc6 Bd6 4.Rd2 Bf8 5.Ka2 g5 6.Ne5 Re4 7.Rd7 Rxd7 8.Nxd7 Be7 9.Nf6+ Bxf6 10.Rxf6 Kg7 11.Rf1 (10.067.993) 935
15 0:10 -0.32 1.Rd8+ Kh7 2.Rd6 f5 3.a5 Re4 4.Nd8 Ra7 5.Ne6 Bf6 6.Rg2 Re1+ 7.Ka2 Be7 8.Ng5+ Kg7 9.Rxg6+ Kxg6 10.Nf3+ Kf6 11.Nxe1 f4 12.Nf3 Rd7 (10.067.993) 935
15 0:10 -0.32 1.Nb8 a5 2.Ra6 Be5 3.Rxa5 Bxh2 4.Rb5 Bd6 5.Ka2 Kg7 6.Rd5 Bf4 7.a5 f5 8.Rg2 Be3 9.Kb3 f4 10.Na6 (10.067.993) 935
15 0:10 -0.76 1.Re2 Bf8 2.Rd4 Rxd4 3.Nxd4 f5 4.Re6 Rd7 5.Nf3 Rd6 6.Re8 Kf7 7.Rb8 Be7 8.Kc1 Re6 9.Kd2 Kf6 10.Nd4 Re4 (10.067.993) 935
-----
16 0:23 -0.32 1.Na5 Be5 2.Rxa6 Bxh2 3.Nc6 Bd6 4.Rd2 Bf8 5.Ka2 g5 6.Rd5 f6 7.Nd4 Kf7 8.Ne6 Rxc2 9.Nxf8 Kxf8 10.Rxf6+ Ke7 11.Rb6 g4 12.Rxb4 (22.274.972) 947
16 0:23 -0.40 1.Nb8 a5 2.Ra6 Rd4 3.Nc6 Rd1+ 4.Ka2 Rc1 5.Rg2 Kh7 6.h4 b3+ 7.Kxb3 Rb7+ 8.Kc4 Rxb2 9.Rxa5 Rcxc2+ 10.Rxc2 Rxc2+ 11.Kd5 (22.274.972) 947
16 0:23 -0.52 1.Rd8+ Kh7 2.Rd6 f5 3.a5 Re4 4.Nd8 Ra7 5.Ne6 Bh6 6.Rg2 f4 7.Ka2 Re5 8.Kb3 Rxa5 9.Kxb4 Rb5+ 10.Kc3 Rab7 11.Rxa6 Rxb2 12.h4 (22.274.972) 947
16 0:23 -0.60 1.Nd8 Rd4 2.Rxd4 Bxd4 3.Rf4 Bc5 4.Rc4 a5 5.Ka2 Kf8 6.Kb3 Ke7 7.Nb7 Bb6 8.Rxc7+ Bxc7 9.Kc4 f5 10.Kb5 Bxh2 11.Nxa5 (22.274.972) 947
-----
17 0:31 -0.32 1.Na5 Be5 2.Rxa6 Bxh2 3.Nc6 Bd6 4.Rd2 Bf8 5.Ka2 Re4 6.Nd4 Bh6 7.Nb5 Bxd2 8.Ra8+ Kg7 9.Nxc7 f5 10.Ne8+ Kh6 11.Nf6 Re6 12.Nd5 (29.528.988) 949
17 0:31 -0.40 1.Rd8+ Kh7 2.Rd6 f5 3.a5 Re4 4.Nd8 Ra7 5.Ne6 Bh6 6.Rg2 f4 7.Ka2 Re5 8.Kb3 Rxa5 9.Kxb4 Re5 10.Rf2 Re7 11.Nc5 a5+ 12.Kc4 Kg7 13.h4 (29.528.988) 949
17 0:31 -0.68 1.Nb8 a5 2.Ra6 Rd4 3.Nc6 Rd1+ 4.Ka2 Rc1 5.Rg2 Kh7 6.h4 b3+ 7.Kxb3 Rb7+ 8.Kc4 Rxb2 9.Nxa5 Rh1 10.Nc6 Rxh4+ 11.Kd5 Bc3 12.Ra7 Kg7 (29.528.988) 949
17 0:31 -0.76 1.Re2 Bf8 2.Rd4 Rxd4 3.Nxd4 f5 4.Re6 Rd7 5.Nf3 Rd6 6.Re1 Kg7 7.Ka2 Kf6 8.Kb3 Rd5 9.Kc4 Ra5 10.Nd4 Rxa4 11.Re6+ Kg5 (29.528.988) 948
-----
18 0:56 -0.12 1.Na5 Be5 2.Rxa6 Bxh2 3.Nc6 Bd6 4.Rd2 Bf8 5.Ne5 Re4 6.Nd3 g5 7.a5 g4 8.Rb6 Be7 9.a6 Ra7 10.Rg2 Kf8 11.Ka2 Bd8 12.Rd6 (54.076.908) 958
18 0:56 -0.20 1.Rd8+ Kh7 2.Rd6 f5 3.a5 Rg4 4.Nd8 Ra7 5.Rfd2 Bf8 6.Rd7+ Rxd7 7.Rxd7+ Kg8 8.Ra7 Rd4 9.Ne6 Rd6 10.Nxf8 Kxf8 11.Ka2 Rc6 12.Kb3 f4 (54.076.908) 958
18 0:56 -0.24 1.Nb8 a5 2.Ra6 Be5 3.Rxa5 Bxh2 4.Rb5 f5 5.Rg2 Kf7 6.a5 Bd6 7.Rb6 Rh1+ 8.Ka2 Rd1 9.Nc6 f4 10.Nxb4 Rc5 11.a6 f3 12.Rf2 Rf5 (54.076.908) 958
18 0:56 -0.76 1.Re2 Bf8 2.Rd4 Rxd4 3.Nxd4 f5 4.Re6 Rd7 5.Nf3 Rd6 6.Re1 Kg7 7.Ka2 Kf6 8.Kb3 Rd5 9.Kc4 Ra5 10.Nd4 Rxa4 11.Re8 b3+ 12.Kd5 Ra5+ 13.Kc4 (54.076.908) 958

Stockfish with improvements at MultiPV=4:

-----
15 0:05 -0.44 1.Nd8 Rd4 2.Rxd4 Bxd4 3.Rf4 Bg1 4.Rxb4 Rc8 5.Nb7 Bxh2 6.Rb6 Rb8 7.c4 Bc7 (4.445.820) 801
15 0:05 -0.48 1.a5 Re4 2.Nb8 Be5 3.Rb6 Bd4 4.Nxa6 Bxb6 5.axb6 Rb7 (4.445.820) 799
15 0:05 -2.00 1.Rf3 Bf8 2.Rd4 Rxh2 3.Nxb4 a5 4.Nd5 Rcxc2 5.Nf6+ Kh8 6.Rh4+ Rxh4 7.Kxc2 (4.445.820) 799
15 0:05 -4.94 1.Nxb4 Rxb4 2.Rd8+ Kh7 3.b3 f5 4.Rg2 (4.445.820) 799
-----
16 0:08 -0.44 1.Nd8 Rd4 2.Rxd4 Bxd4 3.Rf4 Bg1 4.Rxb4 Rc8 5.Nb7 Bxh2 6.Rb6 Rb8 7.c4 Bc7 (6.769.635) 834
16 0:08 -1.55 1.Rf3 Bf8 2.Rd4 Rxh2 3.Nxb4 Bxb4 4.Rxb4 Rhxc2 5.Ka2 f5 6.Rb8+ Kg7 7.Ra8 Rb7 (6.769.635) 833
16 0:08 -4.16 1.a5 Re4 2.Nxb4 Rxb4 3.Rxa6 Bxb2 4.Rb6 Ba3+ 5.Rxb4 Bxb4 6.a6 (6.769.635) 833
16 0:08 -4.89 1.c4 bxc3 2.b3 Bf8 3.Rd4 Rxd4 4.Nxd4 Bc5 5.Rf4 Rd7 6.Nf3 Bd6 7.Rc4 (6.769.635) 833
-----
17 0:14 -0.14 1.Nd8 a5 2.Ra6 Be5 3.Rxa5 Rxh2 4.Rxh2 Bxh2 5.Ra8 Kg7 6.a5 Bf4 7.Rb8 Bg5 8.a6 Be7 (12.725.035) 883
17 0:14 -1.18 1.c4 Bf8 2.Rd4 Rxd4 3.Nxd4 Rxc4 4.Nf3 Kg7 5.Ka2 Rc7 (12.725.035) 882
17 0:14 -1.87 1.Rf3 Bf8 2.Rd4 Rxh2 3.Nxb4 Bxb4 4.Rxb4 Rhxc2 5.Rf6 a5 6.Rb8+ Kg7 7.Ra6 R7c5 8.Rb5 (12.725.035) 882
17 0:14 -5.17 1.Nxb4 Rxb4 2.Rd8+ Kh7 3.b3 f5 4.Re8 (12.725.035) 882
-----
18 0:21 -0.25 1.Nd8 a5 2.Ra6 Be5 3.Rxa5 Rxh2 4.Rxh2 Bxh2 5.Rb5 Kg7 6.a5 Rc8 7.Nb7 (19.612.198) 912
18 0:21 -2.22 1.c4 bxc3 2.b3 Bf8 3.Rd4 Rh5 4.Nb8 Rb7 5.Nxa6 Rxb3+ (19.612.198) 911
18 0:21 -2.64 1.Rf3 Bf8 2.Rdf6 Rxh2 3.Rf1 Bg7 4.Rd6 Bf8 5.Rdf6 (19.612.198) 911
18 0:21 -5.03 1.c3 bxc3 2.b3 Bf8 3.Rd4 Rh5 4.Nb8 Rb7 5.Nxa6 Rxb3+ (19.612.198) 911

So, something broke, why is this?

[hyatt]

Here is what you are basically doing with the default stockfish. You eschew a table hit, and instead elect to do the search. Programs that probe and allow EXACT matches in the PV replace that search with a single hash probe. All this does is save effort. If it weakens the program, then either there is an undiscovered bug, or there is an undiscovered interaction that makes repeating the search somehow help. This could be from poor hash replacement policy, or a subtle change in what gets pruned/reduced. Who knows.

But the idea of the hash is to save effort. That is the _primary_ goal. And not allowing EXACT matches is contrary to any graph theory logic I can speculate on. If anything, I suspect my last suggestion above is the explanation, because it is possible to get EXACT matches without an EXACT depth / draft, which could replace a sub-tree with a significantly different sub-tree. This still suggests something intrinsically wrong somewhere, however...

Your last position is not worrisome to me. What would be more important would be how it does over tens of thousands of games. I suppose I could give this a run on my cluster, modifying stockfish...

[Uly]

Can you comment on the extreme change in behavior in MultiPV after applying the changes? Could it point more to where a bug lies?

[Ancalagon]

Can you comment on the extreme change in behavior in MultiPV after applying the changes? Could it point more to where a bug lies?

A small part of the problem is probably that Stockfish in PV nodes still does not test for EXACT hashmatches but also accepts value >= beta in case of a lower limit or hashvalue < beta in case of an upper limit entry (in ok_to_use_TT()). The PV search is supposed to return an exact value however, if it is within the αβ window, so these upper and lower limits should not be accepted I think, only the VALUE_EXACT ones. If the hashhit is outside the window it is probably better to search on, even if EXACT but that would be the next proposal I suppose. I don't know if code as below would solve the problem or could explain 133 elo difference, that seems a bit doubtful to me. But this is at least in theory more in line with what Bob means I think? This would be my proposal for qsearch()

Code: Select all

    // Transposition table lookup. At PV nodes, we don't use the TT for
    // pruning, but only for move ordering.
    tte = TT.retrieve(pos.get_key());
    ttMove = (tte ? tte->move() : MOVE_NONE);

    if (tte && (PvNode ? tte->type() == VALUE_TYPE_EXACT : ok_to_use_TT(tte, ttDepth, beta, ply)))
    {
        ss->bestMove = ttMove; // Can be MOVE_NONE
        return value_from_tt(tte->value(), ply);
    }

[gaard]

Can you comment on the extreme change in behavior in MultiPV after applying the changes? Could it point more to where a bug lies?

I missed the !PvNode in qsearch. See if this one still exhibits the crazy behavior in Multi-PV mode:
http://dl.dropbox.com/u/11904592/stockfish_201_PH_D.zip

[mcostalba]

Your last position is not worrisome to me. What would be more important would be how it does over tens of thousands of games. I suppose I could give this a run on my cluster, modifying stockfish...

Thanks for your help. I think I have found the problem. Removing !PvNode is not enough.

Problem is that in non-pv nodes we use TT when stored value is a lower bound higher then beta or an upper bound lower then alpha, IOW we use TT to detects fails high /low.

In case of PV-node we need to do the opposite and use TT value when is inside (alpha, beta) interval, not ouside !

So we need a specialized ok_to_use_tt() function tailored for PV nodes, something along these lines:

Code: Select all

  bool ok_to_use_TT_PV(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply) {

    Value v = value_from_tt(tte->value(), ply);

     return   tte->depth() >= depth
           && tte->type() == VALUE_TYPE_EXACT
           && tte->move() != MOVE_NONE
           && v < beta
           && v > alpha;
  }

[gaard]

Your last position is not worrisome to me. What would be more important would be how it does over tens of thousands of games. I suppose I could give this a run on my cluster, modifying stockfish...

Thanks for your help. I think I have found the problem. Removing !PvNode is not enough.

Problem is that in non-pv nodes we use TT when stored value is a lower bound higher then beta or an upper bound lower then alpha, IOW we use TT to detects fails high /low.

In case of PV-node we need to do the opposite and use TT value when is inside (alpha, beta) interval, not ouside !

So we need a specialized ok_to_use_tt() function tailored for PV nodes, something along these lines:

Code: Select all

  bool ok_to_use_TT_PV(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply) {

    Value v = value_from_tt(tte->value(), ply);

     return   tte->depth() >= depth
           && tte->type() == VALUE_TYPE_EXACT
           && tte->move() != MOVE_NONE
           && v < beta
           && v > alpha;
  }

Would it look like this:

Code: Select all

    if (   tte 
	     && ok_to_use_TT(tte, depth, beta, ply) 
		  && ok_to_use_TT_PV(tte, depth, alpha, beta, ply))
    {
        TT.refresh(tte);
        ss->bestMove = ttMove; // Can be MOVE_NONE
        return value_from_tt(tte->value(), ply);
    }

or

if ((   tte 
     && ok_to_use_TT(tte, depth, beta, ply)) 
     && (PvNode && (ok_to_use_TT_PV(tte, depth, alpha, beta, ply))
     || !PvNode)))
    {
        TT.refresh(tte);
        ss->bestMove = ttMove; // Can be MOVE_NONE
        return value_from_tt(tte->value(), ply);
    }

[mcostalba]

It would look like this:

Code: Select all

if (tte && (PvNode ? ok_to_use_TT_PV(tte, depth, alpha, beta, ply) : ok_to_use_TT(tte, depth, beta, ply))
 {
    TT.refresh(tte);
    ss->bestMove = ttMove; // Can be MOVE_NONE
    return value_from_tt(tte->value(), ply);
}

[gaard]

Can you comment on the extreme change in behavior in MultiPV after applying the changes? Could it point more to where a bug lies?

I missed the !PvNode in qsearch. See if this one still exhibits the crazy behavior in Multi-PV mode:
http://dl.dropbox.com/u/11904592/stockfish_201_PH_D.zip

http://dl.dropbox.com/u/11904592/stockfish_201_PH_E.zip

This one incorporates the changes suggested by M.Costalba. At first glance, it appears to be behaving correctly.

[gaard]

It would look like this:

Code: Select all

if (tte && (PvNode ? ok_to_use_TT_PV(tte, depth, alpha, beta, ply) : ok_to_use_TT(tte, depth, beta, ply))
 {
    TT.refresh(tte);
    ss->bestMove = ttMove; // Can be MOVE_NONE
    return value_from_tt(tte->value(), ply);
}

I made the changes both in the normal and qsearch parts of search.cpp. I am seeing a slow down of ~15%. Is this to be expected?