Still waiting on Ed

General discussion about computer chess...
hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Still waiting on Ed

Post by hyatt » Sun Jul 10, 2011 7:46 pm

Rebel wrote:
hyatt wrote:Have not seen a single piece of evidence from Ed. Wonder why?
Cause I have nothing to add nor to distract.

What's causing the huge branch factor difference between 23.2 and 23.3 ?

Whatever the answer, it's not your original idea. You heard it from someone. And that person heard it from another one.

And In the end the origin of the idea comes from the hacked Rybka.

You Robert Hyatt are using idea's in Crafty that smell Rybka.

Hacked Rybka.

Idea's never meant to be yours.

Yet you use them.

23.2 and 23.3? Here is what was changed, and confirmed by using "diff" (anyone can verify this and that your claim is completely false with respect to "using rybka ideas." First, the comments from main.c:

* 23.3 Null-move search restriction changed to allow null-move searches *
* at any node where the side on move has at least one piece of any *
* type. Minor bug in ValidMove() fixed to catch "backward" pawn *
* moves and flag them as illegal. This sometimes caused an error *
* when annotating games and annotating for both sides. The killer *
* move array could have "backward" moves due to flipping sides back *
* and forth, which would cause some odd PV displays. Small change *
* to "reduce-at-root". We no longer reduce moves that are flagged *
* as "do not search in parallel". Check extension modified so that *
* if "SEE" says the check is unsafe, the extension is not done. We *
* found this to be worth about +10 Elo. We also now reduce any *
* capture move that appears in the "REMAINING_MOVES" phase since *
* they can only appear there if SEE returns a score indicating loss *
* of material. We now reduce a bit more aggressively, reducing by` *
* 2 plies once we have searched at least 4 moves at any ply. I *
* tried going to 3 on very late moves, but could not find any case *
* where this was better, even limiting it to near the root or other *
* ideas. But reducing by 2 plies after the at least 4 moves are *
* searched was an improvement. *

Taking the ideas one at a time. Null-move was restricted previously to one side having more than a rook. Not a single piece allows it to do a null-move search. It was significant. And it was done in older versions of Crafty. I have changed this "threshold" more times than I can remember.

Check extension was modified to not extend checks if Swap() (SEE) says the check is unsafe. You think that is in Rybka? Better look.

I used to not reduce captures, ever. I changed that to agree with the do not extend safe checks (based on SEE) so that we now do reduce a capture if SEE says it loses material. Not from Rybka. In fact, when I did this, I found no program that reduced a capture, period, nor one that didn't extend checks, regardless.

Finally, we changed the reduction (LMR) code very slightly. In version 23.2, I reduced by 1 ply. In 23.3 I reduce by 2 plies, with only one exception, that being that the first move searched is never reduced, to handle the case where I have no hash move, no good captures, and no killers. I did only reduce in the "REMAINING_MOVES" phase but that reduces every move if there are none of the moves mentioned above present. The new version reduces the first move by 1 ply, if it seems to be ok to reduce (not a safe capture or check or a passed pawn push, or other similar ideas you can find if you edit 23.3's "search.c" and search for "reduce."

Here is 23.2's code:

Code: Select all

        if (tree->phase[ply] == REMAINING_MOVES && !tree->inchk[ply] &&
            !extensions && moves_searched) {
          if ((Piece(tree->curmv[ply]) != pawn ||
                  mask_passed[wtm][To(tree->
                          curmv[ply])] & Pawns(Flip(wtm)))) {
            extensions =
                Min(-Min(depth - 1 - LMR_remaining_depth,
                    (moves_searched >
                        2) ? LMR_max_reduction : LMR_min_reduction), 0);
            if (extensions)
              tree->reductions_done++;
          }
And here is 23.3's:

Code: Select all

        if (tree->phase[ply] == REMAINING_MOVES && !tree->inchk[ply] &&
            !tree->inchk[ply + 1] && !CaptureOrPromote(tree->curmv[ply])) {
          if (depth - 1 - LMR_depth >= LMR_min_depth &&
              (Piece(tree->curmv[ply]) != pawn ||
                  mask_passed[wtm][To(tree->curmv[ply])] & Pawns(Flip(wtm))))
          {
            extensions -= LMR_depth;
            tree->reductions_done++;
          }
You _really_ want to claim that my improvements came from Rybka, now? :) All you have to do is use "diff". There are some eval changes that are meaningless with respect to EBF. There are some variable name changes to eliminate some confusion in what the variables represented. There are a few similar changes in quiesce dealing with checking moves, which are now filtered using Swap() so that losing checks don't get included (certainly reduces the tree size significantly in a lot of positions).

So, sorry, but the above changes _are_ my _original ideas_. Hate to disappoint you. Again, not _everybody_ has to copy to learn. And this code will likely change again as new ideas are tried and tested.

To end in your style,

You, Ed Schroeder, are about as full of crap as a Christmas turkey, with respect to the claims you have made here. Feel free to try to refute anything I wrote. Note that I compared the source for versions 23.2 and 23.3, completely. There are a total of 1023 lines of output. Only the indicated code was in search.c, the rest was in variable names, or some substantial evaluation changes from Tracy and myself.

For reference, the "diff" output for search.c by itself:

Code: Select all

search.c
3c3
< /* last modified 07/16/10 */
---
> /* last modified 12/21/09 */
20c20
<   register int extensions;
---
>   register int extensions, pieces;
163c163,164
<       HashStore(tree, ply, MAX_DRAFT, wtm, EXACT, alpha, 0);
---
>       HashStore(tree, ply, MAX_DRAFT, wtm, EXACT, alpha,
>           tree->pv[ply].path[ply]);
201,203c202,206
<   if (do_null && alpha == beta - 1 && depth > 1 && !tree->inchk[ply] &&
<       TotalPieces(wtm, occupied)) {
<     register BITBOARD save_hash_key;
---
>   pieces =
>       (wtm) ? TotalPieces(white, occupied) : TotalPieces(black, occupied);
>   if (do_null && alpha == beta - 1 && depth > 1) {
>     if (!tree->inchk[ply] && pieces && (pieces > 9 || depth < 7)) {
>       register BITBOARD save_hash_key;
205,206c208,209
<     tree->curmv[ply] = 0;
<     tree->phase[ply] = NULL_MOVE;
---
>       tree->curmv[ply] = 0;
>       tree->phase[ply] = NULL_MOVE;
208,209c211,212
<     if (ply <= trace_level)
<       Trace(tree, ply, depth, wtm, beta - 1, beta, "Search1", 0);
---
>       if (ply <= trace_level)
>         Trace(tree, ply, depth, wtm, beta - 1, beta, "Search1", 0);
211,229c214,233
<     tree->position[ply + 1] = tree->position[ply];
<     Rule50Moves(ply + 1) = 0;
<     save_hash_key = HashKey;
<     if (EnPassant(ply)) {
<       HashEP(EnPassant(ply + 1));
<       EnPassant(ply + 1) = 0;
<     }
<     if (depth - null_depth - 1 > 0)
<       value =
<           -Search(tree, -beta, -beta + 1, Flip(wtm), depth - null_depth - 1,
<           ply + 1, NO_NULL);
<     else
<       value = -QuiesceChecks(tree, -beta, -beta + 1, Flip(wtm), ply + 1);
<     HashKey = save_hash_key;
<     if (abort_search || tree->stop)
<       return (0);
<     if (value >= beta) {
<       HashStore(tree, ply, depth, wtm, LOWER, value, tree->curmv[ply]);
<       return (value);
---
>       tree->position[ply + 1] = tree->position[ply];
>       Rule50Moves(ply + 1) = 0;
>       save_hash_key = HashKey;
>       if (EnPassant(ply)) {
>         HashEP(EnPassant(ply + 1));
>         EnPassant(ply + 1) = 0;
>       }
>       if (depth - null_depth - 1 > 0)
>         value =
>             -Search(tree, -beta, -beta + 1, Flip(wtm), depth - null_depth - 1,
>             ply + 1, NO_NULL);
>       else
>         value = -QuiesceChecks(tree, -beta, -beta + 1, Flip(wtm), ply + 1);
>       HashKey = save_hash_key;
>       if (abort_search || tree->stop)
>         return (0);
>       if (value >= beta) {
>         HashStore(tree, ply, depth, wtm, LOWER, value, tree->curmv[ply]);
>         return (value);
>       }
271,274c275,276
<           if (SwapO(tree, tree->curmv[ply], wtm) <= 0) {
<             tree->extensions_done++;
<             extensions = check_depth;
<           }
---
>           tree->extensions_done++;
>           extensions = check_depth;
288c290
<  *       the LMR_remaining_depth value).                    *
---
>  *       the LMR_min_depth value).                          *
299,308c301,307
<             !extensions && moves_searched) {
<           if ((Piece(tree->curmv[ply]) != pawn ||
<                   mask_passed[wtm][To(tree->
<                           curmv[ply])] & Pawns(Flip(wtm)))) {
<             extensions =
<                 Min(-Min(depth - 1 - LMR_remaining_depth,
<                     (moves_searched >
<                         2) ? LMR_max_reduction : LMR_min_reduction), 0);
<             if (extensions)
<               tree->reductions_done++;
---
>             !tree->inchk[ply + 1] && !CaptureOrPromote(tree->curmv[ply])) {
>           if (depth - 1 - LMR_depth >= LMR_min_depth &&
>               (Piece(tree->curmv[ply]) != pawn ||
>                   mask_passed[wtm][To(tree->curmv[ply])] & Pawns(Flip(wtm))))
>           {
>             extensions -= LMR_depth;
>             tree->reductions_done++;
436d434
<       tree->moves_searched = moves_searched;
501c499
< /* last modified 07/16/10 */
---
> /* last modified 07/04/09 */
571,574c569,570
<           if (SwapO(tree, tree->curmv[ply], wtm) <= 0) {
<             tree->extensions_done++;
<             extensions = check_depth;
<           }
---
>           tree->extensions_done++;
>           extensions = check_depth;
588c584
<  *       the LMR_remaining_depth value).                    *
---
>  *       the LMR_min_depth value).                          *
599,608c595,601
<             !extensions) {
<           if ((Piece(tree->curmv[ply]) != pawn ||
<                   mask_passed[wtm][To(tree->
<                           curmv[ply])] & Pawns(Flip(wtm)))) {
<             extensions =
<                 Min(-Min(depth - 1 - LMR_remaining_depth,
<                     (tree->parent->moves_searched >
<                         2) ? LMR_max_reduction : LMR_min_reduction), 0);
<             if (extensions)
<               tree->reductions_done++;
---
>             !tree->inchk[ply + 1] && !CaptureOrPromote(tree->curmv[ply])) {
>           if (depth - 1 - LMR_depth >= LMR_min_depth &&
>               (Piece(tree->curmv[ply]) != pawn ||
>                   mask_passed[wtm][To(tree->curmv[ply])] & Pawns(Flip(wtm))))
>           {
>             extensions -= LMR_depth;
>             tree->reductions_done++;
722d714
<         tree->parent->moves_searched++;
750c742
< /* last modified 07/16/10 */
---
> /* last modified 07/04/09 */
764c756
<   register int moves_searched = 0;
---
>   register int first_move = 1;
792,795c784,785
<         if (SwapO(tree, tree->curmv[1], wtm) <= 0) {
<           tree->extensions_done++;
<           extensions = check_depth;
<         }
---
>         tree->extensions_done++;
>         extensions = check_depth;
809c799
<  *       the LMR_remaining_depth value).                    *
---
>  *       the LMR_min_depth value).                          *
819,821c809,812
<       if (tree->phase[1] == REMAINING_MOVES && !tree->inchk[1] && !extensions
<           && moves_searched) {
<         if ((Piece(tree->curmv[1]) != pawn ||
---
>       if (tree->phase[1] == REMAINING_MOVES && !tree->inchk[1] &&
>           !tree->inchk[2] && !CaptureOrPromote(tree->curmv[1])) {
>         if (depth - 1 - LMR_depth >= LMR_min_depth &&
>             (Piece(tree->curmv[1]) != pawn ||
823,828c814,815
<           extensions =
<               Min(-Min(depth - 1 - LMR_remaining_depth,
<                   (moves_searched >
<                       2) ? LMR_max_reduction : LMR_min_reduction), 0);
<           if (extensions)
<             tree->reductions_done++;
---
>           extensions -= LMR_depth;
>           tree->reductions_done++;
851c838
<       if (moves_searched == 0) {
---
>       if (first_move) {
857a845
>         first_move = 0;
918d905
<       moves_searched++;
944d930
<       tree->moves_searched = moves_searched;
973c959
<   if (moves_searched == 0) {
---
>   if (first_move == 1) {
Note that the diffs are almost duplicated because I have a Search() and a SearchParallel() that are similar but not close enough to easily make into one fuction...

Again, ball is in your court. Feel free to tell me what I copied from Rybka? IP and Friends certainly reduce in a wildly different way than _I_ do, and that and the check extension idea are the only two changes that affect tree size with respect to effective branching factor...

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Still waiting on Ed

Post by hyatt » Sun Jul 10, 2011 9:02 pm

BTW, to further put a fly in this ointment, here are my testing results for 23.0 -> current 23.5. There are a few gaps where I deleted experimental 23.5 versions that we have tested...

Code: Select all

   3 Crafty-23.5-1        2656    4    4 30000   62%  2561   21%
   5 Crafty-23.4-2        2654    4    4 30000   62%  2561   21%
   6 Crafty-23.5-2        2653    4    4 30000   62%  2561   21%
   7 Crafty-23.4-1        2652    4    4 30000   62%  2561   21%
  10 Crafty-23.3-1        2642    4    4 30000   60%  2561   21%
  11 Crafty-23.3-2        2641    4    4 30000   60%  2561   21%
  12 Crafty-23.2-1        2592    4    4 30000   54%  2561   21%
  13 Crafty-23.2-2        2588    4    4 30000   54%  2561   21%
  14 Crafty-23.1-2        2573    4    4 30000   52%  2561   21%
  15 Crafty-23.1-1        2571    4    4 30000   52%  2561   21%
  18 Crafty-23.0-2        2482    4    4 30000   41%  2561   19%
  19 Crafty-23.0-1        2482    4    4 30000   41%  2561   19%
 
23.0->23.1 +90.
23.1->23.2 +15
23.2->23.3 +50
23.3->23.4 +10
23.4-23.5 maybe +1 or +2.

TImeline (from my ftp box dates as the .zip files were copied over for release:

-rw-r--r-- 1 500 30 419362 Mar 24 2009 crafty-23.0.zip
-rw-r--r-- 1 500 30 423190 Nov 20 2009 crafty-23.1.zip
-rw-r--r-- 1 500 30 424434 Mar 03 2010 crafty-23.2.zip
-rw-r--r-- 1 500 30 423008 Jul 02 2010 crafty-23.2a.zip
-rw-r--r-- 1 500 30 425519 Jul 22 2010 crafty-23.3.zip
-rw-r--r-- 1 500 30 426295 Nov 08 2010 crafty-23.4.zip

so about 1.75 years from 23.0 to 23.4, +170 Elo, two people keeping a 128-node cluster running almost 24/7 testing changes. Lot of "no changes". A few wins here and there.

It is now 2.75 years since 23.0 and we are not much further, although for the first 6 months of this year, I didn't spend much time on making changes due to the ICGA investigation. For those so interested, you can download each of those versions (ftp.cis.uab.edu/pub/hyatt/source) and look at main.c to see exactly what was changed. If you then believe I intentionally (or accidentally) failed to mention something new, you can "diff" the sources to see exactly which lines were changed, and then look at the code to see what they did.

In any case, none of this has a thing to do with rybka/ip*/robo*/etc...

As I said...

BTW just noticed 23.2a. Do not remember what that was, other than I believe it was some sort of a fix for an engine/GUI issue that someone reported dealing maybe with annotated games or analyzing a game in real-time... It was never tested on the cluster and doesn't show up in my ratings above.

I should add my cluster testing ratings are always somewhat inflated over rating lists, because I don't allow any opening books, nor endgame tables, the games start from common random opening positions and the programs have to do everything for themselves, no books, no book learning, no position learning, etc...

At best, these are upper bounds on the improvements, which are likely somewhat lower when you use real books and such.

Terry McCracken
Posts: 27
Joined: Thu Jun 10, 2010 12:44 am

Re: Still waiting on Ed

Post by Terry McCracken » Mon Jul 11, 2011 2:35 pm

Rebel wrote:
hyatt wrote:Have not seen a single piece of evidence from Ed. Wonder why?
Cause I have nothing to add nor to distract.

What's causing the huge branch factor difference between 23.2 and 23.3 ?

Whatever the answer, it's not your original idea. You heard it from someone. And that person heard it from another one.

And In the end the origin of the idea comes from the hacked Rybka.

You Robert Hyatt are using idea's in Crafty that smell Rybka.

Hacked Rybka.

Idea's never meant to be yours.

Yet you use them.

These are serious allegations you've been putting foward and Bob has refuted all them. Why you would do this Ed?- only you can know but it reflects badly on your character.

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Still waiting on Ed

Post by hyatt » Mon Jul 11, 2011 3:56 pm

I consider it to be just as funny that he is against the ICGA sanctions against rybka, while he did not mind signing the original letter of protest sent to the ICGA to jump-start the ICGA investigation.

Some sort of sour grapes, I suppose. But completely without merit, which is what happens when one makes claims without having taken the time to verify them. I assume, at the very least, that his signature on the ICGA complaint is meaningless, if he spent the same amount of time analyzing existing data related to that issue as he spent looking at Crafty's source.

User avatar
kingliveson
Posts: 1388
Joined: Thu Jun 10, 2010 1:22 am
Real Name: Franklin Titus
Location: 28°32'1"N 81°22'33"W

Re: Still waiting on Ed

Post by kingliveson » Mon Jul 11, 2011 5:09 pm

It is obvious now that Ed is just trolling Bob solely based on emotions regarding Rybka -- nothing more, nothing less. You can't put a value on a good name...please don't go that route Ed.
PAWN : Knight >> Bishop >> Rook >>Queen

Prima
Posts: 328
Joined: Tue Dec 14, 2010 6:12 am

Re: Still waiting on Ed

Post by Prima » Tue Jul 12, 2011 3:06 am

kingliveson wrote:It is obvious now that Ed is just trolling Bob solely based on emotions regarding Rybka -- nothing more, nothing less. You can't put a value on a good name...please don't go that route Ed.
Very good advise but I think it's too late. He opened his mouth bearing false accusations.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Still waiting on Ed

Post by Rebel » Tue Jul 12, 2011 8:43 am

hyatt wrote: Finally, we changed the reduction (LMR) code very slightly. In version 23.2, I reduced by 1 ply. In 23.3 I reduce by 2 plies, with only one exception, that being that the first move searched is never reduced, to handle the case where I have no hash move, no good captures, and no killers. I did only reduce in the "REMAINING_MOVES" phase but that reduces every move if there are none of the moves mentioned above present. The new version reduces the first move by 1 ply, if it seems to be ok to reduce (not a safe capture or check or a passed pawn push, or other similar ideas you can find if you edit 23.3's "search.c" and search for "reduce."
The red (in 23.3 I reduce by 2 plies) typical Rybka. And a more sophisticated implementation in Rybka BTW.

It's Vas who perfected LMR as first one. That's why Rybka could dominate for so long.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Still waiting on Ed

Post by Rebel » Tue Jul 12, 2011 8:51 am

hyatt wrote:I consider it to be just as funny that he is against the ICGA sanctions against rybka, while he did not mind signing the original letter of protest sent to the ICGA to jump-start the ICGA investigation.

Some sort of sour grapes, I suppose. But completely without merit, which is what happens when one makes claims without having taken the time to verify them. I assume, at the very least, that his signature on the ICGA complaint is meaningless, if he spent the same amount of time analyzing existing data related to that issue as he spent looking at Crafty's source.
You should not invent motives while I already clearly stated my motive: it's about you and your hatred against Vas while in the meantime profit from his legacy. If you hate, hate honestly.

veritas
Posts: 111
Joined: Thu Jun 16, 2011 2:35 pm

Re: Still waiting on Ed

Post by veritas » Tue Jul 12, 2011 8:57 am

Rebel wrote:
hyatt wrote:I consider it to be just as funny that he is against the ICGA sanctions against rybka, while he did not mind signing the original letter of protest sent to the ICGA to jump-start the ICGA investigation.

Some sort of sour grapes, I suppose. But completely without merit, which is what happens when one makes claims without having taken the time to verify them. I assume, at the very least, that his signature on the ICGA complaint is meaningless, if he spent the same amount of time analyzing existing data related to that issue as he spent looking at Crafty's source.
You should not invent motives while I already clearly stated my motive: it's about you and your hatred against Vas while in the meantime profit from his legacy. If you hate, hate honestly.
Don't you mean FABIENS legacy

less you forget Vas plagiarized Fruit , ergo . No Fruit No legacy


its one thing for thieves to fall out over £s n $s
but for friends to fall out over thieves beggars belief

Jeremy Bernstein
Site Admin
Posts: 1226
Joined: Wed Jun 09, 2010 7:49 am
Real Name: Jeremy Bernstein
Location: Berlin, Germany
Contact:

Re: Still waiting on Ed

Post by Jeremy Bernstein » Tue Jul 12, 2011 9:42 am

Rebel wrote:
hyatt wrote: Finally, we changed the reduction (LMR) code very slightly. In version 23.2, I reduced by 1 ply. In 23.3 I reduce by 2 plies, with only one exception, that being that the first move searched is never reduced, to handle the case where I have no hash move, no good captures, and no killers. I did only reduce in the "REMAINING_MOVES" phase but that reduces every move if there are none of the moves mentioned above present. The new version reduces the first move by 1 ply, if it seems to be ok to reduce (not a safe capture or check or a passed pawn push, or other similar ideas you can find if you edit 23.3's "search.c" and search for "reduce."
The red (in 23.3 I reduce by 2 plies) typical Rybka. And a more sophisticated implementation in Rybka BTW.

It's Vas who perfected LMR as first one. That's why Rybka could dominate for so long.
So any improvement to late move reduction in any chess software rightfully belongs to Vas' legacy? LMR is off-limits unless you give Vas credit for the idea of making it better? Are you seriously going to contend that?

I can understand why one might conclude that Bob had a chip on his shoulder about Rybka. But I think that 'hate' is a vast overstatement, and this entire line of "argument" is really below your reputation.

Jeremy

Post Reply