Thoughts on Fruit=Rybka EVAL

[Rebel]

Given that every reasonable chess program has all (or almost all) the ingredients as mentioned in the Zach & BB documents (mine certainly has all) makes a comparison hardly possible. Much of what's described in the document I either have done similar (and a lot earlier than good old Fabien ) or is done in a general known but just in a somewhat different way of doing the same thing, namely, to evaluate a positional aspect of a position following his understanding of the aspect.

Problem with EVAL is (and with PST it's similar) there are not so much choices. There are not 10 or 20 ways to calculate mobility, only few. Same with other aspects, king safety, all programs evaluate the squares close to the enemy king. Furthermore a chess programmer is forced to do it in quick and simple solutions for speed reasons. So possible alternatives quickly shrink and programs tend to become similar in the techniques they use.

However what I would accept as damaging is an exact or a great similarity in the order of the evaluation ingredients. I had a quick look at the Fruit and Strelka EVAL and that doesn't appear the case, I leave this to the Rybka/Fruit/Strelka experts.

The point is, I have a hard time to believe that Vas (in 2005) took the Fruit sources as a base and changed the order of Fruit's EVAL because he foresaw a scenario he would become so famous that somebody would re-engineer his program an he possibly could be caught on exact order of the ingredients in EVAL.

I would say, no similarity, not guilty.

[BB+]

See my post in the other thread (in answer to Miguel). The fact that there is a morass of "general chess knowledge" has never been much of an issue. To excerpt:

So I'll say it again (as it's a general principle) until I'm blue in the face: the fact that there is a "general source of knowledge" out there from which almost all engines borrow has little (if any) relevance to the usage of specific examples or encodings therein. This is the accusation against Rybka -- not that Fruit had a bunch of general chess knowledge and so does Rybka, but that they share way too much of the specific aspects of rendering this. The principal "creativity" (and/or originality) is thus in making choices concerning what aspects of the general knowledge to implement, how to do so, whether or not these choices jive well with each other, etc.

Problem with EVAL is (and with PST it's similar) there are not so much choices. There are not 10 or 20 ways to calculate mobility, only few. Same with other aspects, king safety, all programs evaluate the squares close to the enemy king. Furthermore a chess programmer is forced to do it in quick and simple solutions for speed reasons. So possible alternatives quickly shrink and programs tend to become similar in the techniques they use.

EVAL_COMP found somewhat the opposite to be true. For almost any "feature", there were multiple ways doing it. For instance, mobility could count safe squares, forward squares, squares toward the enemy king (and more -- R3 counts a 2-1 forward safe knight move different than a 1-2 forward safe knight move); it could scale it linearly, sigmoidally, or mix the mobility with other facets like king safety or weak pawns; different engines might ignore the mobility of various specific pieces (perhaps considering that "outposts" for knights are more relevant, or "unblocked"/good bishops, or that queen mobility is bad in the opening).

On the overall possibilities for what seems to a "simple" feature, I refer you again to the isolated pawns example.

Fruit 2.1, Rybka 1.0 Beta, and Rybka 2.3.2a all give a penalty for an isolated pawn that depends on whether the file is half-open or closed. Crafty 19.0 counts the number of isolated pawns, and the subcount of those on open files, and then applies array-based scores to these. Phalanx XXII gives a file-based penalty, and then adjusts the score based upon the number of knights the opponent has, the number of bishops we have, and whether an opposing rook attacks it. There is then a correction if an isolated pawn is a ``ram'', that is, blocked by an enemy pawn face-to-face, and also a doubled-and-isolated penalty. Pepito 1.59 has a file-based array for penalties, though the contents are constant except for the rook files. There is also a further penalty for multiple isolani. Faile 1.4 penalises isolated pawns by a constant amount, with half-open files penalised more (same as Fruit/Rybka). RESP 0.19 also penalises isolated pawns by a constant amount, and gives an additional penalty to isolated pawns that are doubled. EXchess also gives a constant penalty for isolated pawns, and further stores it in a king/queenside defect count.

Is your argument that one should just consider this to be "general chess knowledge"? And it is irrelevant whether one engine repeatedly makes the same specific choices as another in how to render this knowledge?

I would rather argue that the slight baubles and nuances here are essentially what make up the content of "originality" for any evaluation feature [with the choice/mix of evaluation features being itself another component to "originality"] -- if not, I don't see why (taking Richard Vida's suggestion to the extreme) the ICGA might not just let there be a "standard" evaluation function that everyone can use (and at that point, one might as well have a "standard" search function too).

[hyatt]

Mark, if someone has actually written a chess program, what you said is already obvious to them without it being stated. I can't count the number of ways I have done things differently. One of the unique ideas in Crafty's pawn scoring is that before I do anything, I compute a bitmap of all the squares that a pawn can safely reach without being subject to a losing capture by opponent pawns. This is certainly not perfect, because there are tempo issues involved. But the intent is to catch what is commonly called "an artificially isolated pawn." Which is a pawn that has advanced so far that it can't be safely defended by own-pawns because the adjacent pawns can not safely reach the necessary supporting square.

I use that "bitmap" in several places because it is a sort of either "pawns can possibly reach these squares given the right circumstances, or pawns can not reach these squares unless we rip off some enemy pawns that are impediments... So even simple isolated pawns can be evaluated in many different ways. And more importantly, each different way as many different implementations, different orders of testing things, and even different things to look for.

I don't believe for a minute that Ed believes his own statements, he is simply making them to oppose the ICGA report. There's plenty of non-authors that can't possibly understand that kind of issue. But no real author would question it at all. The idea of "there's just one way to do a few well-known things" is ridiculous.

And that ignores underlying implementation differences even when two programs do the same thing. Compare Crafty to Cray Blitz. We have some similar evaluation terms. But Cray Blitz incrementally updated a lot of stuff, such as open file info, files with a pawn, passed pawn locations and such. I chose to keep Crafty's MakeMove()/UnmakeMove() code independent of the evaluation, other than where I count individual piece types for material imbalance calculations... In a couple of cases, IDENTICAL idea, enormous differences in how the idea is expressed...

As far as the "not so much choices", I can remember when I did the outside passed pawn code and nobody else was doing it, and on ICC (ICS at the time) commercial programs were getting rolled up in endgames because they didn't understand how powerful a distant passed pawn is when the final piece is traded. Ditto for the trapped bishop (white bishop at a7/h7 with black pawns at c7-b6 or f7-g6. Most do both of those today. I added code for candidate passed pawns at the "demand" of Dzhindi as he was hyper-critical of all chess engines back in 1995-1996-1997 where some appeared to understand distant passedpawns, but not when it involved a candidate passed pawn that was distant.

There are plenty of ideas for evaluation left to try. The trade-off is always gain in knowledge vs loss in speed. To suggest that "chess evaluation" is both done, and there are not many things to evaluate is pretty ridiculous to someone that has actually played chess for 50+ years now and _still_ learn something new here and there regarding my own human "evaluation."

[Rebel]

On the overall possibilities for what seems to a "simple" feature, I refer you again to the isolated pawns example.

Fruit 2.1, Rybka 1.0 Beta, and Rybka 2.3.2a all give a penalty for an isolated pawn that depends on whether the file is half-open or closed.

I do that since 1982/83.

Because,

It's the right way of doing chess knowledge.

Vas is an IM, he knows.

I am so sorry for all your work.

------------

For me (today) the whole issue stands or falls with the part you did not comment, the synchronicity of the order of evaluations between Fruit and Rybka eval.

If there none, innocent.

If there is a clear synchronicity then guilty because that would be a second "0.0" bomb.

[Rebel]

I don't believe for a minute that Ed believes his own statements, he is simply making them to oppose the ICGA report.

Thank you.

I still believe in your innocence regarding the Fruit table in Crafty.

[BB+]

However what I would accept as damaging is an exact or a great similarity in the order of the evaluation ingredients. I had a quick look at the Fruit and Strelka EVAL and that doesn't appear the case, I leave this to the Rybka/Fruit/Strelka experts.

I don't think the evidence says much one way or the other. At the top level, the Rybka/Fruit orderings match a lot, but I can't say this is unexpected (one difference is that Rybka puts pawneval later, after prefetching the hash near the start.

The eval_piece in Fruit runs over a piece list (with a case switch), while Rybka loops over the bitboards, so there's not much of a comparison there. Internal to each piece, some of these are so short (knights and bishops) that any ordering is nugatory, while for queens there is again not much to say (mobility, king attack, 7th rank). Rooks are slightly more voluminous, and the order matches [though again there are not many elements]. Pawns also match (I think).

Code: Select all

   material_get_info(mat_info,board);
   opening += mat_info->opening; endgame += mat_info->endgame;
   mul[White] = mat_info->mul[White]; mul[Black] = mat_info->mul[Black];
   opening += board->opening; endgame += board->endgame; // PST
   pawn_get_info(pawn_info,board); // pawns
   opening += pawn_info->opening; endgame += pawn_info->endgame;
[...]
   eval_piece(board,mat_info,pawn_info,&opening,&endgame);
   eval_king(board,mat_info,&opening,&endgame);
   eval_passer(board,pawn_info,&opening,&endgame);
   eval_pattern(board,&opening,&endgame);
   phase = mat_info->phase; // phase mix
   eval = ((opening * (256 - phase)) + (endgame * phase)) / 256;
   if ((mat_info->flags & DrawBishopFlag) != 0) {    // drawish bishop endgames
      wb = board->piece[White][1]; ASSERT(PIECE_IS_BISHOP(board->square[wb]));
      bb = board->piece[Black][1]; ASSERT(PIECE_IS_BISHOP(board->square[bb]));
      if (SQUARE_COLOUR(wb) != SQUARE_COLOUR(bb)) {
         if (mul[White] == 16) mul[White] = 8; 
         if (mul[Black] == 16) mul[Black] = 8;       }   }

Rybka also has lazy eval at the start.

For me (today) the whole issue stands or falls with the part you did not comment, the synchronicity of the order of evaluations between Fruit and Rybka eval.

I am sorry I was so slow in responding, but I actually had to track down this information (I was missing an annotated Rybka pawneval, collate it, etc.[/sarcasm]

Can I conclude that one can take R4, reorder the evaluation function, and presto!, this is an original engine? [Since R4 has, of course, the "right way" of doing knowledge, for evaluation feature, and indeed choosing such in the first place?]

[BB+]

Fruit 2.1, Rybka 1.0 Beta, and Rybka 2.3.2a all give a penalty for an isolated pawn that depends on whether the file is half-open or closed.

I do that since 1982/83.
Because,
It's the right way of doing chess knowledge.
Vas is an IM, he knows.
I am so sorry for all your work.

Your write-up in How REBEL Plays Chess suggests that, at least at some point, you did otherwise than Fruit/Rybka (which do not depend on the square).

Indexing: REBEL in EVAL wherever it makes sense will use square tables above fixed
values. For instance, when evaluating an isolated pawn it can be given a fixed value as
penalty, such as 0.10, however a square table is more precise, more flexible to tune too.

Penalties for white isolated pawns based on their position on the board.
Background: usually in the middle game an isolated pawn on A2 is not so bad as an isolated pawn in the center.
Advice: use such square tables wherever you can in EVAL.
Remark: this example is only valid for REBEL's middle game, use different values for the end game.

Code: Select all

  +----+----+----+----+-----+----+---+----+ 
8 | 00 | 00 | 00 | 00 | 00 | 00 | 00 | 00 |
  +----+----+----+----+----+----+----+----+
7 | 10 | 12 | 16 | 20 | 20 | 16 | 12 | 10 |
  +----+----+----+----+----+----+----+----+
6 | 10 | 12 | 16 | 20 | 20 | 16 | 12 | 10 |
  +----+----+----+----+----+----+----+----+
5 | 10 | 12 | 16 | 20 | 20 | 16 | 12 | 10 |
  +----+----+----+----+----+----+----+----+
4 | 06 | 08 | 10 | 16 | 16 | 10 | 08 | 06 |
  +----+----+----+----+----+----+----+----+ 
3 | 04 | 06 | 08 | 10 | 10 | 08 | 06 | 04 | 
  +----+----+----+----+----+----+----+----+
2 | 02 | 04 | 04 | 10 | 10 | 04 | 04 | 02 |
  +----+----+----+----+----+----+----+----+
1 | 00 | 00 | 00 | 00 | 00 | 00 | 00 | 00 |
  +----+----+----+----+----+----+----+----+
     a    b    c    d    e    f    g    h

[hyatt]

On the overall possibilities for what seems to a "simple" feature, I refer you again to the isolated pawns example.

Fruit 2.1, Rybka 1.0 Beta, and Rybka 2.3.2a all give a penalty for an isolated pawn that depends on whether the file is half-open or closed.

I do that since 1982/83.

Because,

It's the right way of doing chess knowledge.

Vas is an IM, he knows.

I am so sorry for all your work.

------------

For me (today) the whole issue stands or falls with the part you did not comment, the synchronicity of the order of evaluations between Fruit and Rybka eval.

If there none, innocent.

If there is a clear synchronicity then guilty because that would be a second "0.0" bomb.

"isolated pawns" are not simply "those pawns that have no friendly pawn on either adjacent file. There's the concept of "artificially isolated". Then there's the question of what you do to the pawn. Chess books say "immobilize / blockade isolated pawns" so that they become a target on a fixed square, and so they can't advance enough to exchange one for an opponent's pawn and evaporate the problem. So there is not "just one way to evaluate isolated pawns". There are several. And I'll bet what I do (or did in past versions) is different than what you did, if you just use the bare-bones concept of "isolated". Isolated pawn on open file? weak. Unless opponent has no rooks to attack it. Isolated pawn fixed on a diagonal where the opponent has a bishop of that color? Weak unless there is no bishop. Etc... Lots of things to check, most do not do them all, particularly in the same order in the same way..

[Rebel]

Fruit 2.1, Rybka 1.0 Beta, and Rybka 2.3.2a all give a penalty for an isolated pawn that depends on whether the file is half-open or closed.

I do that since 1982/83.
Because,
It's the right way of doing chess knowledge.
Vas is an IM, he knows.
I am so sorry for all your work.

Your write-up in How REBEL Plays Chess suggests that, at least at some point, you did otherwise than Fruit/Rybka (which do not depend on the square).

Yes, there is an extra fixed penalty for an isolated pair.

I have no PS for double pawns, never was able to find a good reason for that.

[Rebel]

However what I would accept as damaging is an exact or a great similarity in the order of the evaluation ingredients. I had a quick look at the Fruit and Strelka EVAL and that doesn't appear the case, I leave this to the Rybka/Fruit/Strelka experts.

I don't think the evidence says much one way or the other. At the top level, the Rybka/Fruit orderings match a lot, but I can't say this is unexpected (one difference is that Rybka puts pawneval later, after prefetching the hash near the start.

I am not sure if we are talking about the same thing, neither am I a Fruit or Strelka expert. Anyway, I think, it's about:

EVAL.CPP (Fruit) int eval(const board_t * board)
EVAL.C (Strelka) int evaluate(struct pos_info_t * pos_info)

When evaluating the same, or even similar, order of all the aspects (mobility, king safety, pawn structure etc. etc.) that would be a strong indicator Rybka started its life as Fruit, yes ?

But is that the case ?

And so I think the opposite is also true.

Else it would imply Vas (a noname in 2005) forsaw a scenario he would become so famous that someone would re-engineer his program and another someone would see the synchronity in EVAL.C and thus he changed the order of all aspects in EVAL.C for obfuscation reasons. And that is a scenario I don't buy.

Can I conclude that one can take R4, reorder the evaluation function, and presto!, this is an original engine? [Since R4 has, of course, the "right way" of doing knowledge, for evaluation feature, and indeed choosing such in the first place?]

After the total Ryba=Fruit debate plus this discussion I fear potential copy boys will do just that

But the year of discussion is 2005.