Questions for BB about Rybka PST = Fruit PST

[Rebel]

I don't buy any of the 5.5-month-timeframe argumentation either. Converting to bitboards is a very methodical task. I would guess it would take me at most a few days to convert Fruit to fully bitboard without any bitboard code to start from. Even if we are extremely conservative and say one month for a bitboard conversion, that's still several months to play around with tuning and search tricks.

Okay, let's talk about the time frame of 5½ months and if this is doable, even for a brilliant guy like Vas.

You do realize you only addressed one point of the picture I painted above, the alleged bit-board conversion.

On that, 2 days looks mission impossible. You can not edit everything in one round, that new program will crash and crash and the search for the inevitable typo's and small mistakes you have made will drive you nuts for days if not weeks. I have done several conversions from one platform no another one, the latest conversion in 1993 from C to ASM. You do it in small portions, routine by routine and then check the new executable with a testset of selected positions to ensure that every position outputs the same number of nodes. It maybe requires some discipline because it's boring but IMO it's faster because you find the mistakes very easily and more important it's certainly the best thing to do for health reasons.

[BB+]

Did he even read the rule requiring "original work"?

I'm digressing a bit here, but it always personally boggles my mind that people might enter important competitions w/o a good understanding (or clarification) of the rules, but I realise that not everyone has my worldview [and/or cultural perspective of what rules should do/say]. It does get a bit annoying, though, when they complain afterwards that the rules were "unclear" or something. That would be my new #1 rule for ICGA competitions: If you have a question about the rules, ask ahead of time. [As an aside, Rajlich had a similar incident of a problem with interpretation of rules in the 8th PAL/CSS Freestyle event in Apr 2008].

A related question would be whether Rajlich had familiarised himself with the ICGA LION++ decision regarding Fruit. My guess he was too busy to care. It does seem, though, he was familiar with the ElChinito copying of Crafty (in 2004) [he commented in the TalkChess thread], with its similarities to Crafty/preRybka, but then again, that's not the Rybka/Fruit issue so by 2006 this is likely moot as a mental consideration.

Vas did write a strong program, but I doubt he could have made it so strong in that timeframe if he had written its eval on his own from scratch. It might never have become as strong, or at least it probably would have taken 1-2 years longer.

I think I would use this as an analogue. Say I have a new idea for a comedy TV series. I have specific skits written out that I promote at tradeshows, some people are interested, etc. You are also interested in making a comedy TV series, and in fact, you have many good ideas, and big plans to boot. But you take a "short cut", in that for your pilot episode, half the material is taken (perhaps mostly loosely, some more specifically) from one of my works. Your series goes on to much fame, in good part because of your further writing skills, industry contacts, and more. However, there was still a sense that getting your characters "over" with the audience was done via the pilot, which borrowed heavily from my script, etc.

If this sounds arcane, I think there was/is some guy who gave Matt Groening an idea regarding The Simpsons 20+ years ago [during the planning phase], and still gets a (substantial) royalty for each episode.

[BB+]

I think I already mentioned [just as a possibility] back in February that Fruit could conceivably be taken to be a "programming guide", but no one has really pushed that idea, other than maybe Chris at some point.

This is correct. I have been trying to push Chris into that further but time is not on his side. You can see my 5½ month argument as a sort of pilot balloon on this. It's not unlikely Vas at the time of the Fruit sources release already had a full functioning bit-board engine of his own and then "went forward and backwards and took many things".

So shoot

From my point of view, I just can't come up with a mental narrative of events along these lines that passes my "laugh test" so as to make it public. I would be willing (indeed perhaps eager) to hear something specific from Chris (or you), but at this point at least, I haven't heard anything that I'd classify much higher than "quite doubtful", let alone something that should be pursued too much further. Additionally, even if there is an alternative explanation apart from what seems (quite) the most likely from the evidence, I'm not sure that it helps much for us to "guess" at what it might be, as opposed to Rajlich simply stating it. In short, I'm just not (personally) much interested in trying to "fill in the gaps" to a trial balloon, particularly if I find it to be an unenviable underdog already at the starting gate.

There were debates that vaguely touched on this in the Panel: as the consensus was that the Fruit parts that were taken were rather specific [on the scale of things ranging from "direct clone" to "general chess/programming knowledge"], this would lead me to conclude that Fruit was more than a "guide" in Rybka development.

[BB+]

Miguel came up with a different way of making his point, good enough that I will copy it here:

#if 0
/* use this to get Fruit numbers */
static const int TaxicabOpening = 10;
static const int RankPenaltyTo6 = 5;
static const int TaxicabEndgame = 10;
static const int NormalizeOpening = -25;
static const int NormalizeEndgame = -40;
static const int CenterCorrectionOpening = -5;
static const int CenterCorrectionEndgame = -5;
static const int CornerTrapped = 100;
#else
/* use this to get R1 Numbers */
static const int TaxicabOpening = 694;
static const int RankPenaltyTo6 = 358;
static const int TaxicabEndgame = 112;
static const int NormalizeOpening = -1702;
static const int NormalizeEndgame = -448;
static const int CenterCorrectionOpening = -347;
static const int CenterCorrectionEndgame = -56;
static const int CornerTrapped = 3200;
#endif

One can note that both Fruit 2.1 and Rybka 1.0 Beta have the following "small" linear relations between his variables:

TaxicabOpening is -2 times CenterCorrectionOpening
TaxicabEndgame is -2 times CenterCorrectionEndgame
NormalizeEndgame is -4 times TaxicabEndgame
NormalizeOpening is 3 times RankPenaltyTo6 minus 4 times TaxicabOpening

This speaks against "independence" to some degree, though personally I think it's more clear via the KnightRank//Line arrays. Subtracting off the 4 "small" linear relations from the 8 dimensions (variables), one recovers the same number (4) as the count of nonzero Knight PST parameters in Fruit (KnightCentreOpening/KnightCentreEndgame/KnightRankOpening/KnightTrapped).

[zwegner]

I don't buy any of the 5.5-month-timeframe argumentation either. Converting to bitboards is a very methodical task. I would guess it would take me at most a few days to convert Fruit to fully bitboard without any bitboard code to start from. Even if we are extremely conservative and say one month for a bitboard conversion, that's still several months to play around with tuning and search tricks.

Okay, let's talk about the time frame of 5½ months and if this is doable, even for a brilliant guy like Vas.

You do realize you only addressed one point of the picture I painted above, the alleged bit-board conversion.

On that, 2 days looks mission impossible. You can not edit everything in one round, that new program will crash and crash and the search for the inevitable typo's and small mistakes you have made will drive you nuts for days if not weeks. I have done several conversions from one platform no another one, the latest conversion in 1993 from C to ASM. You do it in small portions, routine by routine and then check the new executable with a testset of selected positions to ensure that every position outputs the same number of nodes. It maybe requires some discipline because it's boring but IMO it's faster because you find the mistakes very easily and more important it's certainly the best thing to do for health reasons.

But I did address both parts: I said "that's still several months to play around with tuning and search tricks". Which is basically all the changes from Fruit. Throw in a material table, which probably existed beforehand. What else is there?

I doubt it would take more than two days, and certainly not more than a week or so. I would do it myself just to prove a point if I had two days to waste. My hands need to take a rest anyways, I've been doing enough marathon coding the past week or so...

[Rebel]

From my point of view, I just can't come up with a mental narrative of events along these lines that passes my "laugh test" so as to make it public. I would be willing (indeed perhaps eager) to hear something specific from Chris (or you), but at this point at least, I haven't heard anything that I'd classify much higher than "quite doubtful", let alone something that should be pursued too much further. Additionally, even if there is an alternative explanation apart from what seems (quite) the most likely from the evidence, I'm not sure that it helps much for us to "guess" at what it might be, as opposed to Rajlich simply stating it. In short, I'm just not (personally) much interested in trying to "fill in the gaps" to a trial balloon, particularly if I find it to be an unenviable underdog already at the starting gate.

Laughable is not my dictionary when talking to you. Instead I expected a point by point refutation of the points I addressed.

[Rebel]

I don't buy any of the 5.5-month-timeframe argumentation either. Converting to bitboards is a very methodical task. I would guess it would take me at most a few days to convert Fruit to fully bitboard without any bitboard code to start from. Even if we are extremely conservative and say one month for a bitboard conversion, that's still several months to play around with tuning and search tricks.

Okay, let's talk about the time frame of 5½ months and if this is doable, even for a brilliant guy like Vas.

You do realize you only addressed one point of the picture I painted above, the alleged bit-board conversion.

On that, 2 days looks mission impossible. You can not edit everything in one round, that new program will crash and crash and the search for the inevitable typo's and small mistakes you have made will drive you nuts for days if not weeks. I have done several conversions from one platform no another one, the latest conversion in 1993 from C to ASM. You do it in small portions, routine by routine and then check the new executable with a testset of selected positions to ensure that every position outputs the same number of nodes. It maybe requires some discipline because it's boring but IMO it's faster because you find the mistakes very easily and more important it's certainly the best thing to do for health reasons.

But I did address both parts: I said "that's still several months to play around with tuning and search tricks". Which is basically all the changes from Fruit. Throw in a material table, which probably existed beforehand. What else is there?

I doubt it would take more than two days, and certainly not more than a week or so. I would do it myself just to prove a point if I had two days to waste. My hands need to take a rest anyways, I've been doing enough marathon coding the past week or so...

No, like Mark you did not address all the points I made, just one.

But okay, if you have no time fine, I can understand that. Nevertheless I think I have a valid hypothesis.

And about the 2 days, that includes the flawless typing of all the tables needed for bitboards ?

[BB+]

Laughable is not my dictionary when talking to you. Instead I expected a point by point refutation of the points I addressed.

I agree with Zach that 5.5 months is more than enough time. Do you have a specific argument why this shouldn't be true? I would think a month or two [of 8+ hours/day] should get a "Rybka 1.0 test bed" from Fruit up and going. Throw in the material-imbalance table, a few new pruning ideas, test for a few months, then [obfuscate nodes and depth and] hurriedly release, before putting in underpromotion and working out mate scoring [the first version had problems with KQQ vs K]... As to the "boring" way of implementing a programming (to make it bug-free), Tord Romstad (someone for whom VR had shown notable admiration at the time) was championing things along these lines back then (not sure it was exactly unit testing). .

It's not unlikely Vas at the time of the Fruit sources release already had a full functioning bit-board engine of his own and then "went forward and backwards and took many things".

I think I addressed this in a previous post. He does indeed seem to have had a full functioning bit-board engine [perhaps not wholly of his own], but the evidence is quite strong that this did not form the primary basis of Rybka 1.0 Beta.

[BB+]

Miguel came up with a different way of making his point, good enough that I will copy it here:

Just to make this more clear: I wrote that Fruit's PSTs could be derived from (and indeed are derived from): a specific algorithmic process, an array of weights (like -3 -1 0 1), and however many tunable parameters. What Miguel did (IMO) was point out that the array of weights could instead be embedded in the algorithmic process (via Taxicab considerations, and the like). I don't know whether it is worthwhile to debate which method has more/less "information content", as the process itself must be included in any such comparison. The most confusing thing I found about his presentation is that he doesn't specify the various linear relations between his variables, in the manner that both Rybka and Fruit have, and this can tend to an over-parametrisation of the system, particularly when comparing to other engines.

I don't think this changes the conclusion about the PST evidence (which, as I say, never was even considered by the Panel). Namely, whether one views it an as array plus a process, or just an algorithmic process, what is being done is an encoding of some specific rendition of general chess knowledge (such as "centralisation"). There are relevant 11 PSTs, of which 8 of them have noticeable similarity in their renditions of this.

[BB+]

As to the "boring" way of implementing a programming (to make it bug-free), Tord Romstad (someone for whom VR had shown notable admiration at the time) was championing things along these lines back then (not sure it was exactly unit testing). .

Found a relevant quotation about this (see VR's parenthetical aside):

Subject: Re: Interesting quiescence position
From: Vasik Rajlich
Message Number: 350646
Date: February 22, 2004 at 06:49:33
[...]
I did a number of experiments which involved beefing up the quiescence search and extending aggressively, and the results were disappointing. However, I had two huge bugs in my code, and my testing methodology was very inadequate, so this issue needs revisiting. (Computer chess lesson #1: never work too fast, and validate everything. If it's not proven, consider it wrong.) Generally though the tactical and positional search should be kept balanced. If you overdo the q-search, you undermine the balance which iterative deepening tries to give you. [...]