Questions for BB about Rybka PST = Fruit PST

Code, algorithms, languages, construction...
User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Questions for BB about Rybka PST = Fruit PST

Post by Rebel » Thu Aug 18, 2011 9:00 am

mjlef wrote:
Rebel wrote:It's far more likely I will never enter an IGCA panel that first refuses to enter the only critical voice (how convenient) and then punish out of proportion without even considering their own role in the drama, not even have the politeness of an answer when you address it.
This is not true. I found the panel members to be very critical. Evidence was carefully examined, and corrected and expended when information was found missing or incomplete. Several panel members played devil's advocate, arguing for Vas's side. Stating that Chris wold have been the only "critical voice" implies the panel members were dishonest in their duties. I found the active panel members were serious and scientific, and certainly critical.
That's fine then Mark to hear there were critical voices although I hear other impressions as well, but what about your silence on the below,

http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=22550

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Questions for BB about Rybka PST = Fruit PST

Post by hyatt » Thu Aug 18, 2011 4:44 pm

Several points.

(1) Obfuscation. Vas intentionally misled everyone by doing the following: (a) reporting a false depth; (b) reporting false node counts; (c) reporting incomplete PVs to hide (a) most likely.

(2) several did suggest Rybka was a fruit clone because early versions had similarities. However, it would seem none took it seriously that an IM would resort to such tactics (obviously a mistake in hindsight).

(3) Strelka turned the world on edge when similarities to fruit were noticed and when Vas claimed it was his code.

Until that point there was no credible evidence.

Why wait until now? You signed the formal protest letter. Why did YOU wait until now to do so? Because you chose to let OTHERS do the investigation? And when you looked at what Zach and others had found, you thought "this is pretty bad stuff..."??? :) Disingenuous comment to wonder why it took so long, because you could have filed a protest 5 years ago if you wanted to. Until the letter you signed, there was NO formal protest filed with the ICGA. Do they just investigate out of the blue? Don't think so. For the other cases, there was plenty of suspicion and protest were filed quickly.

BTW critical panel comments led to lots of writing and rewriting. There were plenty of them and we kept at it until the objections went to zero as the report was refined to address them.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Questions for BB about Rybka PST = Fruit PST

Post by BB+ » Thu Aug 18, 2011 10:44 pm

Adam Hair wrote:I think some of the general perception that the Panel had no critical voices is the implied (actually I think that Bob voiced this opinion but I don't have the time to track it down) that the Panel was prosecutorial in nature. Thus, it may have been assumed that the Panel was only engaging in the act of collecting evidence against Vas and perhaps not focused on a critical examination of the evidence.
If you want to dummy up a "courtroom" schematic, Fabien was the plaintiff, VR was the accused, the Board was judge, and the Investigation Panel provided advisory expert opinions regarding technical matters. I'm not sure exactly what the "Secretariat" of the latter actually did, so I can't comment on "recusal" from that. Either FL or VR could challenge the findings of the Panel, as with most expert opinions in a courtroom setting. Neither chose do so in any substantial manner.

Again it's not clear to me where the misconception of the Panel as merely a prosecutional functionary came about. The Rybka Forum undoubtedly suggested such witch-hunting from day zero, while Bob probably mis-spoke about the Panel "being" a prosecution, when in fact it only had some aspects of that (particularly if VR wasn't going to address the situation with the Panel, but rather directly to the Board -- it then became more incumbent on the Panel to state the evidence for FL's case as demonstratively as possible, and VR could use his own experts).
Robert Hyatt wrote:(2) several did suggest Rybka was a fruit clone because early versions had similarities. However, it would seem none took it seriously that an IM would resort to such tactics (obviously a mistake in hindsight).
The most notable was Daniel Mehrmann, though after Rajlich's bluster to the contrary, he apologetically retracted. This post will probably be used in the FSF/Fabien legal action as an example of a competent "outside observer" who could detect (to his eye/testing) specific similarities [PST/mobility] between the two programs. The obfuscation issue will also of course be mentioned therein.

For Ed, I think the best course of action for you and others, is to follow Fabien's protocol and write an open letter to the ICGA(!) [either in your name, or on VR's behalf]. This should contain a cogent description of why you think the case should be reviewed (that is, why the current resolution is insufficient), with some sort of specific suggestion as to a new method of resolution [e.g., set up a committee of competent/noted persons like Schaeffer, Thompson, Romstad (who defended VR vehemently in the beginning, then changed his mind), let FL/VR argue their cases with proper legal procedure, have the committee produce a document explaining its decision, etc.]. I would hope that the "cogent description" would contain more than just inert suggestions that there was some impropriety in the Panel, or something of that nature.

I realise that English is not your native tongue, so if you want help in that matter in drafting such an Open Letter, I'm sure that I or someone else would be happy to assist. I also realise you have about zero desire to engage the ICGA, but unless you address the Open Letter directly to them [and not to the CC community as a whole], it would seem more like propaganda and/or showmanship, and be rather futile.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Questions for BB about Rybka PST = Fruit PST

Post by BB+ » Thu Aug 18, 2011 11:28 pm

Miguel Ballicora wrote:BTW, This may be unrelated, but when I look at Fruit 1.0, it looked to me that Fabien was using a similar mental approach, with a higher edge and corner penalty. Just using the whole 2D-table unrolled, and the bonuses looked more disorganized, but they were there (diagonals were present, bonus for 6th rank for knight were explicit, but he later expressed them neatly with a vector etc.). I think that Fruit 2.1 and 1.0 were conceptually very related.
Due to my method of regenerating the "wave" nature of KnightRank as 2 less than (r-(2<<(r^6)))&7, I found your "SixthRank" parameter to be quite odd when I saw it, but that's just a side comment. I think the PSTs are conceptually related (and indeed, aren't most PSTs [maybe they fall into 2 or 3 "equivalence classes" of similarity], particularly those from the same author?), though quite different in their instrumentation [and as you say, Fruit 1.0 is more "disorganized" -- FL told me that he deliberately tried to reduce the "dimensionality" of the question, so as to tune it easier]. Of course, with Rybka 1.0 Beta we only have the end result of PST, so we can't compare methods directly.

Here is a guess as to how likely the Fruit/Rybka match for "centralisation ramping parameters" should be "at random" -- it uses lots of assumptions, but I think most are reasonable. What are the common elements of the Fruit/Rybka PST? I think it is fair to eliminate KingOpening, as though the file array is the same, the rank differs. Similarly with PawnOpening (the difference in "bonus squares"), and I don't think the fact that Rybka uses nontrivial PawnEndgame PST is anything to speak of. Similarly, I think QueenOpening can be eliminated, as Fruit 2.1 has a parameter that is 0, thus obscuring the picture. With RookEndgame PST being zero, this leaves 7 PSTs, involving 5 pieces. I think the -2,-1,0,1 of RookOpening is reasonable to keep, as there at least is a direct comparison (no matter how perfunctory). One can try to extract "partial information" from the some of the above, but I will not.

So the next step is to choose a "mainstream" ramping array. I think there are at least 3 reasonable ones (my glance at ~50 open-source programs suggests a greater number, but I won't dispute), of which both chose -3,-1,0,1. As you say, the 5 or 7 choices of when to use this are not independent. One can argue that the "times to vary" from the mainstream array consist of either 7-choose-3 (2N and 1R array), or 5-choose-2 (vary for N and R). This is at least a 1 in 10 chance that the "varying" from the mainstream occurs at the same places -- it seems rather unclear to me whether one could argue that it is "natural" that N/R "should" vary from the others, particularly as any "extra" centralisation impact (for N) could be obtained in a similar way by (say) doubling everything. Then there is the choice of "varying" element. If there were 3 reasonable arrays to begin, there is a 1 in 2 chance for each variance to match (I think these are reasonably independent). This gives a total of 3 x 10 x 2 x 2, or about 1 in 120 for the "centralisation" structural match in Fruit/Rybka.

One can additionally adduce the KnightRank, where (say) it might be logical for it to be more "mainstream", such as -3,-1,0,1,2,2,1,0, but I think the argument suffices w/o this.

This is an imperfect mathematical model of the situation, but I think it gives a ballpark estimate. If nothing else, it shows that VR had "something to answer" about the situation, and more specifically than just a vague "took lots of things" comment, or an assertion that this was merely "common chess knowledge" or the like.

ernest
Posts: 247
Joined: Thu Sep 02, 2010 10:33 am

Re: Questions for BB about Rybka PST = Fruit PST

Post by ernest » Fri Aug 19, 2011 8:03 pm

BB+ wrote:The most notable was Daniel Mehrmann,.
Interesting view, there, by Chrilly Donninger.
I wonder what he thinks now! 8-)

mjlef
Posts: 43
Joined: Thu Jun 10, 2010 6:51 pm
Real Name: Mark Lefler

Re: Questions for BB about Rybka PST = Fruit PST

Post by mjlef » Sat Aug 20, 2011 10:50 pm

Rebel wrote:
mjlef wrote:
Rebel wrote:It's far more likely I will never enter an IGCA panel that first refuses to enter the only critical voice (how convenient) and then punish out of proportion without even considering their own role in the drama, not even have the politeness of an answer when you address it.
This is not true. I found the panel members to be very critical. Evidence was carefully examined, and corrected and expended when information was found missing or incomplete. Several panel members played devil's advocate, arguing for Vas's side. Stating that Chris wold have been the only "critical voice" implies the panel members were dishonest in their duties. I found the active panel members were serious and scientific, and certainly critical.
That's fine then Mark to hear there were critical voices although I hear other impressions as well, but what about your silence on the below,

http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=22550
I have been traveling, and so did not even read that thread until just now. In fact, it is so long I only just read a part of it. To summarize what I think you are saying:

a. ICGA somehow did something wrong by not prosecuting Vas sooner. Well, no panel was formed until Fabien complained, although I suspect if someone had put the data together earlier and then sent it to David Levy, he might have acted sooner. I am not sure it would be fair for the ICGA to even try to investigate every allegation, so there needs to be some minimal criteria to begin an investigation. When the programmers got together and sent a letter to Levy, and with Fabien's complaint, this seemed to be enough. Is there some other "trigger event" you think should have started the process?

b. You wanted Vas's punishment "softened". Note the Secretariat report did not have a ban for life like the final ICGA Board decision. It did have a recommendation that program's by Vas not be allowed to compete until Vas could show the programs were no longer derivatives, and not a full lifetime ban. I am sure Levy and the ICGA Board would entertain lessening the "punishment" if people could provide good reasons for it. SOme contrition from Vas would probably help too, but that is up to the ICGA Board.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Questions for BB about Rybka PST = Fruit PST

Post by Rebel » Sat Aug 20, 2011 11:00 pm

Thanks for reply Mark.

You are actually saying that in cycling the organization of the event should only do a doping test after a complaint of a competitor.

wgarvin
Posts: 47
Joined: Thu Jul 01, 2010 3:51 pm
Real Name: Wylie Garvin

Re: Questions for BB about Rybka PST = Fruit PST

Post by wgarvin » Sun Aug 21, 2011 11:33 pm

Rebel wrote:Thanks for reply Mark.

You are actually saying that in cycling the organization of the event should only do a doping test after a complaint of a competitor.
I don't think he said anything of the sort. It should be clear to you that computer chess has historically been run in a way that assumed 'good faith' by the competitors. What they wrote on their entry forms has been assumed to be the truth, unless there was some good reason to doubt it and investigate.

If you think this should be changed, then thats one thing. Some authors might be reluctant to compete though, if they are forced to jump through too many hoops.

Prima
Posts: 328
Joined: Tue Dec 14, 2010 6:12 am

Re: Questions for BB about Rybka PST = Fruit PST

Post by Prima » Mon Aug 22, 2011 12:32 am

mjlef wrote:I have been traveling, and so did not even read that thread until just now. In fact, it is so long I only just read a part of it. To summarize what I think you are saying:

a. ICGA somehow did something wrong by not prosecuting Vas sooner. Well, no panel was formed until Fabien complained, although I suspect if someone had put the data together earlier and then sent it to David Levy, he might have acted sooner. I am not sure it would be fair for the ICGA to even try to investigate every allegation, so there needs to be some minimal criteria to begin an investigation. When the programmers got together and sent a letter to Levy, and with Fabien's complaint, this seemed to be enough. Is there some other "trigger event" you think should have started the process?

b. You wanted Vas's punishment "softened". Note the Secretariat report did not have a ban for life like the final ICGA Board decision. It did have a recommendation that program's by Vas not be allowed to compete until Vas could show the programs were no longer derivatives, and not a full lifetime ban. I am sure Levy and the ICGA Board would entertain lessening the "punishment" if people could provide good reasons for it. SOme contrition from Vas would probably help too, but that is up to the ICGA Board.
The idea that Rybka be not allowed in competition until Vas can show Rybka is no longer a derivative makes sense to me. The problem (for Vas) is that, he won't want to disprove Rybka as a derivative and so the ban is likely to stay.

Of course don't expect most Rybka at-all-cost-zealots to understand this simple concept.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Questions for BB about Rybka PST = Fruit PST

Post by BB+ » Mon Aug 22, 2011 4:35 am

a. ICGA somehow did something wrong by not prosecuting Vas sooner. Well, no panel was formed until Fabien complained, although I suspect if someone had put the data together earlier and then sent it to David Levy, he might have acted sooner. I am not sure it would be fair for the ICGA to even try to investigate every allegation, so there needs to be some minimal criteria to begin an investigation. When the programmers got together and sent a letter to Levy, and with Fabien's complaint, this seemed to be enough. Is there some other "trigger event" you think should have started the process?
I might also note that it was difficult (if not nearly impossible) to get many people even to look at the Rybka/Fruit evidence until Fabien re-appeared this January. E.g., Mark Uniacke only then spot-checked some details (saying it was the first time he had really done any disassembly). Outside the aficionado-based CC forums [and even there only among a select group], there were at best what could be called rumors. For instance, Arne Moll's ChessVibes article (Mar 2010) about chess engine controversies mentioned R/F, but he seems to regard it as something closer to a conspiracy theory to distract from the R3/IPPOLIT issue.

Back in 2008, when Zach first publicised the issue (perhaps prematurely), amidst the TalkChess spammings there were three main contentions:
*) Fabien said he was OK with the situation (from his email to Dann concerning Strelka)
*) Tournaments are the province of organisations like the ICGA, and there is no apparent problem there, especially if Fabien is OK with it. [And: if they aren't doing anything, no one can expect the rating lists to do anything either].
*) The FSF is unlikely to do anything, particularly about UCI parsing code in an obsolete beta version, and when Fabien doesn't mind.
So it seems that 2011 should be a year of "3 strikes" against this.

I agree with Wylie's response to the other comment -- computer chess, at least in 2005/6, tended to trust authors, especially when they make auxiliary statements such as: The Rybka source code is original and pre-dates all of the Fruit releases and As far as I know, Rybka has a very original search and evaluation framework.

Post Reply