Bandwidth vs. Computer Power

[Dave Mitchell]

I need to get the Nalimov Tablebase files for 3, 4, and 5 pieces, and am currently downloading them from Hyatt's UAB site.

Never has my DSL seemed so S-L-O-W!

On the other hand, I have an overclocked (but very stable), i7 with both Linux and Windows 7. (and 12GB RAM).

My question is would it be better to go ahead and generate these tablebase files, on the i7, rather than the tedious downloading?

Is there a better way to download these files, instead of this tedious one or two at a time, method?

Thanks so much!

[Sean Evans]

I need to get the Nalimov Tablebase files for 3, 4, and 5 pieces, and am currently downloading them from Hyatt's UAB site.

Never has my DSL seemed so S-L-O-W!

On the other hand, I have an overclocked (but very stable), i7 with both Linux and Windows 7. (and 12GB RAM).

My question is would it be better to go ahead and generate these tablebase files, on the i7, rather than the tedious downloading?

Is there a better way to download these files, instead of this tedious one or two at a time, method?

Thanks so much!

You will have a long wait the tbs are huge. Buy them at Chessbase.com

[Peter C]

3-4-5 Nalimov tablebases are only 7.5GB. I downloaded them in a few days manually (2 files at a time) and I don't have a particularly fast internet connection. Downloading them is probably the best option.

Peter

[Taner Altinsoy]

You don't need to download all as stated in the link below.

http://stark.forumotion.com/endgame-tab ... b-t436.htm

[Dave Mitchell]

Thanks for all the replies. I did finish d/l'ing the 3-4-5 from Hyatt at UAB.

I have the opinion (based on zero testing ), that the 6 man egtb files would add very little strength to a chess program.

It was the old Chessmaster egtb files that could be easily generated, not the Nalimov type.

[BB+]

I know of at least three different tablebase generators that are available.
* GCP was working on cleaning up the Nalimov code (including 6 piece -- the licensing question is also obscure here).
* Ballicora has some for Gaviota bases (no 6 piece yet).
* RobboBases for IvanHoe. For up to 5 pieces, this is easy (maybe 5 hours on a quad). For 6 pieces, it requires a machine with a lot of RAM (like 16GB), and takes a long time. I'm not sure it works under Windows either.
* There is also a more recent offering http://generatorchess.com that claims it can do 7-piece.

Other than RobboBases, I think that those that build the 6 piece do so from disk, and so the bottleneck is neither bandwidth nor computer power, but streaming I/O speed from the hard drive (which is about 50-100 MB/s).

[BB+]

I guess I should have read a few materials and kept up to date about RobboBase happenings. I could have given you more decent estimates.

First the ROBBO_BASE_BRIEF:

The totality of 6s will emerge in many days or weeks, and you need to tune the memory with 32GB in RAM enough but edgy.

A rule is to multiply by 150-200x for each next piece. The 5 piece take to build in nearly 15 hours of cpu at 3.41GB, and the 6 piece in 3-4 months of cpu when you have the memory (RAM) at 476GB.

So it seems that they take 3-4 cpu-months to build the 6s with 32GB of RAM [though my accounting yields that even 12GB should be OK, though maybe I'm disregarding disk access/caching with promotions], as opposed to downloading 476GB. From their wiki, they claim to have a download site set up for which the throughput can be as high as 10MB/s. At the more reasonable rate of 1MB/s it would take about a week, so my guess is that downloading will win on a strict time comparison in most cases. This should be true for any of the tablebase systems.

GCP was building a single 6 piece endgame in about a day, which seems to indicate around a calendar year to build them all. The Nalimov algorithm is slightly inferior to what (I presume) the RobboBases are using for many cases, as I think it uses the "grandfather" system as opposed to the "counting outs" system (see http://home.hccnet.nl/h.g.muller/EGTB.html for more about this, and he seems to have a 7-piece Leapfrog generator mentioned now). However, the main bottleneck for 6 pieces is I/O in any case, even with your 12GB of RAM.

Reading the original post again, I might mention that wget can be likely queued up to grab multiple files when downloading.

[Dave Mitchell]

Thanks for the info on wget - I could have really used it.

I do hate to see the chess engines fracturing the Tablebases into several different formats, each one unusable by so many other engines. It would see that two types could be standardized for: 1) Nalimov type, and 2) "The King" engine style bitbases (used by Chessmaster).

It's clear that d/l'ing the TB's was the better choice.

[ernest]

2) "The King" engine style bitbases (used by Chessmaster)

Since when are those bitbases?

[Dave Mitchell]

They may not be - it's been several years since I read anything about what TheKing engine uses. Fruit uses them, for sure (from the Fruit 2.1 readme.txt file):

As for Fruit I plan on using selected "bitbases" in the (very far)
future. For now some draws are recognised by the evaluation function,
and - despite the errors - this somewhat reduces the penalty for not
using tablebases.

: