OpenChess

The old v2 experience file format contains :

- 8 bytes to store the position's key (ex : [B4 D3 0C D1 5A 43 43 2D] = [rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1] for the start position)



- 4 bytes to store the move + promoted piece (ex : [00 00 03 1C] = [0 000 001(2) 100(e) 011(4) 100(e)] for the "e2e4" move)



- 4 bytes to store the move score (ex : [00 00 00 46] = 33 centipawns, it means "+0.33/4")



- 4 bytes to store the score depth (ex : [00 00 00 04] = 4, it means "+0.33/4")



- 4 bytes to store the count value (ex : [00 00 00 01] = 1, it means the "e2e4" move was played 1 time)

deeds wrote: ↑

Thu Jul 06, 2023 2:14 pm

- 8 bytes to store the position's key (ex : [B4 D3 0C D1 5A 43 43 2D] = [rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1] for the start position)

v2 : If a position contains several moves, the 8 bytes of its key are duplicated as many times as there are moves.
Example with the starting position which contains its 20 legal moves :
v2 : 20 * 24 bytes [8 (key) + 4 (move) + 4 (score) + 4 (depth) + 4 (count)] = 480 bytes

v3: We could only store the 8 bytes of the position key once and use a 9th byte (=0 to 255) to indicate the number of moves to load.
Example with the starting position which contains its 20 legal moves :
v3: 8 (key) + 1 (nb moves) + 20 * 16 [4 (move) + 4 (score) + 4 (depth) + 4 (count)] = 329 bytes (-31%)

deeds wrote: ↑

Thu Jul 06, 2023 2:14 pm

- 4 bytes to store the move + promoted piece (ex : [00 00 03 1C] = [0 000 001(2) 100(e) 011(4) 100(e)] for the "e2e4" move)

v2: All moves consume 32 bits or 4 bytes

v3: All possible moves can fit in 15 bits or 2 bytes (-50%):
- 12 bits are sufficient from A(000) 1(000) to H(111) 8(111)
- 3 bits are enough for special moves, no promotion (000) or promotion to queen (001), to rook (010), to bishop (011), to knight (100), short castling (101), long castling (110)

deeds wrote: ↑

Thu Jul 06, 2023 2:14 pm

- 4 bytes to store the move score (ex : [00 00 00 46] = 33 centipawns, it means "+0.33/4")

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 3 bytes (-25%) because generally at chess :
- the normal scores go from "-100'000" cp (-1000.00) to "+100'000" cp (+1000.00) so we need a range of 200'001 values
- the mate scores go from "mate -1000" to "mate +1000" so we need an additional range of 2'000 values
From [00 00 00] to [04 00 00] there are already 262'144 available values.
Until [FF FF FF], there are 16'777'215 available values so we even have room to handle tablebases scores.

deeds wrote: ↑

Thu Jul 06, 2023 2:14 pm

- 4 bytes to store the score depth (ex : [00 00 00 04] = 4, it means "+0.33/4")

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 1 byte (-75%) because so far most engines are capping at D245.

deeds wrote: ↑

Thu Jul 06, 2023 2:14 pm

- 4 bytes to store the count value (ex : [00 00 00 01] = 1, it means the "e2e4" move was played 1 time)

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 2 bytes (-50%) because the learning needs at least 500 games/opening so the first learned moves can be played at least 500 times.

deeds wrote: ↑

Fri Jul 07, 2023 6:28 am

v3 : We only need 2 bytes (-50%) because the learning needs at least 500 games/opening so the first learned moves can be played at least 500 times.

v3 : We only need 1 byte (-75%) because if the move is played 255 times, it's even sufficient to obtain its effectiveness

In sum,

v2 : we have 8 bytes/position + 16 bytes/moves
ex : 1 move/position = 24 bytes, 2 moves/position = 48 bytes, 4 moves/position = 96 bytes, etc.

v3 : we have 9 bytes/position + 7 bytes/moves
ex : 1 move/position = 16 bytes (-33%), 2 moves/position = 23 bytes (-52%), 4 moves/position = 37 bytes (-61%), etc.

@deeds what is meant by experience v3?
no code has been released with further learning update by the author of SugaR, am I getting confused?
If so, can you explain better, thanks

Robert57 wrote: ↑

Sat Jul 08, 2023 12:28 pm

@deeds what is meant by experience v3?

SugaR Experience version 1 contains :
- position's key
- move
- score
- depth
- count
- next score
- next 10th score

SugaR Experience version 2 contains :
- position's key
- move
- score
- depth
- count

Etc.

Robert57 wrote: ↑

Sat Jul 08, 2023 12:28 pm

no code has been released with further learning update by the author of SugaR, am I getting confused?

None of the authors of SugaR, Eman, BrainLearn is the author of the learning code. I bet none of them train their engines with the 500 games/opening they advise lol. At best, they fill their experience files by playing on chess servers with default settings (=training mode).

With D.C.S. i already can read/write all their experience file formats but as i'm planning to add a learning feature to any non-learning engines, i'm already working on the experience version 3.