Support for polyglot books

[Sopel97]

I think this was brought up multiple times. With the reason against always being "can use some fork". However recently there are no forks that are up to master, or confirmed to be as strong as master, and people keep asking about polyglot book support in stockfish, and I generally think that it would be a nice feature to have.

[noobpwnftw]

Main problem: I don't like their big hash IV table in code. If we use our own, then it won't load those books.

[Sopel97]

Well, these values are standardized, and maybe for the better. And it's just 8kB.

However if we really want to insist on something along "yes but it needs to be better than polyglot" then perhaps it's the right place for this working draft https://docs.google.com/document/d/1qSj4mKTxCgfKEwwMOqTE1OPa7wYYJSflRTyL04F41NE/edit?usp=sharing

[ianfab]

Besides the arguments that for code complexity reasons it should be maintained in a fork and that polyglot is not ideal, as far as I remember one of the main reasons always was that book support is supposed to be in the GUI, not the engine, and by supporting it Stockfish would encourage the opposite development. I concur with this, at least if the book is only used at the root where it basically has nothing to do with the engine logic. A "book" that is used in the search of course would have to be in the engine, but as described in the link above this is more like a TT dump than an "opening book" and polyglot of course is not an option for that.

[joergoster]

This may be of some interest: f4dcec0

[vdbergh]

Are people really asking for this? What is their use case?

Even if SF had polyglot book support, the user would still need to select the book in the GUI. So whether the book is handled by SF or the GUI does not seem to make any difference from the point of view of user friendliness.

[Sopel97]

I was under the impression that the search can use polyglot books?

[noobpwnftw]

No, too slow for search.

[Sopel97]

if the book is loaded into memory then a query should be just a binary search by key?

[noobpwnftw]

Still dirt slow, I tried this before: #1883

[Sopel97]

Okay... I guess I start working on something better and present some results as otherwise we likely won't move forward with this.

[noobpwnftw]

Nevertheless, as you can see people hate books more than caring about quality of data or their practical use.

[syzygy1]

Are people really asking for this? What is their use case?

It seems the main use case is Fritz: #3145

[NightlyKing]

The main cause for a self altering book is obvious: users with "big boy hardware" can massively help out weak/average hardware and the engine can still improve by further use (even on said weak hardware). It won't be allowed for TCEC but that's fine. While some devs may not care about this feature, it comes up regularly as an issue or comment in forums. I think it's worth to have something like this. I bet we lose less time by having to maintain its code than by always finding reasons in threads like these to not implement it.

[noobpwnftw]

Add book or else, I don't really care about your opinions.

[tillchess]

use Polyglot adapter then you have Polyglot books with any engine and more options

[Sopel97]

use Polyglot adapter then you have Polyglot books with any engine and more options

Sure, I can also manually enter the moves. This will not allow the engine to use the book for analysis though.

[Sopel97]

use Polyglot adapter then you have Polyglot books with any engine and more options

Sure, I can also manually enter the moves. This will not allow the engine to use the book for analysis though.

Polyglot books for analysis is a strange use case.

Do you consider TT a strange use case for analysis?

[stevemaughan]

In Maverick when the application starts I scan the exe's folder to find any "*.bin" polyglot books. I then dynamically create a UCI option that enables the user to select whichever book they choose from a dropdown list. The default is the largest book in the folder. The code isn't too complex and works remarkably well.

[Delphi-Coder]

Stokfish already carrying huge NNUE database (10 times larger than original binary) with its code. Keeping some really useful features such as polyglot book support (de-facto standard for all computer chess community) and also perft command should not be a big deal for developers.