but.. they're using a pretty big neural net themselves (NNEU) as far as I can tell? With datasets of hundreds of Gigs.
Doesn't this remove the significance of the matchup, that was supposed to be about Deep Learning vs more traditional chess bot methods?
Claiming that something that uses a neural net trained on hundreds of gigs of data isn't deep learning .. I mean it's possible, I don't know the details.
What is it about now, open vs closed source? Different methods of deep learning and big data fighting? (Both of these are also interesting ofc)
NNUE is much smaller than Leela's net, and has a much different architecture that's optimized more for CPU. Additionally, Leela uses Monte Carlo Tree Search and Stockfish uses Alpha/Beta pruning.
> Though definitely not directly comparable, dataset of GPT2-xl is 8 million web-pages.
This is irrelevant. You can train GPT3 on a smaller dataset, or a smaller model on the same dataset as GPT3.
> What I mean to say is that this is clearly deep learning.
It's been clear that neural network models are superior since Alpha Go. There's not "Deep Learning vs <something else>" anymore because the <something else> isn't competitive and no one is really working on it.
Its actually really small, mostly because bigger networks take longer to evaluate which slows down the search making it shallower and ending in a less clever algorithm.
NNUE is a 4 layer (1 input + 3 dense) integer only neural network.
It's just over 82,000 parameters.[1]
That's a very shallow, small NN - by comparison something like EfficientNet-B1[2] is 7.8M parameters, and that's considered a small network.
Claiming that something that uses a neural net trained on hundreds of gigs of data isn't deep learning .. I mean it's possible, I don't know the details.
What is it about now, open vs closed source? Different methods of deep learning and big data fighting? (Both of these are also interesting ofc)