The bars above the board show how the bot's search split its time between columns.
This is an AlphaZero-style Connect 4 bot I wrote from scratch: a small residual network that learned the game by playing itself, with Monte Carlo tree search on top. It all runs in your browser, there's no server. The code and a full writeup are on GitHub.
The bars above the board show how the bot's search split its time between columns.
The network looks at the board as two 6×7 grids (my pieces, their pieces) and outputs a probability for each column plus a guess at who's winning. By itself it's fast but shallow. The tree search uses those outputs to decide which lines are worth exploring, and after a few hundred simulations the visit counts give a much better move than the network's first guess. That better move is what the network gets trained to copy, which is the whole AlphaZero loop: the search improves the network and the network improves the search.
The self-play network is what I got out of about 16 hours on my laptop GPU. It beats my alpha-beta engine at every depth I tested but it's far from perfect. I wanted to know if that was because of the network or because of not enough training data, so I trained the same architecture on positions labeled by a perfect solver. That's the second opponent, and it makes about a quarter as many mistakes. So the network was fine, the data was the problem.
The engine, search and network here are ports of the Python code in the repo. The JavaScript is tested against the Python on a few hundred positions so they play the same moves, and the networks are exported to ONNX and run with onnxruntime-web.