
BlokusDuo ML Agents
MCTS guided by a convolutional network won 66% of games as player 1 and 48% as player 2 against standard MCTS, and it was the only agent that consistently beat experienced human players.
The problem
Blokus Duo has a 14×14 board, 21 polyomino pieces per player, and a move space that explodes early. Placement decisions are permanent, rewards are sparse, and naive policy gradients learn to place pieces without really learning territory control.
What I did
I led the team that built and compared four agents for Blokus Duo: minimax, Monte Carlo Tree Search, MCTS guided by convolutional networks, and PPO. For the CNN-guided MCTS, a policy network trained on data from hundreds of thousands of simulated games, run across 46 lab machines, weighted the search toward promising moves. Every agent played random agents, the other agents, and human players.
How it works
The CNN-guided MCTS uses two Keras networks. A value network estimates the chance of winning from a board state, averaged with a hand-built heuristic score. A policy network outputs probabilities over all 32,928 possible actions; illegal moves are masked out and the rest re-normalized so every legal move keeps some chance of being explored. PPO uses a clipped objective with a reward of +1 for a win and −1 for a loss, trained for about 100,000 timesteps.
What happened
MCTS with a CNN produced the highest win rates of the four methods: 66% as player 1 and 48% as player 2 against standard MCTS, over about 2,500,000 simulations across the non-PPO matchups. It was the only agent that consistently beat experienced human players.