# Teaching local LLMs to play chess.

> Teaching small models to play chess: SFT, GRPO with a curriculum and a process reward, then Liquid's d1-omni-600M, which scores 62.5% against Stockfish 1700 with a small search.



- [./play](https://www.manoletre.com/chess/play): play against a finetuned d1-omni-600M
- [./encoder](https://www.manoletre.com/chess/encoder.md): extract chess data out of plain text

## tl;dr

This is a chronicle of me playing with small language models by Liquid AI, trying to teach them to play chess.

I started teaching LFM-2.5-1.2b to play chess (SFT, then GRPO with and without a curriculum and process reward). But I hit a ceiling on my [puzzle benchmark](https://huggingface.co/datasets/manoletre/chess-puzzles-1k) and switched to training a decision model instead (d1-omni-600M) where I needed less data and compute to get good results.

[play against the final model!](https://www.manoletre.com/chess/play)

## evaluating

Chess puzzles are a good measure of raw chess reasoning ability. Of course at the end I care about performance in playing chess, but it takes longer to test the model on 20-30 games, and even on those 20-30 games many possible positions are not reached.

I therefore built a benchmark from the [Lichess puzzle database](https://database.lichess.org/#puzzles) with 1,000 puzzles. It was important to me that it contained varied situations and difficulties. The benchmark contains 20 puzzles for each of 50 themes (mating patterns, tactics, endgames, mates in 3 to 5), spread evenly over five difficulty bands from a 399 to a 2792 Lichess rating. It's [on Hugging Face](https://huggingface.co/datasets/manoletre/chess-puzzles-1k).

*Figure: The puzzle benchmark tried to be as balanced as possible in both puzzle difficulty and puzzle type. The 6 themes on the right don't have 4 puzzles in every band.*

A 50 × 20 grid: 50 themes, 20 puzzles each, 4 in each of 5 rating bands (399–980, 981–1289, 1290–1611, 1612–2000, 2001–2792).

I did many experiments. This is how all of them perform on my benchmark:

*Figure: Performance of every model I finetuned on my benchmark of 1,000 puzzles, where every move of a puzzle has to be right. d1-omni-600M ran in fp32 on a Mac without search (42.1%), and in bfloat16 on a Colab A100 with search (60.2%: 3 plies ahead on the top 4 moves).*

```
LFM2.5-1.2B, prompted       1.5%
+ SFT v1 (24k rows)         6.7%
  + GRPO v1                13.0%
+ SFT v2 (1.3k rows)        3.7%
  + curriculum GRPO        15.4%
  + curriculum + process   13.4%
d1-3B, base                 0.5%
+ LoRA SFT                 16.2%
d1-omni-600M, base          1.9%
+ SFT                      42.1%
  + 3-ply search           60.2%
```

## part 1: Fine-tuning LFM-2.5-1.2b

Before training -> prompting. For that I built a custom harness with three tools: `get_current_board`, `get_legal_moves` and `make_move`. This allows flexibility for playing larger games. Models can choose when to see the board, see a list of all current legal moves and make the last move. It is adapted from [this benchmark](https://www.manoletre.com/paper/2512.01992).

*Figure: One move through the harness: look at the board, ask for the legal moves, reflect, then move.*

The model calls get_current_board, then get_legal_moves, writes a reflection, then calls make_move.

With prompting, the model solved only 15 puzzles. Most of the puzzles failed on the tool protocol.

I created a [dataset for SFT](https://huggingface.co/datasets/manoletre/chess-puzzles-tools-sft) with 24k full puzzle conversations, using deepseek-v4-flash with access to Stockfish traces as the teacher model. SFT taught LFM-2.5-1.2b to use the protocol correctly (0% -> 100% valid tool calls) and improved the performance of the model on chess a bit (1.5% -> 6.7%). But I had hoped that the 24k rows would have had a larger impact.

I then continued with GRPO (sampling 8 times per puzzle) with a simple reward: +1 for the right puzzle move, 0 for a wrong one, −0.5 for an illegal one. Performance on puzzles doubled to 13.0% and illegal moves almost disappeared.

*Figure: How each of the 1,000 puzzles ended.*

prompted: 1.5% solved, 52.9% wrong move, 45.6% illegal move or broken tool call. + SFT: 6.7% / 80.2% / 13.1%. + GRPO: 13.0% / 85.3% / 1.7%.

But two unwanted things started to happen. First the model stopped exploring: 8 samples of the same position gave on average 1.7 different moves.

*Figure: Eight tries at the same position. After GRPO the model keeps playing the same wrong move: Bxf7+ in 6 of 8 tries, where gxf7+ wins.*

Puzzle 5FJz1, 8 samples at temperature 1.0: the SFT model tried 7 different moves (Rh5, gxh7+, Rf5, an illegal Rg7, g7 twice, Ra5, Re5); the GRPO model played the wrong Bxf7+ 6 times and the right gxf7+ twice. Over all curriculum puzzles: SFT 5.7, GRPO 1.7 different moves of 8.

Second, training stalled around step 300 of 600. My hypothesis is that the easy puzzles the model could learn from ran out, and the hard ones only gave a sparse signal.

Before training further, I tested whether sampling at a higher temperature would get the GRPO model exploring again. At 1.4 it found the right move more often across its 8 tries, but the reflections that the model wrote before making a move made no sense. Here is an example:

*Figure: A reflection at temperature 1.4. Highlighted: words in other scripts; underlined: made-up words. The move at the end was still right.*

A reflection sampled at temperature 1.4: mostly word salad, with Russian, Korean and Japanese fragments, ending in the correct move d7d1.

Training at T=1.4 would have rewarded nonsense that happened to end in the right move, so I kept the temperature at 1.0.

*Figure: Raising the temperature made more reflections nonsense but found the right move more often. At the best temperature for moves, 87% of the reflections were nonsense, so the reflection wasn't what picked the move.*

By temperature, nonsense reflections / puzzles with a right move in 8 tries: 0.8 -> 1% / 51.8%, 1.0 -> 3% / 53.8%, 1.2 -> 18% / 55.0%, 1.4 -> 87% / 57.2%, 1.6 -> 100% / 55.9%.

This mainly showed me that the reflections never meant much: both nonsense and clean reflections picked the right moves.

Taking a closer look at the data I used for SFT, most rows named the final move in their first words. This likely taught the model to pick a move first and justify it after. I also realized that many of the rows I used for SFT contained chess inaccuracies.

I decided to start from scratch: I created a [smaller dataset](https://huggingface.co/datasets/manoletre/chess-puzzles-tools-sft-v2) (~1,300 rows, SFT v2) for SFT with claude-sonnet-4-6 and a loop that verified if the reflections had chess inaccuracies.

This smaller SFT run taught the model to use the tools in the harness. The fine-tuned model solved 3.7% of the puzzles against 6.7% for the first SFT. That was expected, I knew most of the learning would happen during RL.

I thought that the model was previously getting to hard positions too soon, and therefore it hadn't had the chance to learn chess basics before attempting the harder puzzles.

So I built a [curriculum](https://huggingface.co/datasets/manoletre/chess-puzzles-curriculum): I divided Lichess puzzles into four stages of difficulty (about 19k each), sampled every stage again with the current model, and the puzzles where none of the 8 tries got a result were tried in a subsequent stage.

*Figure: The curriculum: four stages from easy to hard. Puzzles where none of the 8 tries was right give no signal, so they move on to a later stage.*

Curriculum: 4 stages, easy to hard. Puzzles solved after each stage: 10.8%, 13.4%, 14.2%, 15.4%.

The model after this GRPO run reached 15.4% on my benchmark. But when I read the reflections, they had collapsed into a very specific template: The model again mentioned the final move in the first words of the reflection and had very repetitive wording.

*Figure: A reflection of the final curriculum model on benchmark puzzle 2nNFF. It names the move in its first words and calls it mate. After the curriculum, 95% of the reflections named the move in their first words (0% after SFT v2), and 93.5% contained “immediate checkmate” (3% after SFT v2).*

A reflection of the final curriculum model on puzzle 2nNFF, naming the move first and claiming mate. Across the benchmark: move named in the first 10% of the text in 95% of reflections (SFT v2: 0%); 'immediate checkmate' in 93.5% (SFT v2: 3%).

I think RL taught the model (again) to decide a move before reflecting and then write a chess sounding text.

I also did an experiment: I could review the correctness of the reflections by training a model to extract every move, square and claim in a reflection, and then check the correctness of these claims via python-chess.

I finetuned LFM2.5-Encoder-230M to do that ([how the encoder works](https://www.manoletre.com/chess/encoder#how-it-works), [see it in action](https://www.manoletre.com/chess/encoder.md)). Here is a pre-computed example:

*Figure: The encoder on the template reflection above: five facts in the reflection about the board are true but the mate is not.*

The encoder marks the move, four square facts (all true) and two mate claims, which python-chess shows are false: after Qh8+ Black can play Kxh8 or Kg6.

I thought I could use this encoder model to have a “less sparse” reward and do GRPO on chess puzzles. The GRPO run would penalize runs where the reflection didn't have the final move at the end and had chess mistakes.

This last experiment (the same SFT v2 start and curriculum) where I tried rewarding good chess reflections had mixed results.

*Figure: Rewarding true reflections made them truer, not better at chess.*

With the process reward: false facts 53% -> 24%, self-contradictions 39% -> 5%, puzzles 14.2% -> 13.4%.

*Figure: The GRPO runs with a curriculum on the chess benchmark as they trained.*

Benchmark solved by GRPO step. SFT v2 + curriculum: 3.7, 10.8, 13.4, 14.2, 15.4% at steps 0, 250, 500, 750, 1,000. SFT v2 + curriculum + process reward: 3.6, 10.1, 11.4, 13.4% at steps 0, 250, 500, 750.

False facts in reflections fell from 53% to 24% and self-contradictions from 39% to 5%, but puzzles stayed at 13.4%, against 14.2% for the curriculum alone after the same stage.

I stopped this last run after stage 3. The model wasn't learning much. After every stage I had to change the reward or retrain the encoder model, because of reward hacking: RL kept finding ways around the encoder model and my evaluation. Probably I could have continued this game with more patience and compute, but I had neither.

## part 2: a decision model

My goal was to play against a language model trained by me and actually lose. LFM-2.5-1.2b wasn't getting there. And then Liquid released d1-3B and d1-omni-600M. So I decided to pivot this project.

*Figure: The same puzzle, two ways to get a move.*

LFM-2.5-1.2b calls get_current_board and get_legal_moves, writes a reflection token by token and then calls make_move. d1-omni-600M scores every legal move in one forward pass.

From here on I trained on chess positions instead of conversations given that the decision models answer directly.

More specifically, I trained the decision model to answer two questions. First “Which move is best?” where the options are the legal moves on the specific position. And “How good is this position for the side to move?” where the options are 10 levels of win chance (0-10%, 10-20%, …, 90-100%). Stockfish's evaluation gives the right answer for both.

*Figure: The two questions that I fine-tuned d1-omni-600m on.*

d1-omni-600M's real answers next to the labels, on one position from the end of the Lichess evaluation database, a part training never read. The label comes from Stockfish's score in that database: win% = 50 + 50 * (2 / (1 + exp(-0.00368208 * cp)) - 1), and a forced mate counts as 100 or 0. Over 999 such positions, d1's expected win chance is off by 7.8 points on average, its tallest bar is within one level of the label on 90%, and its top move is Stockfish's best on 29%. The prompts: '<bos> <state> FEN: … / White (to move): Ke1, Qd1, … / Black: … <q> You are a strong chess player. White to move. Which move is best? (<opt> <mask> e2e4 </opt>)* <decide>' for the move question (one option per legal move, in random order), and the same state with '<q> How good is this position for White, who is to move? (<opt> <mask> level i: … </opt>)* <decide>' for the win-chance question (levels: lost (0-10% win chance), 10-20%, …, winning (90-100%)). In a game the state also lists the last 3 moves. d1 scores each option at its <mask> token. Question 1: which move is best? The options are the legal moves, and d1 plays the move it scores highest. Question 2: how good is this position for the side to move? The options are 10 win-chance levels, and the expected win chance is their average. In the search, question 1 picks which moves to try and question 2 scores the positions at the end.

I first ran SFT on d1-3B with [120k Stockfish-labelled positions](https://huggingface.co/datasets/manoletre/chess-positions-stockfish-120k). 6k came from Lichess puzzles and 114k from [Lichess games](https://database.lichess.org/#standard_games) (September 2026, three positions per game). This got d1-3B to 16.2% performance on the puzzle benchmark.

Given that the decision models use less compute (one forward pass per move instead of ~150 tokens), I thought that I could afford RL on whole games. d1-3B could play thousands of games against Stockfish and I could reward the wins. But this didn't work.

First playing games takes long. One iteration was 128 games against Stockfish, and then training on ~3k moves from those games took ~7.5 minutes on an A100. A game is ~47 moves of d1, but it ends in one result, so most of the time goes into playing for very little signal. In an early experiment with 28 iterations (~3.5 hours, ~4k games) blunders stayed at ~8.5% of moves, and the model got worse at puzzles (16.2% -> 12.9%).

*Figure: One game of RL ends in one result. The same moves labelled by Stockfish give each a usable signal.*

RL on a game: ~47 d1 moves, then one result (win, draw or loss). SFT: Stockfish labels every position, so 47 moves give 47 signals.

At the same time I was considering this RL run, I found the [Lichess evaluation database](https://database.lichess.org/#evals). It contains millions of positions that Stockfish already analysed at high depth. Suddenly I had a lot of already evaluated data that I could use directly for SFT.

With my limited compute and this much data, trying the smaller model made sense. d1-omni-600M trains about 5× faster than d1-3B, so I could train it for longer on more data.

I trained d1-omni-600M for 9 rounds on ~1.5M positions: ~1M from the evaluation database plus ~0.5M from new games Stockfish labelled on the VM's CPUs while the GPU trained. It got to 42.1% of puzzles without search, and it was still improving when I stopped. I could probably have continued training d1-omni-600M, but I didn't want to burn all of my Colab credits yet (more experiments are coming).

I then added a small search on top and it helped a lot: its score against Stockfish 1700 goes from 12.5% to 62.5%.

*Figure: Score against Stockfish, 40 games each from balanced positions at move 5, with one forward pass per move and with 4 plies of search on the top 3 moves. A win counts 1 and a draw ½. d1-omni-600m ran on a Colab A100 (PyTorch, bfloat16). sf-N is Stockfish limited at Elo N (0.1 s a move, one CPU thread), and sf-1320 is its Skill Level 0 (0.05 s a move).*

Score against Stockfish, one forward pass vs search: sf-1320 61% vs 88%; sf-1500 19% vs 79%; sf-1700 13% vs 63%; sf-2000 5% vs 35%.

The search chooses a move by using both questions from above. The move question picks which moves to try: d1's top 3 moves, then the top 3 replies to each (simulating the opponent), then d1's top 3 answers to those. That gives 27 positions at the end of the tree. The win-chance question scores each of them.

But d1 doesn't just pick the best of the 27, because the second move of every line is the simulated opponent's. So the scores go back up the tree: at each position, the side to move takes the move that is best for them. A first move is only as good as what is left after the opponent's best reply, and d1 plays the first move with the best overall result.

*Figure: The search on an example tree. The 27 end positions get a win chance for d1, and the scores go back up: at every move, the side to move picks its highest win chance. The opponent's win chance is 100 minus d1's.*

Search, example numbers. d1's best answer to each reply, as d1's win chance: first move A 91, 30, 70; B 62, 70, 64; C 45, 80, 55. The simulated opponent sees 100 minus those and picks its highest: 70 under A, 38 under B, 55 under C, which leaves d1 30, 62 and 45. d1 plays B (62), even though the single best end position (91) is under A.

So this is where it stands: a 600M decision model that solves 42.1% of my puzzles in one forward pass and, with a small search on top, scores 62.5% against Stockfish 1700. [Can you beat it?](https://www.manoletre.com/chess/play)

To fit the model in the browser I quantized it. The matrix weights go from 32 to 4 bits and the embeddings from 32 to 16 bits (fp16). This makes the model go from 1.5 GB to 339 MB. On the 1,000-puzzle-benchmark the quantized version picks the same move as the full model 91% of the time, and solves 40.5% of them (close to the 42.1% of the non-quantized version).

That was still too much for phones. iPhones closed the tab on the model's first search. So I further quantized the embeddings from 16 to 8 bits. The model file size goes from 339 MB to 272 MB and uses about a quarter less memory in the browser, and on the benchmark it plays the same as before: 40.5% of the puzzles solved, and the same move as the full model 91% of the time.

*Figure: The full model (fp32) and the two quantized ones, on the 1,000 benchmark puzzles (2,378 positions), one forward pass per move. The one with 8-bit embeddings runs in your browser.*

fp32 (1.5 GB) vs 4-bit (339 MB) vs 4-bit with 8-bit embeddings (272 MB, the one in the browser), one forward pass per move: puzzles solved 42.1% vs 40.5% vs 40.5%; first move right 56.7% vs 56.0% vs 56.0%; same move as fp32 in 91.4% and 91.2% of positions.

But this improvement was still not enough. The site kept crashing on my iPhone. So I let Claude Code run an optimization loop overnight. It did a change at a time and measured the result in a browser (Chrome for desktop, Safari's engine for the iPhone) on speed, memory and download size. This loop ran on the copy in the browser, which I had also fine-tuned to understand voice. A change was only kept if d1 still solved about the same 38.4% of puzzles in one pass and understood the same 80.8% of my recorded voice commands. 19 changes were kept and 19 dropped.

That voice part needs explaining, because this is where I decided to make the model listen to me. d1-omni-600M also takes audio as input (that's the “omni”), and until then I wasn't using that piece of the model at all. Before fine-tuning it got 0 of 25 spoken moves right. So I fine-tuned it on about 1,100 commands I recorded myself, plus generated speech and written commands. Now the browser copy gets 22 of those 25 right, and you can hold a button and say your move. It cost a little chess: this copy solves 38.4% of the puzzles instead of 40.5%.

*Figure: A real recording of me saying a command, from the held-out set that training never used, and what the browser copy does with it. One pass asks what I want, and a second picks the move. This clip is one it gets right: over all 52 held-out recordings it gets 80.8% fully right.*

A held-out recording of me saying 'knight e5', with White to move in r1bqk2r/pp3ppp/2nbpn2/2pp4/3P1B2/2P1PN2/PP1N1PPP/R2QKB1R. Pass 1, what do I want: move 86%, nothing 4%, highlight 4%, hint 3%, undo 3%. Pass 2, which of the 39 legal moves: Ne5 89%, Be5 0.4%, then the rest. The move is played. Over all 52 held-out recordings the browser copy gets 80.8% fully right, and 22 of the 25 moves.

The biggest wins were not in the model. d1's prompts only use 1,930 of its 65,536 tokens, so cutting the rest of the embedding table made the download 65 MB smaller and each position a quarter faster. On the iPhone the problem was onnxruntime itself: my hypothesis is that compiling its 27 MB WebAssembly file was what closed the tab. A build of the same engine for JSPI is 17 MB and uses about a third less memory, so phones now search 3 moves ahead too. In the search, looking deeper beat looking wider, and adding the move's own probability to its score found 1-3% more puzzle moves.

The last step only runs in desktop Chrome: most of the model's matrix multiplications in 16-bit floats instead of 32. Each position takes 59 ms instead of 72, and it solves one puzzle fewer (38.3% instead of 38.4%). That is the only accuracy the whole loop lost.

*Figure: The kept steps of the overnight loop. Pick a number to see it change; hover a step to see what it was. The y axis starts near the lowest value, not at 0. The accuracy lines stay flat; fp16 costs one puzzle. Phone numbers are Safari's engine on a Mac pretending to be an iPhone.*

Kept steps: 0 start, 1 vocabulary (1,930 of 65,536 token embeddings), 2 answer cache, 3 voice encoder loaded up front, 4 phones on onnxruntime 1.22 with a saved game, 5 onnxruntime's JSPI build, 6 phones search 3 moves ahead, 7 policy blend in the final pick, 8 fp16 in desktop Chrome. Start -> now: time per position 97 -> 59 ms; desktop memory while playing 3.39 -> 2.61 GB; download 309 -> 219 MB; first spoken command 1.4 -> 0.32 s; phone memory 2.67 -> 1.69 GB; puzzle moves found on the phone 64.9% -> 78.9%. Chess puzzles solved 38.4% at every step until fp16 (38.3%), voice commands right 80.8% at every step.

[play against the final model!](https://www.manoletre.com/chess/play)

When learning about building a curriculum for RL I stumbled on [this paper](https://arxiv.org/abs/2505.14970), for which I built [this video](https://github.com/manoletre/explainer-video/releases/download/v0.1.0/SECExplainer.mp4) with [my skill](https://www.manoletre.com/projects?open=explainer-video). Try the skill!

Right when I finished this write-up, Liquid released a new set of models (LFM3). So this chronicle is maybe not over. [Follow me on X](https://x.com/manoletre) to see what comes next.
