chess / commentary encoder
A fine-tuned LFM2.5-Encoder-230M reads chess commentary and tags every move, piece, square and claim; chess.js then checks each claim against the board. I built it to reward true reasoning during RL (the story, and how it works below). It runs in your browser: the first load downloads the model (~145 MB) from Hugging Face. Code and data are on GitHub. Pick an example or write your own.
how it works
When fine-tuning LFM-2.5-1.2b to play chess I needed a way to verify the correctness of the reflections it was generating, so that I can positively reward the “good” reflections. To do so I needed to parse chess data out of a text: which words are moves, which are pieces and squares, and which are claims like “gives check” or “wins the queen”. That's token classification, which a small encoder does well and fast.
LFM2.5-Encoder-230M turns every token into a vector of 1,024 numbers. On top of it I trained a single linear layer with 36 outputs, split into five heads. Each head is its own classifier over the same tokens with its own softmax and cross-entropy loss.
| head | labels | outputs |
|---|---|---|
| mention | move, piece, square, file, rank, diagonal, region | 15 |
| line | new, next, alternative | 3 |
| stance | chosen, rejected, considered | 3 |
| frame | now, after | 2 |
| claim | check, mate, wins, attacks, flight, eval | 13 |
With these classifications, we can turn the labels into questions that are programmatically verifiable using python-chess or chess.js. For example “rook can swing to g8” becomes verifiable. If the rook is in a position where it can't move to g8 we can't do that.
The encoder doesn't say which rook. It only marks “rook can swing to g8” as a move, and the checker keeps the legal moves that fit what the text says: a rook, landing on g8. If exactly one move fits, that's the move (here the rook on g2 going to g8); if none fits, it's false; if several fit, it can't tell and leaves it unverifiable. A piece with a square (“the rook on g2”) is looked up on that square, and one without is found only if it's the only piece of its kind on the board.
Then rules turn tags into facts. Each claim is linked to the nearest move or piece before it in its clause, and checked on the board as it is now or after that move, depending on its frame: python-chess during training, a chess.js port of the same rules here in the browser (both give the same verdict on all 34,621 facts I compared). A fact is true, false or unverifiable. Evaluations like “the attack fizzles” can only be checked when the solution is known. The checker also reports the chosen move, moves the text rejected and then played, and false refutations: reasons against a move that aren't true.
The rook can swing to g8 and deliver check, but Black's king simply steps away.
- ✓ “rook can swing to g8” (a move, rejected): White's only rook that can reach g8 is on g2, so it means Rg8
- ✓ “deliver check” (check, after the move): after Rg8 the black king on f8 is in check
- ✗ “simply steps away” (the king escapes, after the move): after Rg8 the king has no legal move: the rook covers e8 and g7, the bishop covers f7 and protects the rook on g8, and e7 holds its own bishop
false refutation: the text rejected Rg8 for a reason that isn't true. Rg8 is mate, the puzzle's solution.
Here is the whole reflection that sentence comes from, one head at a time.
White's rook on g2 is aimed at the g8 square, and the black rook on f1 is too far to interfere. The rook can swing to g8 and deliver check, but Black's king simply steps away and the attack fizzles. The bishop on b3 already controls g8, so the rook can land there and deliver immediate checkmate. So the move is Rg8#.
data
704 training texts are real reflections from the LLMs I trained to play chess, from different points of RL, so the encoder sees how their wording drifted. Claude Opus 5.5 (through my Claude Code subscription) labelled them, and they are split by puzzle so no test puzzle shows up in training. The rest are synthetic chess-related text that is LLM-written.
The model is on Hugging Face[1], and the code, the labelling guide and the data are on GitHub[2].