Engine Analysis vs AI Explanation: Key Differences

A chess engine calculates. Given a position, Stockfish searches the tree of possible moves with an alpha-beta search algorithm and scores the resulting positions with an evaluation function, today an efficiently updatable neural network known as NNUE [2]. The output is a number, the evaluation in centipawns (hundredths of a pawn), and a principal variation: the sequence of best moves it found. Nothing in this pipeline involves language. The engine does not know that the position came from a nine-year-old's Sunday rapid game; it knows material, structure, and the millions of positions it examined.

Two properties make engine output trustworthy in a way nothing else in this article is. It is reproducible: the same position at the same depth produces the same evaluation, so a claim like 'this move dropped half a pawn' can be re-derived by anyone. And it is essentially free: Stockfish is open source and costs nothing to run [1], which is why it sits underneath Lichess analysis, most coaching tools, and ChessCore's own game reviews. When an engine is wrong about chess, it is usually in positions that are exotic for club play, like deep fortress draws; for the games an academy reviews, the engine is the closest thing chess has to ground truth.

The engine's weakness is the flip side of its strength: it is unhelpfully precise. A parent reading '+1.7 after 18...Qe7' has learned nothing. The engine answers what is true, never why it matters or what the student should practice. That gap is the entire reason the second machine exists.

A language model predicts text. Given everything written so far, it produces the most plausible next words, and it does this so well that the output reads like understanding. Handed a structured summary of a game (the evaluations, the move where the game turned, the coach's note), it can write 'Aarav held a worse endgame for thirty moves and only let it slip in time trouble', a sentence a parent actually absorbs. That translation, from numbers to meaning, is the model's real job in coaching software.

The model's failure mode is the mirror image of the engine's. The engine is rarely wrong but always unreadable; the language model is always readable and sometimes wrong, and wrong in the worst way: fluently and confidently. There is no error message, no falling eval bar, no signal that distinguishes a correct sentence from an invented one. This is why frameworks for deploying AI responsibly, like the NIST AI Risk Management Framework, treat the validity of generated output as a risk to be managed rather than assumed [4], and why a human still approves anything a model drafts for a parent.

Most confusion about AI in chess coaching dissolves once the two systems sit in one table. They differ on every axis that matters: what they are, what they emit, how they fail, what they cost, and what you should trust them for.

Read the last two rows twice, because they are the entire argument. Each machine has a domain where it is the best tool available and a domain where using it is malpractice. Every failure we have seen in AI chess products comes from sliding one machine into the other's column.

A language model cannot reliably evaluate positions because it has no search and no guaranteed board state. Evaluation in chess is not a fact you remember; it is a computation you perform, and the model performs a different computation entirely: predicting plausible text. Those two limitations are worth unpacking separately, because together they explain every strange chess claim a chatbot has ever made.

No search means no calculation. When Stockfish says a position is winning, that verdict rests on examining an enormous tree of concrete continuations; the number is a summary of work actually done [2]. When a language model says a position is winning, no tree was searched. The model produces the kind of sentence that tends to follow this kind of prompt. For famous positions that appear in its training data, that imitation can look like analysis. For your student's game from last Tuesday, a position no one has ever written about, there is nothing to imitate, and the model's verdict is a well-dressed guess.

No board state guarantee means the model can lose track of the position itself. A chess engine holds the position in an exact internal representation; it is structurally incapable of believing a knight is on a square it is not on. A language model holds the position as text, a FEN string or a list of moves in Portable Game Notation [5], and as a conversation gets longer it can drift: referencing captured pieces as if alive, moving a bishop through a pawn, asserting a check that does not exist. The errors are not rare curiosities; they are the predictable result of asking a text predictor to be a board.

The conclusion is not that language models are useless in chess software; it is that their output must never be load-bearing for chess truth. Any product where the model is the source of an evaluation, a best move, or a tactical claim is built on the wrong machine. Any product where the model only rephrases what an engine and a database already established has confined the model to the one task it is genuinely superb at. The difference is invisible in a demo and decisive over a season.

The correct architecture runs the two machines in series, in one fixed direction. Stockfish analyzes the student's synced games and decides what is true: where the evaluation swung, which move was the mistake, what the better line was. Those verdicts, plus database facts like attendance and rating history, become the structured input to the language model, whose only job is to phrase them for a human audience: a parent, a 10-year-old, a coach skimming before class. The model writes 'the game turned when the queens came off into a lost pawn endgame', and the eval graph that justifies the sentence is one click away.

The direction must never reverse. The language model is never asked to evaluate a position, suggest a move, or fill a gap in the engine's output; if the data is missing, the draft says so instead of improvising. Numbers like ratings and attendance are injected from records rather than generated, and rating claims stay verifiable against the platform's public history through the same APIs the games are synced over [3]. The final guardrail is human: every drafted explanation waits for a named coach to approve it, the approval queue pattern we covered in its own article.

People get burned the moment they let the fluent machine do the calculating machine's job. The pattern shows up in three places: coaches pasting positions into general chatbots, products quietly using a model where an engine belongs, and buyers grading software on how impressive the prose sounds.

• The chatbot game review: a coach pastes moves into a general chatbot and gets a confident, plausible review that misidentifies the losing move. The student studies the wrong lesson, and nobody notices because the prose was excellent.
• The invisible substitution: a product shows polished explanations but cannot show the engine line behind each claim. If the vendor cannot point at the Stockfish output, the model is doing the evaluating.
• The demo trap: buyers compare tools on the eloquence of the explanation, which measures the translator, not the truth. The right comparison is whether every claim traces to engine analysis of real games.
• The inverted pipeline: software that asks the model to pick the critical moment and then runs the engine to decorate it has the dependency backwards; the engine must select, the model may only describe.

None of these failures announce themselves. That is the common thread, and it is why the architecture question belongs in your buying process rather than your incident reviews. Our piece on AI guardrails in coaching software turns this into a vendor checklist.

Choose software where the two machines are visibly separated and correctly ordered. In practice that reduces to a handful of questions you can ask on any sales call: Which engine produces your evaluations, and can I see its output next to every explanation? Is the language model ever the source of a chess claim, a number, or a recommendation? What happens when data is missing? Does anything the model writes reach a parent without a named human approving it? A vendor with the right architecture answers in seconds and shows you the screen; a vendor with the wrong one talks about how advanced their AI is.

For ChessCore the answers are: Stockfish produces every evaluation, games arrive through Lichess sync and read-only Chess.com import [3], the model only phrases engine findings and locked database records, missing data is flagged rather than papered over, and every draft waits for coach approval. Our AI page documents the pipeline and the integrations page covers the syncing underneath it. Whichever tool you pick, the test stands: the engine decides what is true, the language model decides how to say it, and the day a product gets that backwards is the day its explanations stop being worth reading.