Four models, each larger than the last, none of them inheriting a single parameter from anywhere else. The thesis does not change with scale: put the knowledge in the context, not in the weights — then the model's job is to read carefully and to stop at the edge of what it was given.
Two are on Hugging Face and can be downloaded and run on a CPU today. One is kept private. One is still training.
The first one that worked. It knows what it is, holds a short conversation, and — the useful part — reads a supplied <|context|> and copies the right value out of it, including out of noisy search results full of distractors, and says so when the answer is not in the passage.
The same design 2.5× over, sharing a tokenizer with Sane-47M. Six hours on a TPU v5e-8 on a WSD schedule — and it clears GPT-2 small on two of the three tasks it was measured against, on a fraction of GPT-2's budget. The third it loses, which is also on the card.
| Zero-shot · 200 examples | Sane-118M | Random | GPT-2 small |
|---|---|---|---|
| HellaSwag · acc_norm | 36.0 | 25.0 | ~31 |
| ARC-Easy · acc | 47.5 | 25.0 | ~44 |
| LAMBADA · acc | 22.0 | ~0 | ~33 |
The step between the small models and Synth-2, and the reason there is one: it is the check that the recipe survives past a hundred million parameters. Trained, kept private, not released.
The current work, and the first one carrying the idea in its architecture rather than in its training data. A small head on every layer predicts the model's own error for each token, so an answer can show which span to check rather than asking to be trusted whole. It reads a search results page and answers only from it.
A parallel project, not a model: a desktop agent that does the work on your own machine, with your own model keys — editing a project and running commands in it, answering from your own folders, reachable from a phone.
Every figure on these pages comes from a measurement that can be repeated. If one looks wrong, saying so is worth more than a compliment.