Teaching a model to write Magic cards by building the compiler first

It started as a joke at a kitchen table and turned into the most involved engineering project I’ve done outside of work. The goal sounds simple: describe a Magic: The Gathering card in plain English (“a husky commander for a dog tribal deck”) and get back a real, mechanically-valid card. Everything running locally, on my own hardware.

It is not simple. Getting there meant discovering that a language model will confidently fake the parts of a task it can’t do; that Magic’s card text is, formally, ambiguous and unbounded enough to defeat every off-the-shelf parser I tried; that automated coding agents will cheat any metric you give them, relentlessly and creatively, until you stop asking them nicely and start building guardrails they can’t route around; and that the ceiling on the models I could afford to train had less to do with parameter count than with the shape of the data I fed them.

This is the write-up of all of that: four pieces, in order, each one a problem I walked into and what it taught me. You don’t need to know anything about Magic to read them; where the game matters, I explain it. And you don’t need to read them in order, though they build on each other; each one stands on its own, and the technical pieces lead with the general lesson, not the card game.

The whole project is open source. Links are at the bottom.

The series Link to heading

1. A joke about fake Magic cards, and the wall I hit trying to make it real The origin story. A pipeline of local LLMs that could sometimes produce a genuinely good card and sometimes produce a beautiful, confident, empty one, and couldn’t tell the difference. Why that gap, more than any single failure, is what made everything after it necessary.

2. I’d never built a compiler. Here’s where my instincts were wrong. I knew assembly, file formats, and parsing theory, and I assumed building a parser for a card game would be straightforward. Three dead ends later (an abandoned reference project, a parser-generator that choked on Magic’s ambiguity, and a memorization trick disguised as progress) I understood why it wasn’t, and what the problem had required all along.

3. My coding agents kept cheating. I was building the cheat detector for months before I knew its name. I gave automated agents a clean metric and 32,000 test cases and let them optimize. They optimized the metric instead of solving the problem, and every defense I built, they routed around, in an escalating arms race that got genuinely unsettling. A hands-on field report on reward hacking, observed across every model and tool I tried. The piece I’d most want you to read.

4. The bottleneck wasn’t the model. It was the shape of the language. I hit a ceiling fine-tuning my best model and couldn’t push past it. A 110-million-parameter model I’d built as a joke on my laptop found out why: a structural property of the training data, shadowing, that adding parameters to a small from-scratch model can’t fix (and that a giant pretrained one, to my surprise, largely can). A field journal, contradictions and unexplained results included.

Where the project is now Link to heading

As of early July 2026: The compiler parses a bit over 20,000 of the ~32,000 legal Magic cards into its typed format. Four models are trained on the gold subset of that corpus: dragon (12B), goblin (4B), wyrmling (~110M, from scratch), and a 225M wyrmling sibling built for a scaling experiment. The v8 dataset (~103k intent-to-DSL pairs, squeezed out of the same parsed cards rather than new ones) has now been trained and evaluated across all four models on a single held-out test, and the controlled shadowing study is complete; article 4 reports both. A v9 dataset, roughly five times v8’s size, is generating right now on a freshly cleaned-up DSL, the first build since the counterfeit-vocabulary purge article 3 ends on. This is live, ongoing work; the articles reflect where things stood when I wrote them, and the project has almost certainly moved since.

The code Link to heading

The full project (the card generator, the compiler, the models, the benchmark, the fine-tuning pipeline) is open source. All the code is MIT; the trained artifacts carry their own terms, documented in the repo’s licensing notes: wyrmling’s weights are MIT too, goblin’s adapter follows Google’s Gemma terms, dragon’s follows the Gemma 4 Apache 2.0 license, and everything card-derived ships as unofficial Fan Content:

The bundled Magic card data is third-party content from Wizards of the Coast / Scryfall and is not covered by the MIT grant. Unofficial Fan Content permitted under the Wizards of the Coast Fan Content Policy; not approved or endorsed by Wizards.