I trained a small model on ten short poems until it wrote like the person who wrote them, and documented every step. The first six parts are how a language model actually works, for a reader who is new to all of it. The last four are the project: the data, the run, and an honest look at whether it beat a well-written prompt.
1 September 2026·10 parts·104 min in total·aihomelabtraining
Every part is written to be read in order, and the first six have no commands in them at all. If you already know how a transformer works, start at part seven. If you write software and remember your linear algebra, skip part two.
The corpus is ten short poems by an invented writer, and you can download the whole kit: the ten poems, the two JSONL files, and the files that take you from an empty directory to a trained adapter: build_corpus.py, train.py and generate.py, a requirements file, and the Axolotl config if you would rather drive the run from YAML. There is no repository yet. There will be one eventually, and until then each file is fetched on its own from that folder. Everything here is reproducible against the same bytes I used.
The training run needs one GPU. Mine is an RTX 3090 and the run uses about 11 GB, but part nine carries a second track on a smaller model that fits in roughly 6 GB, which is a laptop GPU or a free hosted notebook. You do not need the machine I have.
The whole machine end to end, in one figure: text becomes tokens, tokens become vectors, a stack of identical blocks transforms them, and the top produces a probability for every possible next word. Start here if any of that is new. Everything after it is one stage of this part, enlarged.
A short refresher: how to read a shape like [5, 4096], one worked example small enough to check by hand, and how to count a model's parameters yourself. Not a mathematics course and it says so. Skip it if you write software and remember your linear algebra.
The stage nobody thinks about, which explains more of the famous stupid failures than anything else in the architecture. Where the vocabulary came from, why rare words shatter into pieces, and the special tokens that a fine-tune gets wrong at its peril.
The only place in the entire architecture where one word can affect another, and therefore the mechanism behind everything context does. The part worth slowing down for. Also why long prompts cost what they do, and what that implies for how you write one.
What surrounds attention: the other half of a block, and two pieces of plumbing that look like footnotes and are the reason a model this deep can be trained at all. Then the cache that actually fills your graphics card, and why a model that loaded fine dies an hour later.
The top of the stack, where a vector becomes an actual word. Softmax, the sampling settings that live outside the model and change its character more than people expect, and how generation knows to stop, which is also one of the commonest ways a fine-tune breaks.
It is real training and it moves the weights permanently. What it moves reliably is form. What it moves badly is fact, and getting those backwards produces a system that is fluent, confident and wrong. Also the two free things to try first, one of which may be enough.
The part that decides the outcome, and the one most likely to be skipped because none of it looks like machine learning. Turning ten poems into a training file, why two are held back, and the hardest question here: what to write on the prompt side when your source material is answers with the questions missing.
Every command, from a bare virtual machine to a saved adapter, on a single GPU. The training script annotated setting by setting, what the loss curve is and is not telling you, and the four things that went wrong when I did it. A second track runs on about 6 GB if you do not have a big card.
Merging the adapter, serving it, and testing it against two poems it was never shown, compared against both the untouched model and a carefully prompted one. That last comparison is the one most write-ups quietly skip because it can make the whole exercise look unnecessary.