Fine-tuning is real training. It runs the same optimizer as the original, it moves the weights permanently, and it does not move them back. What it moves reliably is form: voice, shape, habit, the way a thing is said. What it moves badly is knowledge, and confusing those two is the most expensive mistake available to a beginner.
Part 07 of 10·13 min read
Let me concede the important thing before I spend the rest of this piece on limits, because the limits get quoted and the concession does not.
Fine-tuning is training. Not a lesser thing that borrows the word. It computes a loss, it propagates gradients back through the network, and it updates parameters with an optimizer, which is the same sequence of operations that produced the model in the first place. The weights that come out are different from the weights that went in, permanently, and the difference is a real artifact you can save, ship and serve. Anybody who tells you it is merely prompting with extra steps is wrong about the mechanism.
What is smaller is the scale, not the nature. Pretraining is trillions of tokens across thousands of accelerators for weeks. What I am about to do is a few thousand tokens on one graphics card for under an hour. Same machinery, four or five orders of magnitude apart, and that gap is exactly why one is available to me and the other is not.
The argument in this piece is about what that small amount of training reliably changes. It changes how the model writes. It is a poor and unreliable way to change what the model knows. Those are different axes, they get conflated constantly, and getting them separated in your head before you start is worth more than any hyperparameter you will pick later.
Part one ended with eight billion numbers and the claim that training is nothing more than moving them. Here is the loop that does the moving, because every setting you will touch in part nine is a knob on one of these four steps.
Take one training example: a prompt and the response you want. Run it through the model exactly as if you were generating, but instead of sampling a token, look at the probability the model assigned to the token that actually comes next in your example. If it assigned a high probability, good. If it assigned a low one, that is error. Sum that error across every token in the example and you have the loss, a single number saying how surprised the model was by your text.
Now the part that feels like magic and is not. Because every operation in the network is differentiable arithmetic, you can compute, for every single parameter, how much a small change to that parameter would have changed the loss. That is the gradient, computed by working backwards from the loss through every layer, which is why it is called backpropagation. It is expensive, it is mechanical, and it produces one number per parameter.
Then you nudge each parameter a small step in the direction that would have reduced the loss. How big a step is the learning rate, and it is the setting people get wrong most often. Too large and the model lurches past every good configuration and the loss goes to nonsense. Too small and nothing happens in the time you are willing to wait.
Repeat for the next example, and the next, and then go back to the first one and do it all again. Each complete pass over your data is an epoch. That is the whole algorithm. Everything else in a training configuration file is a refinement of those four steps.
The loss is the only thing the run optimizes. Not quality, not style, not truth. The model is being pushed toward assigning high probability to your exact text, token by token. Every good outcome and every pathology in this series is downstream of that one objective being narrower than what you actually want.
Training uses several times the memory of generating. Because gradients have to be stored for every trainable parameter, the optimizer keeps running statistics on top of that, and the intermediate values from the forward pass have to be kept around for the backward pass to use. This is the arithmetic that decides whether a run fits on your card, and it is worked out properly two sections down.
Nothing about the run is aware of your goal. You want a model that writes in a particular voice. The optimizer wants to reduce a number. Those coincide often enough to be useful and they are not the same thing, which is why evaluation cannot be skipped and why part ten exists.
Here is what a small fine-tune is genuinely excellent at, and it is worth being enthusiastic about because the rest of this piece is a warning.
Form transfers. If every example in your data has the same shape, the model picks that shape up quickly and holds it consistently. Rhyming couplets, four beats to a line, a stanza break every two lines, a technical register with no exclamation marks. Doc Quatrain's surface is a set of properties that recur in every single example, and recurring properties are precisely what gradient descent is built to find.
The reason this works so much better than knowledge is a counting argument. Every token of every example is evidence about the form, because every token is inside the form. A three-hundred-word poem is four hundred separate pieces of evidence that this writer rhymes. Whereas a fact stated once in one poem appears in a handful of tokens in one example, and gets averaged against everything else.
That asymmetry is the whole of the matter. Fine-tuning is a statistical process, it responds to what is consistent across your data, and form is the most consistent thing there is.
Voice and register move first and move furthest. Sentence length, vocabulary level, how formal it is, whether it uses contractions, how it opens and closes. These show up within the first few passes over a small dataset and they are the reason style transfer is the canonical hobbyist fine-tune.
Output format moves almost as reliably. If every response is valid JSON with the same keys, or always three bullet points, or never starts with a preamble, the model learns that. This is the most common commercial use of fine-tuning and it is not glamorous: teaching a model to shut up and return the schema.
Structural habit moves too, and it surprises people. Doc Quatrain always opens by naming his own mistake and always closes on what will bite you next. That is not surface, it is the skeleton of the argument, and it recurs in all eight training examples the same way the rhyme does. Whether it actually transfers is the most interesting open question in this project, and part ten answers it.
What it will not do is give the model something to say. The counterweight, and I would put it first in any room where people are getting excited. A model that has absorbed a voice still generates its content the same way it always did. If the underlying model is weak on your subject, you will get weak content in an excellent voice, which is worse than weak content in a plain one because it reads as authoritative.
Fine-tuning changes how a model says things. Retrieval changes what it knows. Reaching for the first to fix the second is the most expensive wrong turn available to somebody starting out.
The failure I see most often, by a wide margin, is somebody with a pile of internal documents who wants a model that knows what is in them, and who reaches for fine-tuning because that is the word they have heard.
It mostly does not work, and the way it fails is worse than not working. Feed a model your company handbook as training data and you do not get a model that can answer questions about the handbook. You get a model that has become slightly more likely to produce text in the style of a handbook, and that will confidently invent a policy that sounds exactly like your policies and is not one. You have taught it the register of your documents without reliably installing their contents.
Three reasons, and they compound. A fact stated once is a handful of tokens against a corpus of millions, so the gradient signal is tiny. The model has no mechanism for storing a fact as a fact, only for adjusting probabilities, so a half-learned fact is not absent, it is fuzzy. And the same training that pushed it toward your documents pushed it away from saying it does not know, because none of your examples were an admission of ignorance.
The tool for knowledge is retrieval. Keep the documents in a searchable store, find the relevant passages at query time, and put them in the context window with the question. The model reads them the way it reads anything else in its context, which is the thing it is genuinely reliable at. Facts stay editable, they stay attributable, and the model can cite what it actually saw.
In practice most real systems want both and use each for its own job. Retrieval supplies what to say. Fine-tuning supplies how to say it. Doc Quatrain is a pure form problem with no knowledge component at all, which is one of the reasons he makes such a clean demonstration, and I want to be explicit that the clean case is not the common one.
Before any of this, two things cost nothing and might be sufficient, and skipping them is how people end up with a fine-tuned model that performs worse than a paragraph of instructions would have.
The first is simply asking. Modern instruct models follow style directions well. "Write in rhyming couplets, four beats to a line, open by naming your own mistake, concede a point against yourself past the midpoint, and close on what will go wrong next" is a system prompt, it takes a minute to write, and it will get you a surprising distance. If it gets you all the way, you are finished and you have saved an evening.
The second is showing rather than telling. Put two or three complete examples of the target voice directly in the prompt and ask for another in the same style. This is few-shot prompting, it is often dramatically better than description, and for a voice as distinctive as this one it works well enough that the honest question becomes why fine-tune at all.
The examples cost you context on every single request. Three Doc Quatrain poems in the prompt is roughly twelve hundred tokens that you pay for on every generation, forever. A fine-tune folds that into the weights and the prompt goes back to one line. At scale this is the entire economic argument, and at my scale it is a convenience rather than a saving.
Prompted style drifts over a long output, and trained style does not. Instructions sit at the beginning of the context and their influence fades as generation runs on. A model asked to rhyme will rhyme for twelve lines and start producing near-rhymes by line forty. A model trained to rhyme has no beginning of the context to drift away from, and this difference is sharper than I expected before I measured it.
Consistency is where training genuinely wins. Prompting gets you the voice most of the time. Training gets you the voice essentially every time, including on inputs unlike anything you anticipated, because the behavior is no longer conditional on instructions that a long or unusual request can crowd out.
Try the prompt first anyway, and keep it as the baseline. Not only because it might be enough. Part ten compares the fine-tuned model against a well-prompted version of the same base model, and without having done the prompting work honestly there is nothing to compare against. A fine-tune that cannot beat a good system prompt is a result, and it is a result a lot of writeups quietly decline to check for.
Say you have decided to train. The obvious approach is to update all eight billion parameters, and it is worth working out what that costs, because the number is what created the entire field of cheaper alternatives.
Each parameter needs its value stored, at two bytes. Its gradient, another two. And the standard optimizer keeps two running statistics per parameter in full precision plus a high-precision copy of the parameter itself, which is another twelve. Call it sixteen bytes per parameter, all resident at once, which for an 8B model is about 130 GB before you account for the intermediate values of the forward pass.
My machine has four cards with 24 GB each. A full fine-tune of even a small model does not fit on any one of them and does not comfortably fit across all four, and this is a model widely described as small. That wall is where almost everybody arrives, and the way around it is to stop updating every parameter.
The trick is called low-rank adaptation, and the observation behind it is that the update a fine-tune wants to make is simpler than the thing it is updating. Rather than changing a large weight table directly, freeze it and learn two much smaller tables beside it whose product has the same shape. Add that product to the frozen table when the model runs. If the small tables have an inner dimension of sixteen, you might train ten or twenty million parameters instead of eight billion, which is a fraction of a percent.
The frozen weights need no gradients and no optimizer state, so the memory collapses. The base model sits there as a fixed sixteen gigabytes and only the tiny adapters carry the training overhead. Now it fits on one card with room to spare.
The adapter is a separate file, and that is a practical gift. A few tens of megabytes against sixteen gigabytes of base model. You can keep a dozen of them, swap between them, throw one away and retrain in an hour, or merge one permanently into the base weights when you are happy. Part nine keeps them separate and part ten merges.
The rank is the capacity dial. The inner dimension of those two small tables. Higher means more capacity to absorb the training data and more risk of memorizing it. For a pure style transfer on a small corpus, low is correct and the instinct to turn it up is usually wrong.
QLoRA quantizes the frozen base and is what I actually run. Since the base weights are never updated, storing them at four bits instead of sixteen costs very little, and the sixteen gigabytes becomes under five. The adapters stay at full precision because they are the part being trained. This is what makes an 8B fine-tune comfortable on a single consumer card, and it is the configuration in part nine.
Adapters can behave differently once merged. Worth knowing before it puzzles you. A model serving an adapter at inference time and the same model with that adapter merged into its weights and requantized are not guaranteed to be identical, and the requantization is usually the culprit. If the merged model seems slightly worse than the one you tested, that is where to look first.
Two ways this goes wrong, and with a corpus of eight poems both are live risks rather than theoretical ones.
The first is overfitting, which at this scale is memorization. Pass over eight examples enough times and the model stops learning the properties they share and starts learning the examples themselves. The loss on your training data keeps falling, which looks like success, while the model quietly loses the ability to do anything except reproduce what it has seen. Ask it for a poem about a topic in the training set and you get that poem back, lightly rearranged. That is not a model that has learned a voice, and telling the two apart from the inside is impossible, which is precisely why two of the ten pieces are held back.
The second is forgetting. Every gradient step that pushes the model toward your data pushes it away from something else, and the model has no way to distinguish the general competence you want to keep from the behavior you are trying to change. Train hard enough on rhyming poems and you get a model that rhymes when you ask it to summarize a document. It has not been improved and then narrowed. It has been dragged, and the parts it was dragged away from are not marked.
Both are governed by the same controls and both are the reason a small fine-tune should be gentler than instinct suggests. A low learning rate, few passes over the data, a modest adapter rank, and a habit of saving the model at several points during the run so you can compare rather than guess. The version that scores best on your training loss is very often not the version you want.
Fine-tune when the behavior has to be consistent rather than usually right. A prompt gets you the voice most of the time. If most of the time is acceptable, stop. If the whole value is that it never breaks character, that consistency is the thing training buys and prompting does not.
Fine-tune when the instructions have grown longer than the work. A system prompt that has crept to eight hundred tokens of style rules is a fine-tune waiting to happen. You are paying that on every request and it is still drifting on long outputs.
Do not fine-tune to install facts. Covered above and worth repeating because it is the mistake with the worst failure mode. Use retrieval. If the facts change, or need attribution, or need to be removable, retrieval is not merely better, it is the only one of the two that can do the job at all.
Do not fine-tune to fix a model that is too small for the task. Training will not add capability that pretraining did not put there. If an 8B model cannot reason about your problem, an 8B model trained on four hundred examples of your problem still cannot. Reach for a larger base before reaching for a longer run.
Fine-tune when you have the data already and it is honestly consistent. The precondition everybody skips. The value is entirely in the corpus, and an inconsistent corpus teaches inconsistency very efficiently. If assembling the examples feels like the boring part, that feeling is the mistake, and the next part of this series is about nothing else.
The version of this argument I distrust is the one that says fine-tuning is overrated. It is not overrated, it is misaimed. Pointed at form it is cheap, fast and startlingly effective, and I would reach for it without hesitation. Pointed at knowledge it produces a confident, fluent, subtly wrong system, and the fluency is what makes it dangerous rather than merely disappointing.
So the question to settle before you start is which of the two you actually have. Is the model failing because it does not know something, or because it does not sound right? The first has a different answer and this is not it.
For Doc Quatrain the answer is unambiguous. There is no knowledge to install. There is a voice, a set of habits, and a structural skeleton, and every one of those is form. That makes this a best case rather than a typical one, and I would rather say so now than let a demonstration that goes well imply more than it should.
Next is the corpus itself: ten poems, why two of them never go near the training run, how to turn plain prose files into the format a trainer will accept, and the question that turns out to be the hardest one in the whole project, which is what the prompt on the other side of each example should say.