Part one used notation like [5, 4096] and told you it was worth learning to read, without ever giving you the rule. This part is that rule, and one worked example small enough to check by hand. It is not a mathematics course and it does not try to be.
Part 02 of 10·7 min read
Let me be clear about what this is, because a part like this can go badly in a predictable way.
This is not a course on linear algebra or calculus, and I am not qualified to write one. It is the small amount of arithmetic that is actually happening inside the machine described in part one, at the level where knowing it changes what you do. Everything in this part gets used later. Nothing is here for completeness.
Specifically left out: transposes, broadcasting, batch dimensions, determinants, inverses, eigenvectors, and every derivative. Some of those are genuinely important to people building these systems. None of them changes a decision anywhere in the remaining eight parts, which was the test I applied.
If you have written a nested loop you already understand the operation and can skim to the parameter counting. If linear algebra is a distant memory from a lecture hall, this is a twenty-minute refresher compressed into six, aimed only at the parts that recur.
Start with the notation, because it is what error messages are made of and reading one correctly saves a great deal of guessing.
A shape written [5, 4096] means a grid with 5 rows and 4,096 columns. In part one that was five tokens, each holding 4,096 numbers. Rows first, columns second, always. Numbers get read out of it by row and column, so row 2 column 900 is one specific value.
Two of those grids can be multiplied together, and there is exactly one rule about when.
Worth noting that order matters. Multiplying A by B and multiplying B by A are different operations with different results, and often one of them is not legal at all because the inner dimensions only line up one way. This is unlike multiplying ordinary numbers and it is a reliable source of confusion.
Now the operation itself, on a pair small enough to verify by hand. It is worth actually checking one cell rather than reading past it, because that is the moment the notation turns into something you can reason with.
That is the entire operation. Somebody who has written a triple-nested loop has implemented it: iterate over output rows, over output columns, and over the shared inner dimension, accumulating a sum. Twelve lines, no cleverness.
What matters is not the loop but a property of it. Every output cell is computed independently. Nothing in cell one is needed for cell two, no cell depends on any other, and there are no branches anywhere in the computation. That property is why this arithmetic ended up on graphics hardware, and it is the section after next.
Here is the payoff, and it is short. A layer of a neural network is one matrix multiplication.
Take your input, multiply it by a grid of learned numbers, and usually add another small learned grid on top. In notation that is roughly output = input x W + b, where W and b are the learned parts. That is a layer. It is the whole thing.
So when part one said that a model is a stack of blocks containing tables of numbers, the tables are W, and running the model is a long sequence of these multiplications with a nonlinear function sprinkled between some of them. The feed-forward layer in part five is two of these back to back. The query, key and value in part four are three of them applied to the same input. The output head in part six is one of them against a very wide grid.
This is why part one could say that training moves numbers and nothing else. The numbers being moved are the entries of W, everywhere, and the architecture is the fixed pattern of which multiplications happen in what order.
One vector times a grid is the common case. During generation the input is often a single row, so the shape is [1, 4096] x [4096, 512], giving [1, 512]. Reading that left to right: take 4,096 numbers in, produce 512 numbers out, using a grid of learned values that connects every input to every output. That connect-everything-to-everything property is what "dense" or "fully connected" means.
Many rows at once is the same operation, and it is nearly free. Processing 500 tokens is [500, 4096] x [4096, 512]. Same grid of weights, 500 rows instead of one. This is why prefill in part five costs so little per token compared to decode: the expensive part was reading W out of memory, and it gets read once regardless of how many rows go through it.
The nonlinear function between layers is not optional. Part five makes this point and it follows directly from the arithmetic here. Multiply by W1 and then by W2, with nothing between, and the result is the same as multiplying by a single grid you could have computed in advance. Stacking would be pointless. The nonlinear step between them is what stops the layers from collapsing into one.
One more piece of arithmetic, and it is the one I would most want a reader to leave with, because it turns every specification sheet from a claim into something you can check.
A grid of shape [a, b] contains exactly a x b numbers, and every one of them is a learned parameter. That is all a parameter count is: add up the sizes of every grid in the model.
Work an example. The embedding table in part three is [151936, 4096], which is about 622 million parameters, in one table, before any layer exists. In each block, the attention projections are four grids of roughly [4096, 4096], or about 67 million, and the feed-forward layer is two grids of about [4096, 14336], adding roughly 117 million. Call it 185 million per block, times thirty-six blocks, which is about 6.7 billion. Add the embedding table and you are in the neighborhood of the eight billion the model is named for.
The arithmetic is rough and the real number depends on details I have skipped, but the shape of it is right, and being able to do that estimate is genuinely useful. It tells you where a model's parameters actually live, which is mostly in the feed-forward layers. It tells you why the embedding table is worth reusing at the output end. And it is the same calculation that makes low-rank adaptation obvious rather than magical.
This is how the memory figures in part one were derived. Parameters times bytes each. Eight billion parameters at two bytes is about 16 GB, and at four bits is about 4 GB. There is no other step. Once you can count parameters from shapes, you can predict whether something fits before downloading sixteen gigabytes to find out.
It is also how the LoRA argument works. Part seven replaces a [4096, 4096] grid, which holds about 16.8 million numbers, with a pair shaped [4096, 16] and [16, 4096]. By the shape rule their product is [4096, 4096], the right shape to add on. By the counting rule they hold about 131,000 numbers between them, which is well under one percent. The whole technique is those two rules applied together, and it should feel obvious by the time you get there.
The count does not tell you the compute cost. A grid used once per token and a grid used once per token per position are the same size and very different amounts of work. Attention is the case where cost grows with the square of your input length while the parameter count stays fixed, which is why part four talks about cost separately from size.
Last piece, and it is the one that connects all of this to the hardware decisions in part nine.
Recall the property from the worked example: every output cell is independent, and there are no branches. A processor designed for general code is built around the opposite assumption. It has deep pipelines, elaborate branch prediction and large caches, all aimed at running one instruction stream as fast as possible, because most software is a sequence of decisions where each step depends on the last.
Matrix multiplication has no decisions in it at all. It is millions of multiply-and-add operations that could all happen at once, in any order, with no coordination. A chip aimed at that workload does not need branch prediction or deep pipelines. It needs an enormous number of simple arithmetic units and, above all, the memory bandwidth to keep them fed.
Which is what a graphics card is, and not by coincidence: rendering has the same character, with millions of independent pixels. These models did not get built for graphics hardware. Graphics hardware happened to already be the right shape, which is a large part of why this field moved as fast as it did once somebody noticed.
The practical consequence runs through the rest of the course. Modern cards have far more arithmetic capacity than they have bandwidth to feed it, so a great many real workloads spend most of their time waiting on memory rather than calculating. That is why part five distinguishes prefill from decode, why the FlashAttention discussion in part four is about memory traffic rather than arithmetic, and why the cards in my own machine sit at a reduced power limit for serving with almost no measurable cost.
Four things, and none of them requires anything further.
Shapes are written rows first. Two grids multiply when the inner dimensions match, and those inner dimensions vanish while the outer two become the result. Every output cell is one row against one column, multiplied pairwise and summed. And a grid of shape [a, b] holds a x b learned numbers, which is the only arithmetic behind every parameter count and every memory figure in this course.
That is enough to read a shape error, estimate whether a model fits on your hardware, and understand why the low-rank trick in part seven works rather than taking it on faith. Anything beyond it, I would go and learn properly from somebody who teaches mathematics rather than from a write-up about graphics cards.
Next is the first stage of the pipeline, which is the one nobody thinks about and which explains more of the famous stupid failures than anything else in the architecture.