A year-in-review written from a rack in a spare room, which is a narrow vantage point and an honest one. I ran open models daily on my own hardware for twelve months. That tells you a great deal about some of what happened this year and nothing at all about the rest, so I have tried to be exact about which is which.
20 December 2025·19 min read
Before any of it, the vantage point, because a year in review is only worth reading if you know where the author was standing.
I have four 3090s in a box in my house. That is 96 GB of VRAM, which sounds like a lot until you start reading model cards. Everything I say below about latency, quantisation, tool-call reliability and what a model feels like on the fiftieth turn of an agent loop comes from running these things every day, on my own hardware, with nobody's marketing in the loop.
Everything I say about frontier models comes from using them as a customer, like everyone else. I cannot inspect them, I cannot host them, and I have no visibility into how they were trained or what was thrown away before release. Where I am repeating a lab's own account of its own product, I have tried to say so.
And there is a third category that matters more than people admit in these posts: things nobody outside a handful of companies could evaluate this year. Training runs. Safety evaluations at scale. Whether the capability curves are being bought with compute or with method. I have opinions. I do not have evidence, and I have tried not to confuse the two.
Here is the shortest version of 2025 I can write. The models stopped being things you talk to and became things that do. Everything else this year, the good and the ugly, follows from that one change.
In 2023 and 2024 the interesting question was whether a model could produce the right answer. In 2025 the interesting question became whether a model could be handed a goal, a set of tools, credentials, and a long afternoon, and be trusted with the result. That is a different question with a different failure mode. A wrong answer is an inconvenience. A wrong action is an incident.
The capability story, the security story, the crime story and the money story are all the same story told from four angles. Once a model can act, its usefulness and its danger become the same property, and you cannot buy one without the other. That is not a rhetorical flourish. It is the practical reason prompt injection is still unsolved at the end of this year, and the reason a person with no meaningful technical skill could run a credible extortion campaign in August.
The frontier first, briefly, because it is the part everyone else has already covered.
The year opened with reasoning. DeepSeek-R1 arrived in January with visible chain-of-thought and open weights, and within months every major lab shipped a model that thinks before it answers, or a control for how long it should think. By December, deliberate reasoning is not a feature, it is the default posture, and the argument has moved on to how much of it you can afford per request.
The releases came steadily. OpenAI shipped GPT-5 in August and GPT-5.1 in November. Anthropic shipped Claude 4 in May and Opus 4.5 in November. Google shipped Gemini 3 in November. The specific ordering matters less than the shape: at the end of 2025 no lab holds a decisive, obvious lead, and the differences that survive contact with real work are about agentic reliability and price, not about which one wins a benchmark.
Which is the more interesting development, and the one I did not expect at the start of the year. Benchmarks stopped being the story. Not because they were solved, but because they stopped predicting anything I cared about. A model that scores well and then loses the thread on turn forty of an agent loop is not useful to me, and no leaderboard in wide circulation measures turn forty.
The most honest measurement I saw all year came from METR in March, which asked a better question: how long a task can a model complete, measured by how long it takes a human professional, at a fifty percent success rate. Their answer was that the length has been doubling roughly every seven months, with the recent period faster than that. Whatever you think of the extrapolation, the framing is right. Duration of autonomy is the axis that matters now, and it is the axis my own experience is organised around.
The open-weight story is the one I have the strongest view on, because it is the one I lived, and it went somewhere I did not predict.
Two things were true about open weights in 2025, and they pull in opposite directions.
The first is that the open frontier moved decisively to China. DeepSeek in January and again in August, Qwen through the whole year, Moonshot's Kimi K2 in July, Zhipu's GLM line from the summer. The releases were fast, permissively licensed, and good. Whatever the geopolitics of that, as a practitioner it meant the best model I could actually download in a given month was usually Chinese, and it stopped being remarkable by about April.
The second is that open stopped meaning accessible, and this is the part that gets glossed over. Kimi K2 is a trillion parameters total with about 32 B active. DeepSeek's flagship is in the same weight class. These are genuinely open weights, published under permissive licences, and I cannot run any of them. Not slowly. Not at all. The weights alone are several hundred gigabytes, and 96 GB of VRAM is not in the conversation.
So "open" split into two meanings during 2025. Open as in auditable, forkable, and not subject to a vendor's deprecation schedule, which is real and valuable and applies to the trillion-parameter models as much as the small ones. And open as in "I can run this myself", which quietly became a property of a narrowing band of models in the 20 B to 120 B range. The band I live in is now a middle class, with a frontier above it I can read about but not host.
OpenAI's gpt-oss release in August was the interesting counter-example, and I will come back to it, because I did test that one and it did not go the way the announcement suggested.
Kimi K2. DeepSeek V3.1. Llama 4 Behemoth, which never shipped in a form anyone outside Meta could evaluate. Any frontier closed model at the weights level. Anything that needs more than 96 GB, which by the second half of the year meant most of the models generating the most excitement. When I say a model is good or bad below, I am talking about the ones that fit.
In 2024 the argument about open models was mostly about licensing, and by mid-2025 that argument was substantially over. Apache 2.0 and MIT-style terms became normal for serious releases. The bottleneck moved to memory, and memory is a harder problem than a lawyer, because you cannot negotiate with it.
What actually ran in my house this year, with the numbers, because a review with no measurements in it is a press release.
The rig came online in January with Qwen2.5-Coder-32B, which measured about 36 tok/s single-stream against a theoretical 42. That gap between expected and measured is the constant of the whole year: quantisation overhead and pipeline-parallel communication on consumer cards without P2P take a consistent bite, and anyone quoting you a number that does not have that bite in it has not measured it on hardware like mine.
The R1 distills in January were the first time the box felt like it was thinking rather than autocompleting, and I do not think I appreciated at the time how much of the year that would define. QwQ-32B in March was a dedicated reasoner at a size I could actually hold. Gemma 3 in March brought multimodal work in reach locally. Qwen3 in April, with its hybrid thinking and non-thinking modes, became the daily workhorse, and its sparse variants were the first clear signal of where the whole stack was heading.
Llama 4 Scout in April was the year's most instructive disappointment for me. On paper it is a big mixture-of-experts, 109 B total. In practice 17 B of those parameters fire per token, which is heavy per-token compute compared to the 3 B-active Qwen sparse models, so it ran, and it ran slower than its quality justified. It taught me to read active parameters before total parameters, which is now the first number I look for.
May was the turn. Devstral Small was the first model I ran that was built specifically for agentic software engineering rather than for completing code, and the difference showed up exactly where I did not expect it: not in code quality, in tool-call reliability. The agent loop stopped falling over. July's Qwen3-Coder, with a 256K context and about 75 tok/s measured, is the point where local coding stopped feeling like a demonstration and started being how I work. August's GLM-4.5-Air was the first non-Qwen model all year that seriously competed for the daily-driver slot.
And then gpt-oss-120b, which is the honest failure in my year and the reason I trust my own numbers over announcements. OpenAI released it in August under Apache 2.0, a roughly 120 B mixture-of-experts in MXFP4, described as efficient on consumer hardware. Its quantisation is tuned for Hopper. On Ampere it falls back to a different kernel path, and I measured about 30 tok/s against an expected 44, the worst expected-to-measured ratio of anything I ran all year. At roughly 63 GB it also needs every card in the box, so it can never sit beside a coder. It is a capable model that my hardware is the wrong shape for, and no benchmark table was going to tell me that.
The agentic turn is the thing I would put at the centre of any honest account of 2025, and it happened in the tooling layer rather than in the models.
Model Context Protocol went from a thing one lab published to something close to a default this year. Whatever you think of the specification, having a common way to hand a model a set of tools mattered more in practice than a few points of benchmark score, because it meant a capability built for one client worked in another. Standards are boring and they are how a capability becomes an ecosystem.
Coding agents were the first place any of this became load-bearing rather than impressive. There is a specific threshold, and I crossed it in the summer: the point where you stop reading every diff before it is applied and start reading the ones that fail. That is a change in your relationship with the tool and it happens quietly, and I think a lot of people crossed it this year without noticing they had.
What still breaks, at the end of 2025, is duration. Not intelligence. Error compounding over a long horizon is the unsolved practical problem: an agent that is 97 percent reliable per step is a coin flip over twenty steps and useless over a hundred. Every serious agent harness I used this year is fundamentally a machine for catching and correcting the model's own mistakes, and the good ones are mostly scaffolding, not model.
The corollary, which is uncomfortable and which I believe: most of the observed capability gain in agentic work in 2025 came from harness engineering, not from the models getting smarter. That is not a criticism. It is a statement about where the leverage was, and it is why I am cautious about extrapolating capability curves that were measured with a moving harness.
Which brings us to the part of the year I find genuinely difficult, and where I think the industry spent twelve months being less honest than it should have been.
Prompt injection was not solved in 2025. It was not substantially mitigated. It is, at the end of this year, a structural property of the way we build these systems, and the year produced a run of demonstrations that should have been more alarming to more people than they were.
The clearest was EchoLeak, disclosed in June as CVE-2025-32711, a zero-click vulnerability in Microsoft 365 Copilot. An attacker sends an ordinary-looking email with instructions hidden in it. The victim does not open it, does not click anything, does not do anything at all. Later, the assistant retrieves that mail as context for an unrelated question, follows the instructions it finds there, and exfiltrates data the user had access to. No user error at any point in the chain. The exploit is that the assistant was helpful.
Simon Willison's framing of the lethal trifecta is the most useful thing written on this all year, and it is three bullet points long: access to private data, exposure to untrusted content, and the ability to communicate externally. Any system with all three can be turned against its user, and the trouble is that all three are exactly what makes an assistant worth having. You cannot patch this one. You can only decline to combine the ingredients.
Then in October the industry shipped agentic browsers, which is the trifecta as a product category. ChatGPT Atlas launched on 21 October. Within days, researchers demonstrated that its combined address and command bar could be fed a string that looks like a URL and is treated as an instruction. Perplexity's Comet had structurally similar problems. To OpenAI's credit they said out loud what most vendors would not, which is that prompt injection is unlikely to ever be fully solved, in the way that scams and social engineering on the open web have never been solved. I believe them. I also notice they shipped it anyway, and so did everyone else.
My own position, arrived at the boring way by being burnt in small ways: I do not let an agent hold credentials to anything I would not hand a stranger for the afternoon. Read-only where possible. Separate accounts for anything an agent touches. Nothing with money in it, ever. This is not sophisticated advice and I am aware it will not scale to the products people are actually shipping. It is, however, the only thing I have found that works, and I would rather write down something that works and does not scale than something that scales and does not work.
And the part almost nobody wanted to write about, which is what happens when these tools reach people whose goals are not yours.
Two disclosures this year should be read together, both from Anthropic, both about its own product, which is worth noting because a company publishing detailed accounts of criminals using its tools is doing something the industry does not reward.
In August they described an actor running a data theft and extortion campaign against at least seventeen organisations across healthcare, emergency services, government and religious institutions, using an agentic coding tool to automate reconnaissance, credential harvesting, network penetration and the analysis of stolen data. The model was also used to work out which victims could pay the most, and to write the ransom notes for maximum psychological pressure. Demands ran from seventy-five thousand dollars to over half a million. The same report described ransomware kits being sold by an operator who did not have the skills to write ransomware.
In November they described something further along: a state-linked group that used an agentic tool orchestrated through MCP servers to run an espionage campaign against roughly thirty organisations, breaching at least four, with the company assessing that the model performed eighty to ninety percent of the operation independently. The humans were approving checkpoints, not directing steps.
Here is what I think this means, and it is not the thing that got headlines. The headline was that AI can now conduct a cyberattack. The more important fact is that the floor collapsed. Offensive security has always had a steep skill curve, and that curve was itself a control: it limited how many people could run a competent intrusion, and it limited how many targets each of them could work at once. Both limits were structural, and both were load-bearing for everyone's threat model whether they knew it or not.
An agentic tool removes the second limit entirely and lowers the first one a great deal. A person who could not previously have run a campaign at all can now run one, and a person who could run one against a single target can run thirty in parallel. The ceiling of what the very best attacker can do is not what changed this year. The floor did, and the floor is where the volume lives.
The defensive side of this is real too, and I do not want to be one-sided. The same automation helps defenders, and defenders have more to automate. But the asymmetry favours the attacker in the short run for a boring structural reason: an attacker needs one path to work and can fail silently a thousand times, while a defender has to be right everywhere and every failure is visible. Automation multiplies both, and the multiplication is worth more to the side that only needs to be lucky once.
The money, which I have watched with the particular interest of someone who spent about seven thousand dollars on GPUs and thinks about the arithmetic more than is healthy.
Capital expenditure on AI infrastructure in 2025 was not like anything I have seen in my career. The largest cloud and infrastructure providers between them reported quarterly capex figures growing at rates that would have been considered implausible two years ago, with the third quarter alone running well over a hundred billion dollars across the big five, up around seventy-five percent year on year. The commitments for next year are larger again.
Alongside that, the deals got circular in a way that is hard to unsee once noticed. A chip vendor investing in a model lab that then commits to buying that vendor's chips. A cloud provider taking an equity position in a customer whose primary expense is that cloud. None of this is illegal or even unusual in a capital-intensive buildout, and it does make the revenue figures harder to read, because some fraction of the demand is being financed by the people booking it as demand.
Against all that, the MIT NANDA report in August landed hard, claiming that around ninety-five percent of enterprise generative AI pilots were producing no measurable return, against thirty to forty billion dollars of enterprise spending. The number got repeated more confidently than its methodology supports, and I would treat it as directionally interesting rather than precise. But the finding underneath it matches everything I have seen from the inside of an engineering organisation: the failures are not model failures, they are integration and workflow failures, and buying a tool is the easiest five percent of the job.
I do not know whether this is a bubble. I know that the phrase gets used to mean two different things, and they should be separated. If it means the technology is not real, no, and anyone who has watched a local model refactor a codebase overnight on hardware that costs less than a used car will tell you it is real. If it means more capital has been committed than the near-term revenue supports, that is arithmetic rather than opinion, and the arithmetic is currently uncomfortable. Both of those can be true at once. They usually are.
Regulation, briefly, because 2025 was the year the rules arrived and immediately started being renegotiated.
The EU AI Act's obligations for general-purpose AI models took effect on 2 August 2025, which made it the first serious regime anywhere to place direct duties on model providers rather than only on deployers. That is a real milestone regardless of what you think of the text.
Then, on 19 November, the Commission published a Digital Omnibus proposing to defer the high-risk obligations that were due next August, pushing them out as far as December 2027. The stated reason is that the standards and guidance the rules depend on are not ready. The unstated reason, which everyone understands, is competitive pressure.
I am not going to pretend to a confident view on whether the delay is correct. I will note the pattern, because it is the pattern that matters more than any individual provision: this is a field where the deployment speed exceeds the rule-making speed by enough that the rules are being adjusted to the deployments rather than the other way round. That is worth watching without treating it as a scandal, because it is also what happened with the internet, with cloud, and with every general-purpose technology I can think of.
The honest inventory. Things I have opinions about and no ability to verify, which is most of the things that matter.
The trillion-parameter open models. The large dense frontier weights. I read the cards and the third-party evaluations like everybody else, and I have no measured basis to tell you whether they are as good as claimed. When I say a model is good this year, I mean a model that fits in the box.
I use them. I cannot inspect them. Every claim about training data, refusal behaviour, evaluation results or safety mitigation in a closed model is a claim I am taking on trust from a party with an interest, and the fact that I mostly find those parties credible does not turn it into evidence.
My box serves one person. Everything I know about throughput is single-stream or small-batch. Concurrency changes the shape of every number I quoted above, and organisations making infrastructure decisions should not use my measurements as anything more than a floor.
Four consumer cards without NVLink is an inference machine, for reasons I worked through when the rig was built: no peer-to-peer, so PCIe becomes the bottleneck the moment gradients need to move. So the entire question of whether capability gains are coming from scale, from data, or from method is one I can only read about.
I run agents for hours, not weeks. The interesting failure modes of a system that has been operating autonomously for a month, accumulating state and drift, are ones I have not observed and cannot speak to. I suspect this is where the next round of unpleasant surprises comes from.
Genuinely unknown to me and, I would argue, to most people asserting confident positions in either direction. Evaluations are run by the labs on their own models with their own methodology. That is not nothing, and it is not independent verification either.
What I expect from next year, written down so it can be held against me.
Every agentic deployment shipped in 2025 with credentials attached is an incident waiting for an attacker who has read the same research I have. I expect at least one very large, very public breach in 2026 whose root cause is an indirect prompt injection into an agent that a company gave real permissions to, and I expect the post-mortem to describe it as sophisticated when it was not.
The 3 B-active-of-80 B pattern is the most useful architectural trend of the year for anyone running local hardware, because it decouples quality from per-token compute. If that continues, the band of models a person can self-host stays meaningfully capable. If everything goes dense and large, the self-hosting story narrows to hobbyists inside two years.
I expect the useful evaluations of 2026 to be about how long an agent can be left alone before it needs a human, and I expect the honest answer to stay much shorter than the marketing.
The misuse story of 2025 was that capable offensive tooling reached people who could not previously build it. Nothing about 2026 obviously reverses that, and the defensive response is slower than the offensive adoption because it has to go through procurement.
The thing I keep coming back to, at the end of twelve months of this, is that 2025 was the year the interesting questions stopped being about the models and started being about everything around them. The harness. The permissions. The blast radius. The person on the other end.
That is a less exciting story than the one being told, and it is better news for practitioners, because those are all things you can actually engineer. The model is somebody else's research programme. The system around it is yours, and this was the year that stopped being a detail.