Broadcom closed the VMware deal and, within a few months, the free hypervisor I'd run for years was gone. This is the survey I did before touching anything: what Broadcom actually changed, why they changed it, and which hypervisors were realistic replacements for a home lab that leans on PCI passthrough.
4 February 2024·27 min read
Broadcom closed its acquisition of VMware in November 2023, and the changes started almost immediately. Perpetual licences were retired in favour of subscription only. The product catalogue, which had run to thousands of SKUs, was collapsed into a small number of bundles built around VMware Cloud Foundation and vSphere Foundation. Licensing moved to a per-core model with a minimum core count per CPU, which quietly repriced anything running on modest hardware.
Then, in February 2024, the free vSphere Hypervisor was discontinued. That was the one that mattered here. Free ESXi was how an enormous number of people ran a home lab, learned the platform, and then went to work and specified it. It cost nothing and it was the on-ramp.
None of this was erratic, which is worth saying plainly. It is the same playbook Broadcom ran at CA Technologies and again at Symantec: buy an established enterprise vendor, concentrate on the few hundred largest accounts, drop or reprice everything below them, convert licences to subscription, and let margin rise as the long tail leaves. Hock Tan has been unusually candid that this is the strategy rather than a side effect of one.
Under that model a free hypervisor is not a marketing expense, it's a cost with no attributable revenue. Neither is a small partner, or a customer with two sockets and a support contract worth a few thousand dollars a year. The people running VMware at home were never the customer being optimised for, and in Broadcom's arithmetic the goodwill they generate doesn't show up on a line.
The part I think Broadcom underrates is where that goodwill came from. A generation of infrastructure engineers, myself included, learned vSphere on hardware they paid for themselves, at home, on a free licence. That's how it became the default answer in procurement meetings for twenty years. You can stop paying for that pipeline and keep the revenue for a long time before the effect shows up, but the effect is real.
Before comparing anything I wrote down what the lab actually had to do. It's easy to pick a hypervisor on features you'll never use, so this is the list that could disqualify an option outright.
Non-negotiable. Half of what makes the lab useful is hardware handed directly to guests: HBAs for storage, network cards, a GPU. It's also exactly where hypervisors differ most in practice: IOMMU grouping, VFIO binding, whether a card initialises cleanly, and how much of it fights you at boot.
Server boards and consumer parts in the same rack. Anything with a narrow HCL was going to be a problem.
VM snapshots alone don't count. I wanted a restore path I'd actually rehearsed, with ZFS underneath rather than bolted on.
The whole point. If the answer costs enterprise money, I may as well have kept paying VMware.
When passthrough breaks at midnight, forum posts and a wiki matter more than a support SLA I don't have.
None of these would rule an option out on their own. One node runs fine without any of them. But they're the difference between a platform I'd still be happy with in three years and one I'd outgrow, and when two candidates were otherwise close, this is what separated them.
One node today. But I didn't want to pick something where adding a second node meant changing platform, which is exactly the trap I'd just climbed out of. Whether clustering is built in or sold separately also tells you how a vendor thinks about the people who aren't buying support.
Nothing here needs to survive a host failure unattended, and HA on a single box is theatre. It mattered as a signal rather than a feature: a platform that has thought properly about fencing and quorum has thought about the boring, unglamorous parts, and those are the parts you meet at 2am.
VLANs get you a long way and a managed switch gets you the rest. But overlays, isolated tenant networks and VXLAN without touching switch config would let me build lab topologies that don't exist physically. Being able to stand up an isolated three-tier network to break, and then throw it away, is most of what a lab is for.
These are the ones I looked at seriously, roughly in the order I worked through them.
Considered, then ruled out
The obvious first move: VMUG Advantage still offered lab licences for a few hundred dollars a year. It would have kept everything exactly as it was, which is worth something. I ruled it out less on price than on direction. Everything Broadcom had done in three months pointed the same way, and building the next few years of my lab on a licence programme that existed at their discretion felt like renting on a month-to-month with a landlord who'd just doubled the rent next door.
Front-runner
Debian underneath, KVM and LXC on top, ZFS as a first-class citizen, and a web UI that doesn't hide the fact that there's a real Linux system down there. Passthrough is well-trodden ground with a large body of community documentation, which matters more than elegance when you're binding a controller to VFIO and it won't come up. Clustering is built in rather than a separate product, and the paid subscription buys a support contract and the enterprise repo rather than unlocking features.
Strong, but a harder fit
Well engineered, properly open, and the closest thing to a like-for-like vSphere replacement in feel: XCP-ng as the host, Xen Orchestra as the management plane. The reservations were specific to my use case rather than the product. Xen's passthrough story is less commonly travelled than KVM's, which means fewer people have hit and written up the exact failure I'm about to hit, and the nicest build of Xen Orchestra is the paid appliance.
The honest baseline
Every option here except one is KVM wearing a different coat, so running libvirt directly with virt-manager or Cockpit was always on the table. Maximum control, no abstraction to fight, and the passthrough documentation on the internet is largely written against exactly this. What you give up is everything around the edges: no clustering, no integrated backup, no snapshot UI, and a lot of XML.
Wrong centre of gravity
Linux-based, ZFS-native, KVM underneath, and it would have consolidated storage and compute on one box. But it is a storage appliance that also runs VMs, and I wanted the reverse. Making the hypervisor a feature of the NAS puts the thing I care most about downstream of someone else's release priorities.
Good, aimed elsewhere
Deservedly popular in home labs, strong at mixed-size disks and containers, and passthrough is well supported. It's licensed rather than free, and its storage model is built around a parity array rather than ZFS pools. For a media-and-Docker box I'd recommend it without hesitation; for a passthrough-heavy lab host it was solving a different problem.
Too much machinery
SUSE's HCI platform, KubeVirt on top of Kubernetes, and architecturally the most interesting thing on this list. Also the most infrastructure to operate before the first VM boots. If the lab were a Kubernetes lab it would be a great answer. It isn't.
Free, but not free-standing
A real HCI stack with a free community edition and a management experience that makes most open-source UIs look dated. The catch is that CE exists to introduce you to the commercial product, it is fussier about hardware than the alternatives, and I had just finished learning what it feels like when the free tier of a commercial platform is reconsidered.
Not a serious candidate
Worth naming because people ask. Microsoft had already discontinued the free standalone Hyper-V Server, so the realistic path was a Windows Server licence, and device passthrough on Hyper-V is materially more awkward than on KVM. It would have meant swapping one vendor's licensing weather for another's.
Once the shortlist was written down it split cleanly in two, and the split mattered more than any individual product. Some of these are hypervisors that happen to manage storage. The others are storage platforms that happen to run VMs. That ordering decides what breaks first when you push on it.
The hypervisor is the product. It installs on bare metal, owns the hardware, and everything else is arranged around scheduling CPU, memory and PCIe devices to guests.
The file server is the product. It owns the disks, presents shares, and virtualization arrived later as a feature alongside containers and apps.
For a media server, backups and a dozen containers, the storage-first platforms are the better answer and it isn't close. That's most home labs, and I'd have picked one without much agonising.
Mine wasn't that. What I cared about was hardware passed straight into guests, which is precisely where the ordering bites. Passthrough on a storage appliance is a feature that has to keep working through upgrades driven by entirely unrelated priorities, and there is no way to escalate when it doesn't.
So the choice narrowed before I ever compared feature lists: I needed the hypervisor to be the product. Everything after that was picking which one.
XCP-ng was the option I wanted to win. Coming off years of vSphere it was the only candidate that felt familiar in the right way: a type-1 hypervisor on the metal, a separate management plane in Xen Orchestra, pools that behave like clusters, live migration where you expect it. Everything else on the list asked me to think differently. This one let me keep thinking the way I already did, and after the month I'd had, that was worth something.
It also has a strong story where vSphere charged me extra. Delta backups, continuous replication and scheduled jobs are built into Xen Orchestra rather than sold alongside it. If someone asked me to argue for XCP-ng, that's where I'd start.
Then I started reading the documentation properly, and the operational surface turned out to be much larger than the interface suggests.
Xen Orchestra comes two ways. XOA is the packaged appliance, and the tiers worth having are paid. XO from sources is a legitimate path that plenty of people run happily, but it makes the management plane something I build and maintain: a Node application I keep current, rather than an appliance that updates itself. Neither is wrong. But I had just walked away from a platform over licensing pressure, and choosing one where the polished version is the commercial version needed more thought than I expected.
Dom0 is the control domain, and it is small on purpose: a modest root filesystem and a fixed slice of RAM that does not grow just because the host has plenty. Everything administrative goes through it. Disk exports are attached to Dom0 and served over XAPI HTTP, so backup throughput is a function of how much headroom Dom0 has, and an overloaded Dom0 shows up as unrelated-looking failures: disks slow to attach, requests timing out. It is a tuning surface that exists whether or not you know about it, and vSphere never made me think about the equivalent.
The storage documentation is clear that the stack uses VHD, and VHD tops out at 2 TiB per virtual disk. You can use raw instead and go bigger, but raw gives up snapshots and live migration, which are most of the reason to run a hypervisor. Provisioning also depends on which SR type you chose at install: Local LVM is thick, Local EXT and NFS are thin. That's a decision made early, on incomplete information, that determines how much snapshots cost you forever after.
Deleting a snapshot doesn't free the space; it queues a coalesce, and the parent chain is merged in the background. The chain limit is 30, but the documentation is blunt that with large disks you hit the SR space limit first. Take backups faster than coalescing completes and the chain keeps growing until backups start failing. Xen Orchestra is good about this, detecting uncoalesced disks and skipping them rather than compounding the problem, but "my backups stopped and the reason is a background merge queue" is a class of incident I did not want to add to the lab.
This was the one that mattered most, because ZFS was on my non-negotiable list. XCP-ng can do ZFS, but the documentation says plainly that it is offered as-is and without official support, and recommends at least 16 GB of Dom0 RAM because ZFS in a standard configuration will use half of Dom0's memory as cache. So the storage layer I considered essential is a community-maintained addition that competes with the control domain for RAM. On Proxmox, ZFS is a first-class install option.
In fairness, this is where XCP-ng is strong. Pools behave like vSphere clusters, so clustering is there when I want a second host. Pool HA exists, though it wants shared storage for the heartbeat SR, which is a real prerequisite rather than a checkbox. And the SDN Controller plugin in Xen Orchestra does private networks over GRE and VxLAN, so overlays are properly available. All three of my nice-to-haves are covered. They just weren't enough to outweigh the list above.
Xen does PCI passthrough properly and it works. But the number of people who have passed hardware through Xen is a fraction of the number who have done it through KVM and VFIO, and that ratio is what you're actually leaning on at midnight when a card won't initialise and the error is four words long. I wasn't only choosing a hypervisor, I was choosing which forum I'd be searching at 1am.
None of these is damning on its own, and I want to be fair: XCP-ng is a serious, well-engineered platform run by people who clearly care, and its documentation is honest about its own constraints in a way I wish more projects were. Half of what's above I know because Vates wrote it down plainly rather than hiding it.
But the list adds up. A 2 TiB disk ceiling, a provisioning model fixed at install, a background coalesce process that can stop backups, a control domain to tune, a management plane to either buy or maintain, and ZFS as an unsupported add-on that eats Dom0 RAM. Individually, all manageable. Together, that's a lot of platform to hold in your head for a lab that exists so I can do something else.
The option that felt most like home turned out to be the one that fit my actual workload least.
TrueNAS got the closest to being the answer, because it was already in the house. The file server the rest of the network depends on runs TrueNAS, and it has been the least dramatic piece of infrastructure I own. ZFS underneath, snapshots that work, replication that works, scrubs that find things before I do. When SCALE moved the platform onto Linux with KVM built in, consolidating storage and compute onto one box stopped being a thought experiment.
So I tried it properly rather than dismissing it on architecture. Spun up VMs, passed hardware through, and pushed on the parts I'd actually depend on. The verdict wasn't that it's bad. It's that the further I got from "serve files, run a couple of always-on VMs", the more I could feel which half of the product had the institutional weight behind it.
ZFS wants ARC and is very good at using it. A VM host wants that same RAM sitting inside guests. On one box those appetites are in direct competition, and the storage side wins by design, because starving ARC to feed a VM is how you make the file server slow for everyone in the house. On a dedicated hypervisor that tension simply doesn't exist.
This is the one that settled it. A NAS upgrade is driven by storage priorities, and if a release changes something underneath the virtualization layer that's a perfectly reasonable trade for the product to make. The file server matters more than the lab does, so I want it boring and I want to patch it without a second thought. Putting the lab's passthrough on the same box turns every routine storage update into a change that could take the rest of the lab down with it.
SCALE's VM layer arrived years after the storage engine and is maintained accordingly. It is fine for always-on service VMs. It is thinner exactly where I needed depth: PCIe passthrough, which is a niche within a feature that isn't the product's centre of gravity.
Clustering for VMs doesn't exist. Neither does HA. Networking is bridges and VLANs, with nothing resembling an overlay, so the software-defined side of my wishlist was simply off the table. That's entirely consistent with what the product is, and it means the ceiling is one box forever. Fine for a file server. A dead end for the thing I'd be building on top of it.
Consolidation cuts both ways. Collapsing storage and compute onto a single machine means one reboot takes out both, and the thing that makes me reboot is usually the thing I care least about. Keeping them separate costs an extra box and buys the ability to break one without the other.
That's a separation-of-concerns argument more than a criticism, and it's why TrueNAS is still the file server here and always will be. It just isn't the hypervisor. There's more to say about the storage side than fits in a page about choosing a hypervisor, so I'll write that up properly on its own.
The one I expected to be a compromise and wasn't. Installed it on the spare box, had an HBA passed through to a guest the same afternoon, and spent the rest of the evening looking for the catch. This section is longer than the others because it's the one I'm still running, and because a page that only lists a winner's virtues isn't worth reading.
Not because Proxmox does anything clever, but because it's KVM and VFIO underneath, which is what the overwhelming majority of passthrough writing on the internet is written against. Every error I hit had already been hit, written up and argued about by someone else. IOMMU groups, binding to vfio-pci, the cards that won't reset, the ones that need Above 4G Decoding: all of it is documented by people who were annoyed enough to write it down. For the requirement I could least afford to get wrong, that community depth outweighed every feature comparison I could have made.
Pick it at install and you get a root pool, snapshots in the UI, replication between nodes and scrub scheduling without assembling anything. After reading XCP-ng's "offered as-is, and give Dom0 16 GB because ZFS will eat half of it" caveat, having the storage layer be a supported first-class choice mattered more than it sounds. It also means the hypervisor and the storage are versioned together and tested together, rather than one being a community addition to the other.
LXC alongside KVM turned out to be more useful than I expected. Most of what a lab actually runs, whether that's a DNS server, a reverse proxy or a metrics collector, doesn't need a full VM, and running those as containers left the RAM where I wanted it. It's the kind of thing that doesn't appear on a comparison table and changes how you use the machine daily.
Corosync clustering is in the box. So is ha-manager, with real fencing rather than a best-effort restart. And SDN shipped by default in 8.1 in November 2023, a few months before I was looking, covering VLAN, VXLAN and EVPN zones. There's no clustering edition, no per-socket unlock, no feature that appears when you pay. Every other option either lacked one of the three or put it behind a licence, and after the year VMware had just given me that was close to the whole argument.
Worth being honest that in early 2024 the SDN stack had only just landed as a default. I built a VXLAN zone to see it work and then left it alone rather than making anything depend on it. It counted as a credible direction of travel more than a feature I was relying on that month.
The free repository is the complete product. Paying gets the enterprise repository and a support contract, which is a business model where the vendor's incentive is to be worth paying rather than to make the free version worse. That distinction is now permanently near the top of how I evaluate infrastructure.
The honest annoyances. Without a subscription you get a "no valid subscription" dialog on every single login, forever. It is a small thing that feels larger than it is, and there is a cottage industry of scripts to remove it. Related: the enterprise repository is enabled by default, so a fresh install throws apt errors until you switch it to the no-subscription repo. Both are documented, both take five minutes, and both are the first thing every new user hits.
There's a real Debian underneath and the UI does not stop you reaching it. That's exactly what I wanted, and it is also the sharpest edge here. Nothing prevents you editing a config the UI owns and finding out later that they disagree. An appliance would have protected me from myself. I'd rather have the shell, but that is a preference, not an objective virtue.
Corosync wants quorum, and two nodes can't provide it on their own. You need a third vote, usually a QDevice on something small like a Pi. Worth knowing before you plan a two-node cluster, because the obvious next step after one node is two, and that's exactly where you meet it.
None of the annoyances are architectural. They're a login dialog, a repository line, and a quorum rule, and I'd trade all three for the passthrough documentation alone. It has been the lab hypervisor ever since, and nothing on this list has made me reconsider.
The honest baseline. Nearly everything else on the list is KVM wearing a coat, so I ran libvirt directly with virt-manager for a while to see what the coat was actually worth.
This is where the documentation is written. Binding cards to VFIO, checking IOMMU groups, dealing with a card that won't reset. All of it is a solved, well-lit path. If passthrough were the only requirement, this would be the answer.
No clustering, no HA, no integrated backup, no snapshot UI worth the name, and a lot of XML. Each individually solvable, all of it maintained by me forever. I'd be rebuilding Proxmox badly.
Clustering means standing up Pacemaker and Corosync myself. HA means configuring fencing myself. Overlay networking means Open vSwitch and hand-written bridges. All three are achievable and all three are things I'd own the maintenance of, and a nice-to-have is supposed to work the other way round.
Popular for good reasons, and I gave it a fair run because half the home lab internet recommends it. It's aimed at a different machine than the one I was building.
The parity array is clever for mixed-size disks and for growing a pool one drive at a time. But I'd already decided ZFS was non-negotiable, and building around a different storage model to get there defeats the purpose.
No clustering, no HA, and no overlay networking. That's not an oversight, it's the product's whole shape: one well-loved box doing a lot of jobs. It does mean all three tiebreakers score zero, and that whatever I built would stay on one host permanently.
The cost is modest and fair. But I'd just been taught an expensive lesson about depending on someone else's licence terms, and I was in no mood to sign up for another set.
Architecturally the most interesting thing on the list: KubeVirt on Kubernetes, HCI built the way someone would build it today. I wanted to like it more than I did.
The abstraction is real and so is its weight. A problem with a VM can be a problem in the VM, in KubeVirt, in Kubernetes, or in the storage layer under all three. That's a lot of surface for a lab whose point is running models, not running Harvester.
This is the strongest option on the list for all three. Clustering and HA are native because Kubernetes is native, and SDN comes via Multus and Kube-OVN with more capability than anything else here. If those had been requirements rather than tiebreakers, Harvester wins. They were tiebreakers, and it lost on the requirement above.
A real HCI stack with a free tier and a management experience that makes most open-source UIs look dated. It got further than I expected and then hit the same wall twice.
CE has firmer opinions about what it will run on than the alternatives, and a lab built from server boards and consumer parts is exactly the kind of machine those opinions are aimed at.
Clustering and HA are the product, because it's an HCI platform and that's the point. But Community Edition is explicitly not for production and is limited in ways the commercial version isn't, and Flow, the SDN piece, is a licensed add-on that CE doesn't include. So on paper it scores well and in the version I could actually run, two of the three come with an asterisk.
The wall. I had just spent a month dealing with the consequences of exactly this arrangement, and evaluating another one on its merits turned out to be beyond me. That's arguably unfair to Nutanix. It was still disqualifying.
Included because people ask, and because I did put it on the list before taking it off again. It didn't survive first contact.
Microsoft had already discontinued the standalone Hyper-V Server, so the realistic path was a Windows Server licence. Swapping one vendor's licensing weather for another's is a lateral move at best.
Failover Clustering gives clustering and HA, and the Network Controller gives SDN, so nominally all three are present. Practically, the SDN piece is Datacenter edition only and the clustering wants shared storage or Storage Spaces Direct. Everything on my wishlist exists and everything on it is a line item.
Discrete Device Assignment works, but it's materially more awkward than VFIO, and the community that has done it on consumer hardware is small. Same problem as Xen, with a licence attached.
Worth stating plainly, because doing nothing was a real option and the cheapest one to evaluate. VMUG Advantage still offered lab licences for a few hundred dollars a year.
Everything would have kept running exactly as it was. But every change Broadcom had made in three months pointed the same direction, and the programme existed entirely at their discretion. Building the next few years of the lab on it felt like signing a month-to-month with a landlord who'd just doubled the rent next door.
Clusters, DRS, vMotion and vSphere HA were the reason everyone learned this platform, and none of it is in question. But SDN meant NSX, which was a separate licence and never in reach on a lab programme. So the honest scoring is two of three, plus a permanent reminder that the third was available to people with a purchase order.
The strongest reason to stay was that vSphere skills are what employers ask for. But if the free tier is gone, the next generation isn't learning it at home either, and that pipeline was the thing that made those skills valuable in the first place.
Written down, the list collapsed quickly. Half the options were solving a different problem than mine. Two were commercial free tiers of exactly the kind that had just been withdrawn from under me, which was hard to un-notice. One was the baseline everything else is built on.
What I needed was KVM passthrough with community documentation deep enough to get me out of trouble at midnight, ZFS underneath, clustering available for the day I want a second node, and no vendor holding a lever they could pull the way Broadcom had just pulled one. Proxmox was the only option that had all four without an asterisk.
It has been the lab hypervisor ever since. Everything built after this decision sits on it, and none of the reasons I chose it have needed revisiting.
The wider point has stayed with me longer than the decision did. The free tier was never charity. It was distribution, training and defaults, bought cheaply. Removing it books an immediate saving and an invisible cost, and the invisible one lands years later, in rooms where nobody remembers why VMware used to be the obvious answer.