Nobody Is Patching MySQL 5.7

A database version reaching end of life is unlike any other deadline in operations, because nothing observable changes on the day. There is no error, no degradation, no alert. The software you were running on the thirty-first of October is bit-for-bit the software you are running today, and it will keep doing its job for years if you let it. What ended was not the software. What ended was the arrangement under which somebody else fixes it.

Oracle's lifetime support policy puts MySQL 5.7 at general availability in October 2015, Premier Support ending five years later in October 2020, and Extended Support ending on the thirty-first of last month. After that comes Sustaining Support, which is indefinite and which is worth reading carefully, because indefinite sounds generous and means something narrow: you may keep calling, and you get the fixes that already exist. Not new ones. A vulnerability discovered in 5.7 next month has no patch coming for 5.7, and that is not a lapse, it is the published terms.

So the exposure is not that the version stops working. It is that the version stops being maintained while continuing to work, which is a much worse shape, because the thing that would prompt action is exactly what has been removed. A server that broke would get fixed. A server that is fine and quietly uninsurable gets a ticket in a backlog, and it stays there until somebody outside the organization sets a date.

Which somebody usually does. If the database is running on a managed service, the provider is on the hook for patching an engine that no longer receives patches, and no provider carries that indefinitely. They publish their own end-of-support date for the major version, some months after the upstream one, and when it arrives instances still on that version get upgraded on the provider's schedule rather than yours, in a maintenance window, to a version the provider chooses. That is the deadline that actually governs, and it is the one nobody has written on a plan.

The vendor clock is the one everybody quotes. Premier, then Extended, then Sustaining, on a published schedule years in advance. This is the date that appears in the compliance spreadsheet and it is genuinely useful, because it is announced long enough ahead to plan against. It is also the least urgent of the five, because Sustaining Support means the phone still works and an organization can talk itself into treating that as coverage for another two budget cycles.

The platform clock is the one that moves you. A managed provider set its own launch date for the version, some months after the upstream release, and it sets its own retirement date the same way. That date comes with an action attached rather than a status change: instances that have not moved get moved. Several providers have now added a paid extension past it, metered by the hour, which is worth reading as what it is - a price on the delay rather than a reprieve from it. Find the provider's dates, not the vendor's, and treat the gap between them as the entire planning window you actually have.

The client clock is the one nobody owns. Connectors, ORMs and drivers drop support for old servers on their own schedules, and they do it quietly, in a minor release, in a changelog. The bind is that the application's dependency updates and the database's version become coupled without anyone deciding they should be: a security patch to a driver arrives requiring a newer server, and now a database upgrade is on the critical path of an unrelated fix.

The audit clock is arbitrary and binding anyway. An unsupported database component is a finding in most frameworks, and a finding has a remediation date attached that bears no relationship to engineering capacity. This clock is the one people find infuriating and it is the one I have seen actually cause upgrades to happen, because it is the only one attached to a person whose job is to insist.

The mechanical upgrade is close to a solved problem and has been for years. On a managed service it is a console action, in place, with a documented rollback window. Self-managed, it is a package upgrade and a data dictionary rebuild. Either way the work is measured in a maintenance window, it is well documented, and thousands of people have done it. If the mechanics were the cost, this would be a scheduling problem and it would already be done.

The cost is that the new version does not behave like the old one. Not in the sense of bugs, but in the sense of deliberate, documented, correct changes to what the server does with input it used to accept and with queries it used to plan a particular way. Your application was written against the behavior, not against the manual, and the parts of the behavior it depends on were never enumerated because there was never a reason to enumerate them.

This is the estimating error that runs through every one of these projects. The plan is written against the mechanics because the mechanics are visible, documented and easy to put a number on. The actual work is a search problem over your own code and data, whose size is unknown until the search is under way, and which has no natural end other than running out of things to find. A project that takes an afternoon of engine work and six weeks of finding out what broke gets estimated at an afternoon, every time.

Which reframes what an upgrade project is. It is not an infrastructure change with an application impact. It is an application change that happens to be triggered by an infrastructure date, and the people who need to be assigned to it are the ones who know what the queries mean rather than the ones who know how to run the upgrade.

The character set was never what its name said. MySQL's utf8 is a three-byte encoding covering part of Unicode, and the manual now calls it a deprecated alias for utf8mb3. Full Unicode is utf8mb4, and 8.0 makes it the default. The conversion is not a settings change: it rewrites every affected table, it can move rows across the wire between application and server differently, and any place a length was measured in bytes rather than characters now measures something else.

And there is the index prefix limit that ambushes the conversion. InnoDB caps an index key prefix at 767 bytes under the older row formats and 3,072 bytes under the newer ones. A VARCHAR(255) indexed column under three-byte utf8 needs 765 bytes and fits. The same column under utf8mb4 needs 1,020 and does not. So a conversion that looks like a character set change turns into a row format change on some subset of tables, discovered one failed ALTER at a time, usually on the largest table you own.

GROUP BY stopped sorting, and nothing tells you. Through 5.7 a GROUP BY also sorted its output as a side effect of how grouping was implemented, and a great deal of application code was written against that accident without anyone knowing they had relied on it. MySQL 8.0 removed the implicit sort. The queries still run, still return the same rows, and return them in a different order, so what breaks is a report's row order, a paginated list, or a comparison against a stored fixture. There is no error and no warning anywhere in it.

Optimizer plan drift produces no errors at all. Newer versions merge derived tables that used to be materialized, cost differently, and in 8.0 add histograms and invisible indexes. Every one of those is an improvement on average and a regression somewhere specific, and the failure mode is the nastiest in the set: correct results, arriving slower, on a query nobody changed. There is no exception to catch and no log line to grep. It shows up as load, days after the upgrade, when the causal link is no longer obvious.

Replication format and topology constrain the order. The default binary log format became row-based in 5.7, and replication is only supported from an older source to a newer replica, never the reverse. Those two facts together dictate the sequence of a rolling upgrade and remove most of the clever ordering people propose. They also mean the rollback plan cannot be "fail back to the old replica," because once the new version has written to it, there is no supported path home.

Authentication, reserved words and a removed cache block cutover. MySQL 8.0 changes the default authentication plugin, so older connectors that cannot negotiate the new one fail at connect time rather than at query time. It adds reserved words, which turns an unquoted column named for a windowing function into a syntax error. And it removes the query cache outright, so a configuration file carrying query_cache_size from 2014 stops the server from starting at all. None of the three is hard. All are the kind of thing found at two in the morning on cutover night if nobody ran the application against the new version first.

There is no waystation on this one, which is what makes it different from the last of these projects most teams ran. From 5.6 you could stop at 5.7, take the cheap half of the behavior work and defer the rest. From 5.7 the supported step is 8.0 and there is nothing in between, so the entire set of changes arrives in a single project and the only variable left is when.

Which turns the decision into one about buying time, because the providers have started selling it. Where the upstream version is out of support and the platform has set its own retirement date, several now offer a paid extension past that date, metered by the hour on the instance. That is a real option and it is worth pricing honestly: it is not support in the sense of new upstream fixes for the engine, it is the provider carrying backported patches for a version its vendor has stopped maintaining, at a rate designed to be more expensive than moving.

I would use it only for something with a date attached to the other end. Buying a quarter to finish a migration already under way is sound. Buying a year because the migration has not been scoped is the same annual deferral that produced the position in the first place, now with an invoice, and the invoice is the only part of it that will get anyone's attention. If the extension is being renewed a second time, the extension has become the plan.

One more constraint that shapes the whole plan: MySQL 8.0 has no downgrade. The data dictionary changes on the way up and there is no supported route back, which means the rollback is a restore from backup and the recovery objective for the upgrade is whatever your restore time actually is, measured on a full-sized dataset, not the number in the runbook. Anyone who has not timed that recently is planning around a figure they have not verified.

Capture real queries and replay them, not a test suite. The behavior differences live in the queries your application actually sends, including the ones generated by an ORM that nobody has read. A capture of production traffic replayed against the new version finds the strict-mode failures, the reserved word collisions and the plan regressions in one pass, and it finds them in proportion to how often they matter. A test suite finds the ones somebody thought to write a test for, which is a different and much smaller set.

Get the slow query log before you have anything to compare it to. Plan drift can only be detected against a baseline, and the baseline has to be captured on the old version, at representative load, before anything moves. This costs an afternoon in advance and is impossible to reconstruct afterward. Without it, the argument three weeks after cutover about whether a report was always this slow has no evidence in it and will be settled by whoever is most confident.

Time the restore, on real data, before you need the number. Since there is no downgrade, the rollback is a restore, and the restore duration is the actual blast radius of a failed upgrade. Measure it on a full-sized copy rather than a sample, because the relationship is not linear and the number people quote is usually from a test dataset. If the measured figure is unacceptable, that is not a reason to skip the test. It is the finding, and it changes the plan.

Put the provider date in the plan, in writing. Every one of these projects I have seen run late did so because the date being planned against was the vendor's rather than the platform's, and the platform's was months earlier. Write down which date is binding, which clock it comes from, and what happens automatically if it passes. A forced upgrade in somebody else's maintenance window, to a version somebody else chose, is a strictly worse version of the project you were already going to do.

Staying on 5.7 was frequently the right call at the time, made by people who correctly judged that the upgrade had a large unknown cost and no user-visible benefit, and who had something in front of them with both. The failure is not the decision, which was defensible each year it was taken. The failure is that it was never converted into a dated plan, so a series of reasonable annual deferrals added up to a position nobody chose and nobody owns.

Database end of life is the rare deadline with no symptom attached. Almost everything else in operations announces itself: the disk fills, the certificate expires and the browser complains, the queue backs up and somebody notices. This one arrives as a paragraph in a support policy and changes nothing you can see, right up until the morning your provider upgrades you on its own schedule and you find out, in production, which of your queries were relying on the old behavior.