The Programming Language Rankings Disagree

Put the current lists side by side. The Stack Overflow Developer Survey, fielded in May 2023 among more than ninety thousand self-selected respondents, has JavaScript most used at 63.6 percent. GitHub's Octoverse 2023 also has JavaScript first, and reports TypeScript passing Java for the first time. RedMonk has JavaScript first as well. TIOBE, alone among them, does not.

Three out of four agreeing sounds like corroboration until you read how they are built. RedMonk says plainly that it extracts rankings from GitHub and from Stack Overflow and combines them. Octoverse is GitHub. The survey is Stack Overflow's own audience answering questions about itself. So the three-way agreement is two data sources counted three times, and the fourth instrument, the only one not drawing on either, is the one that dissents.

TIOBE dissents because it is not a usage measure at all. Its published definition is a formula: search for the string +"<language> programming" across twenty-five search engines weighted by traffic, normalize each engine's hits, average them. What that counts is how much text about a language exists on the indexed web. It is a real quantity, measured consistently for years, and it is not the quantity the headline implies.

Which is the whole difficulty in one line: there is no measurement of "most popular language", because nobody has ever said what would be counted. Four organizations each picked something countable, and each is now quoted as though it had settled the general question. The useful move is not to pick a winner among the four. It is to work out which of their disagreements is about the decision in front of you.

A survey measures the people who answer surveys. Stack Overflow's 2023 survey was fielded in May and drew over ninety thousand responses, which is a large sample of a population nobody defined: people who read Stack Overflow, saw the banner, and had twenty minutes to spend on it. That population is not developers. It is developers who still have questions, which skews toward whatever is new, whatever is unfamiliar, and whoever is early enough in a career to be looking things up in public.

And its percentages are shares of people, not of code. The used-language question takes multiple answers, so the figures describe how many respondents touched a language at all in the past year and cheerfully sum past 100. A language you write four lines of counts exactly as much as the one you live in. Admiration is a separate population again, computed only among current users, so the reported figure above 80 percent for Rust means four in five Rust users, not four in five developers.

TIOBE measures the supply of documentation, and confusion. Because the input is search hits, everything that produces text about a language lifts it: tutorials, migration guides, error messages people paste, and the sheer volume of blog posts explaining a thing that is hard to explain. A language whose documentation answers the question the first time generates fewer pages than one that does not. TIOBE also excludes markup and frameworks by rule, which is why HTML appears in the survey's most-used list and never in TIOBE's.

RedMonk is unusually honest about its own limits. The ranking correlates pull request activity on GitHub against question volume on Stack Overflow, and its January 2023 run states the caveat in the post: the intent is "not to offer a statistically valid representation of current usage" but to correlate discussion and usage as a read on future adoption. I would take that at face value. It is a leading indicator of attention, published as one, and quoted as a census.

Octoverse counts public repositories, which is the tell. The 2023 report measures activity in public repositories carrying an open source license, over a window running from 1 October 2022 to 30 September 2023. That is a precise, reproducible thing to count, and it is structurally blind to every line written inside a company and never pushed anywhere public. The blindness is not a flaw in the report. It is a flaw in reading the report as a statement about software rather than about open source.

RedMonk's most recent rankings cover January 2023 and were published on 16 May 2023, four months after the period they measure. As I write this in the first week of January 2024, they are still the current ones. Anybody quoting RedMonk today is quoting a picture of the language landscape taken a year ago, and there is nothing wrong with that until it appears next to a number from last quarter.

The others run on their own clocks. The survey was fielded in one month, May, and reflects what people were doing then. Octoverse closed its counting window at the end of September and published in November. So a slide that stacks four rankings into one table is comparing early 2023 with mid 2023 with a year that ended in the autumn, and presenting the result as a snapshot of the present.

For a language that is not moving, none of this matters. For one that is, the calendar alone can produce the disagreement. A language that gained ground through 2023 looks weakest in the oldest instrument and strongest in the newest, and a reader who assumes the four were taken on the same day will interpret an artifact of publishing schedules as a methodological dispute worth having.

Every usage signal in every one of these is public: a public repository, a public question, a public page on the indexed web. The software that runs payroll, adjudicates claims, dispatches trucks, settles trades and prints the statement that arrives in the mail is on none of those surfaces. It was written by people who were not blogging about it, for an audience of one company, and it will never generate a pull request anybody outside the building can count.

The instruments also skew heavily toward beginnings. A new repository, a new question, a new project's first commits: these are the moments that produce a countable event. The moments that make up most of an engineering career, reading something written six years ago and changing four lines of it without breaking the other nine thousand, produce nothing at all. Maintenance is silent by construction, and silence is what these instruments read as absence.

Ask the question the rankings imply they are answering and you find there is no instrument for it. Which language has the most lines currently under maintenance in production? Nobody knows. Nobody can know, because the answer lives in private version control across every company on earth, and the only people who could count it have no reason to and no way to agree on what a line is.

That gap has a shape worth naming. The most discussed language and the most written language are not the same, and the distance between them is roughly the distance between new work and existing work. Which also means a high signal is ambiguous in a way the chart cannot resolve: a language generating a great many questions is either being adopted quickly or confusing people badly, and the count is identical either way.

The hiring pool is local rather than global. A worldwide ranking says nothing about whether you can hire two people at your level, in your market or your time zone, within a quarter. That is a question with a real answer and a cheap way to get it: run the searches you would run if you were hiring tomorrow and read what comes back. A global top-five position is compatible with an empty local market, and an unranked language can have a deep one in a particular city.

The ecosystem means twelve libraries and not a language. You do not depend on a language's community, you depend on a specific driver, a specific HTTP client, a specific parser for the format your partner sends. The researchable facts are per library: date of the last release, number of people with commit rights, how long open issues sit before somebody answers. A language high on every list can hand you a critical dependency maintained by one person who has stopped replying.

The support horizon is a published date. The question behind "will this still be maintained" is usually a question about a runtime, and runtimes publish support dates. Long-term support windows, end-of-life calendars and the length of the security-fix tail are all documented, dated, and specific to a version rather than to a language. That is a fact you can put in a decision record. A position in a ranking is not, and it will not tell you when the patches stop.

None of the three appears in a top-ten list. Which is the awkward part. All three are researchable in an afternoon, and the ranking is researchable in ten seconds, so the ranking is what gets pasted into the architecture review. I have watched a chart do work in a room that no one present believed it could do, precisely because the alternative was three separate pieces of homework and the meeting was already running long. The instrument that wins an argument is rarely the best one available. It is the one somebody had time to fetch.

A large community is a real asset and it compounds. The question you are about to ask has been asked and answered. The odd binding you need exists and somebody keeps it current. The strange interaction between the runtime and your operating system has a bug report with a workaround in it. None of that shows up in a cost model, and all of it is the difference between an afternoon and a fortnight.

The reverse is a standing tax that architecture reviews consistently underweight, because it arrives in small installments rather than as a line item. Every integration is one you write yourself. Every hire is a retrain. Every upgrade is one you test alone, because nobody has hit the problem before you. None of these is large enough to argue about on the day, and together they are most of the cost of the choice.

So the rankings are not useless, they are just coarse. What they can honestly do is warn you that a candidate sits far outside the mainstream and that the tax above is coming. That is a first filter and nothing more, and it should never survive contact with the three specific questions, which will tell you what the ranking was gesturing at and will sometimes tell you the opposite.

I should be careful not to aim this at the people who publish the lists. Each of the four is a competent measurement of a well-defined question, each states its method where anyone can read it, and RedMonk goes out of its way to say what its numbers are not. The failure is downstream. A figure lifted out of its methodology and set on a slide next to three other figures from three other methodologies stops being a measurement and becomes a vibe with a decimal point.

What stays with me is which room these instruments are pointed at. They see the lit part: the public repository, the question asked in the open, the page somebody wrote for strangers. The system that has run the business since 2009, that four people understand, that has never produced a search query or a pull request anyone outside could count, is in the dark part, and it is where the hours go. No ranking has ever measured it, and the next one will not either.