Blog · Oct 14, 2026 · By Velocity11

AI felt 20% faster. It measured 19% slower. Then they measured again.

Illustration of a relaxed developer with his feet up, a large stopwatch above him split between a sleek blue half and a cracked red half — Illustration made with AI.

Last year, a research group called METR ran an experiment that made a lot of developers uncomfortable. Instead of asking whether AI makes coding faster, they measured it with a stopwatch. The answer was no - and the developers in the study refused to believe it. This year METR ran the experiment again with newer tools. The answer changed. Both results are worth understanding.

The first experiment, in plain words

METR found 16 experienced developers who work on large open-source projects - codebases they know well, with on average more than 22,000 stars and over a million lines of code. Nearly all had dozens to hundreds of hours of experience with AI tools. [1]

Each developer listed real tasks from their own projects: bug fixes, features, cleanups. 246 tasks in total. For each task, a coin flip decided whether they could use AI or not. The tools were the best available in early 2025, mostly Cursor Pro with Claude 3.7 Sonnet. Work counted as done only when the code was good enough to pass review, including style, tests and documentation. [1]

The numbers that surprised everyone

With AI, the developers took 19% longer to finish their tasks. Not faster. Slower. [1]

Here is the strange part. Before the study, the developers expected AI to speed them up by 24%. After living through the slowdown, they still believed AI had made them 20% faster. The feeling and the stopwatch pointed in opposite directions. [1]

Bar chart of METR numbers: developers expected AI to make them 24% faster, felt 20% faster after the study, and were measured 19% slower — METR, Jul 2025. Chart drawn by Velocity11 from the published numbers.

Why slower? The study points at a pattern. Real projects have rules that are hard to write down: documentation habits, test coverage, formatting, the way this team does things. AI does not know these rules. So the developer prompts, waits, reads what the AI wrote, and fixes it to fit the project. For work that must pass a careful review, that checking and correcting can eat the time the AI saves. [1]

Then they measured again

In early 2026 METR published a follow-up with newer tools: 57 developers, 143 codebases, more than 800 tasks. This time the numbers moved. Developers from the original group were about 18% faster with AI. Newly recruited developers were about 4% faster. [2]

Two honest caveats. The uncertainty ranges on these newer numbers are wide and include zero, so treat them as "pointing up", not "proven". And something bigger was happening: the experiment itself started to break. [2]

Bar chart comparing METR's two rounds: early 2025 measured 19% slower, late 2025 measured about 18% faster for the same developers and 4% faster for new developers — METR, Jul 2025 and Feb 2026. Chart drawn by Velocity11 from the published numbers.

The twist: people would not give up the AI

To measure AI's effect, you need people to sometimes work without it. METR found that 30 to 50% of developers skipped tasks rather than do them without AI - even though they were paid $50 an hour for work of their own choosing. One developer from the original study said: "I'd like to help provide updated data on this question but also I really like using AI!" [2]

This matters for the numbers. The tasks people refuse to do without AI are probably the tasks where AI helps most. So the follow-up likely understates the real speedup. METR's own conclusion: developers are probably more sped up by AI now than their early-2025 result suggested. [2]

Fair warnings

The first study had 16 developers, in one kind of work. The authors agree that is a small group. [1]
Both results are snapshots of fast-moving tools, about a year apart. METR says the early-2025 result no longer reflects the current impact of AI models on developer productivity. [2]
Feeling fast is still not being fast. In both rounds, developers' own estimates were unreliable. [1][2]

What to do with this

Time yourself once. Pick a task you do often. Do it with AI one week and without the next. Write down the minutes. Most people have never done this.
Use AI for starts, not finishes. First drafts, boilerplate, exploring an unfamiliar library - the evidence says AI helps most where you know the least.
Keep the checking habit. Every AI answer needs one verified claim before you trust it, exactly like we wrote in our post about thinking skills.

Try it this week

Take one task you normally hand to AI. Set a timer. Next time, do it yourself and set the timer again. Five minutes of honesty tells you more than a month of vibes.

We use AI to help write these posts, and we say so on every one: this post was written with AI help and reviewed by a person.

Has AI actually made you faster - and did you measure it? Tell us on Instagram @velocity11ai.

Sources

[1] METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", Jul 10, 2025. metr.org

[2] METR, "We are Changing our Developer Productivity Experiment Design", Feb 24, 2026. metr.org

The hero image was made with AI. The charts were drawn by Velocity11 from the published METR numbers. This post was written with AI help and reviewed by a person.