Council Post: The AI Validation Bottleneck: Why 80% AI Adoption Has Not Made Software Delivery Faster
Steve Taplin is CEO & founder of Sonatafy Technology, a software consulting & engineering firm focused on delivery, quality & accountabilitygettyTwo numbers from the last eight months should be sitt...
Steve Taplin is CEO & founder of Sonatafy Technology, a software consulting & engineering firm focused on delivery, quality & accountability
getty
Two numbers from the last eight months should be sitting together on every CTO’s desk, because separately they are misleading.
The first: Roughly 80% of engineers now use AI agents daily or more often, up from 47.3% a year earlier, according to Temporal’s 2026 State of Development Report, a survey of 554 engineers and engineering leaders in the U.S. and UK. The median respondent has five agents deployed.
The second: Developer trust in the accuracy of AI output fell from 40% to 29% year over year in Stack Overflow’s most recent Developer Survey, even as tool usage climbed to 80%.
Adoption is accelerating. Confidence in the output is declining. Both are real, and the space between them is where a great deal of engineering budget is disappearing.
The Bottleneck Moved, And Most Orgs Did Not Notice
For 30 years, the scarce resource in software delivery was the ability to write code. Every process we built assumed it: sprint planning, story points, headcount models. In many organizations, that constraint is gone. What replaced it is the ability to determine whether the code that now exists is correct, appropriate to the architecture and safe to release.
Nobody planned for that. Review capacity did not expand when generation capacity did. It is still one senior engineer reading at roughly the same speed as in 2019. The Stack Overflow data puts a name on the failure mode. The single largest frustration developers reported, cited by 45%, was AI solutions that are “almost right but not quite.” Sixty-six percent said they are spending more time fixing that “almost-right” code.
The work did not disappear. It changed shape, moved downstream and stopped being counted.
The Measurement Problem Is Worse Than The Capacity Problem
This should concern anyone reporting engineering performance to a board.
METR ran a randomized controlled trial with 16 experienced open-source developers on 246 real issues in their own repositories. The developers forecast that AI would make them 24% faster. When permitted to use AI tools, they were 19% slower. Afterward, having just lived through the slowdown, they still estimated AI had made them about 20% faster.
The sample is small, and METR says so plainly. The direction still matters: a 40-point swing between perceived and actual performance among skilled practitioners on code they knew well. That is not a story about bad engineers. It is what happens when a feedback loop breaks.
The Temporal data shows the same split inside a single population. There, 85.5% trust agent output at least somewhat. In the same survey, 41.1% encounter agent-related issues daily or more often.
I call this the AI validation gap: the absence of evaluation, measurement or trustworthy feedback loops on AI initiatives. It usually gets discussed as a product problem (e.g., hallucinations reaching customers or model drift going undetected). The same gap has now opened inside the delivery system itself, and it is harder to see there because the output looks like ordinary code.
None Of This Is Actually New
In the Software Delivery Failure Index 2026, my own firm analyzed 195 long-form interviews with software leaders and found 48 that met our criteria for a documented delivery failure. Those 48 traced back to five recurring root causes. The largest single overspend in the set was $42 million.
AI did not invent a new way for software projects to fail. It removed the friction that used to slow the old ways down, faster than most organizations updated their instrumentation.
Four Things Worth Doing Before The Next Planning Cycle
1. Instrument Instability, Not Just Velocity: If you are tracking pull request throughput, deployment frequency or lines merged without tracking change failure rate, rework rate and time to restore, you have built a scoreboard that cannot report a loss. Add the second half.
2. Treat Review As A Capacity Constraint With A Number Attached: An engineering leader can usually state the headcount to the person. Ask that same leader how many pull requests their senior engineers can meaningfully review in a week, and the answer is often a shrug. That number exists whether or not anyone writes it down, and it is the actual ceiling on delivery.
3. Stop Accepting Self-Reported Productivity Gains As Evidence: The METR result makes this non-negotiable. Ask what changed in the delivery data, not how the team feels about the tooling. If the only evidence for a 30% gain is that people report a 30% gain, it is not evidence.
4. Name An Owner For Verification: Generation almost always has an owner, usually whoever ran the tool rollout. Verification frequently has none. It gets assumed into everyone’s job, which is the same as nobody’s job. Assign it, budget it and give that person the authority to slow a release.
The organizations getting real leverage from AI tooling are not the cautious ones. They are the ones who expanded review, testing and evaluation capacity at the same time they expanded generation capacity and can therefore tell you, with data, whether it worked. The rest have an unmeasured system running at a higher volume. That is not acceleration. It is the same delivery problem, arriving faster.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?