Ronacher runs the same agent harness against successive model generations with identical tool schemas and documents a consistent regression: newer models argue with schemas, invent optional parameters, skip required ones, and wrap tool calls in prose that breaks parsers. He argues the improvements in reasoning benchmarks have come at the direct cost of the disciplined, schema-compliant tool-use behavior that older models delivered for free.
The editorial argues that the agent-framework industry's assumption — that better models automatically produce better agents — is wrong. Model quality on reasoning benchmarks and tool-use reliability in production are decoupled dimensions, and labs are visibly optimizing the first at the expense of the second through RLHF traces that reward verbose chain-of-thought over disciplined schema compliance.
The 131-point front-page thread is full of engineers running LangChain, Mastra, custom harnesses, and the OpenAI Agents SDK all reporting the same pattern: a model version bump that improves eval scores but breaks production agent flows that had been stable for months. The consensus in the thread is not that new models are dumber, but that they require an entirely different prompting posture to reach the tool-use reliability that older models provided by default.
Armin Ronacher — the Flask/Jinja/Sentry author who has been quietly running one of the more honest AI-tooling shops on the web — published *Better Models: Worse Tools* on July 4. The claim is blunt and, if you've been shipping agents, uncomfortably familiar: the newer frontier models are measurably smarter on reasoning benchmarks and measurably worse at using tools.
Ronacher's evidence is not a leaderboard, it's a workshop. He runs the same agent harness against successive model generations from the same lab, holds the tool schemas constant, and watches the failure modes migrate. Older models return valid JSON, fill required arguments, and stop when told. Newer models argue with the schema, invent optional parameters, skip required ones, or wrap tool calls in prose the parser then chokes on. The models got better at thinking about the tools and worse at actually calling them.
The HN thread (131 points, top of the front page) is full of practitioners nodding along. People running LangChain, Mastra, custom harnesses, and the OpenAI Agents SDK all report the same shape of regression: a version bump that improves eval scores but breaks production flows that had been stable for months. Nobody is claiming the new models are dumber. They're claiming the new models require a completely different prompting posture to hit the same tool-use reliability the old ones gave you for free.
The agent-framework industry has spent two years telling you that model quality is the ceiling. Better model, better agent. Ronacher's post is the polite version of the counterargument: model quality and tool-use reliability are separate axes, and the labs are optimizing the first at the cost of the second.
The mechanism is not mysterious. Frontier post-training now leans heavily on RLHF traces where the model reasons out loud, second-guesses itself, and produces the kind of chain-of-thought that scores well on math and code benchmarks. That posture is directly at odds with the terse, obedient, "emit exactly this JSON and shut up" behavior that tool-calling requires. A model trained to explain its work will explain its work inside your tool call, and your parser will die.
This matters for three groups. First, anyone maintaining a production agent: your working prompt is version-locked to the model you tested against. Swap to the newer, cheaper, smarter model and you inherit a week of debugging malformed calls. Second, the framework maintainers: LangChain, LlamaIndex, and the OpenAI Agents SDK have all shipped abstractions that assume tool-calling is a reliable primitive. When the primitive regresses, the abstraction leaks in ways users can't diagnose. Third, and most quietly, the labs themselves: if the tool-use tax keeps growing, the pitch that agents are the next platform starts to erode. You cannot build a durable agent economy on top of a substrate that requires prompt rewrites every quarter.
Ronacher's own suggested fix is unfashionable: fewer tools, simpler schemas, more explicit instructions in the system prompt, and — critically — pin the model. He's not the only one saying this. Simon Willison has been quietly making the same argument for a year: the agent that works is the one with three tools and a boring model, not the one wired into MCP with forty capabilities and the latest Claude or GPT. The industry keeps building for a future where models handle complexity for you; the practitioners keep succeeding by keeping complexity out of the model's way.
There's a benchmark gap hiding underneath this. Tool-use evals like BFCL and Tau-bench are optimized for single-turn, well-formed calls. They don't measure what Ronacher measures: multi-turn stability, argument hallucination rates, how often the model refuses to stop, how often it inlines prose into a JSON field. Until those become standard, model cards will keep advertising tool-use gains that don't survive contact with a real harness.
If you run an agent in production, three things change starting now.
Pin your model explicitly and version your prompts alongside it. Treat a model version bump the way you'd treat a Postgres major version upgrade: staging, regression tests against your actual tool traces, a rollback plan. "Just point at the latest" is how you end up with a Monday-morning incident whose root cause is a silent post-training tweak at OpenAI or Anthropic.
Build a tool-use eval that mirrors your production surface. Ten to twenty real traces from your logs, replayed against candidate models with your real schemas, scored on "did it call the right tool with valid arguments in one shot." This is a weekend of work and it will save you every future migration. If you can't reproduce your production tool calls in a test harness, you don't have an agent — you have a very expensive Rube Goldberg machine.
Cut tools ruthlessly. Every optional parameter is a hallucination surface; every tool you "might use someday" is a distractor the model will pick when it shouldn't. Ronacher's implicit thesis is that the models that get smarter also get more creative, and creativity is exactly what you don't want when the correct behavior is "call `get_user(id=42)` and stop talking." Fewer tools, tighter enums, required-not-optional wherever possible.
The uncomfortable read on Ronacher's post is that this gets worse before it gets better. The labs are racing on reasoning, thinking-time, and agentic autonomy — three axes that all push the model toward more verbose, more argumentative behavior. Tool-calling reliability is a compliance metric, not a marketing one, and nothing in the current incentive structure rewards a lab for shipping a model that's 3% worse at math but 15% better at emitting clean JSON. Expect the tax to compound until either (a) a lab decides tool-use is the differentiator and ships a model tuned for it, or (b) the frameworks give up on native tool-calling and go back to constrained decoding and grammar-based output. Both are plausible. Neither is imminent. Pin your model.
This is easily solved with good error messages.Claude always gets the syntax wrong on my tool calls.So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool.The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It h
When building agent integration for my serverless backend https://saasufy.com/, I decided to not use MCP but to put curl commands inside skill markdown files instead: https://github.com/Saasufy/skillsThe curl command is extremely popular so models seem to be really
Once models get better, we could avoid paying for a cache read on edit or write calls, and have the model assume they succeeded and not interrupt the stream to get output. We can then just parse the output and once we encounter such a silent toolcall execute it. With high probability its correct (gl
There's a spectrum of possible explanation, from "this is a model training artifact which for now they correct via the harness" through to "this is deliberate, and creates a constantly moving target to make third-party harnesses less efficient for lock-in purposes".I'd
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
For the smaller local model I use (DS4 Flash) I just modified the tools to match the common error case of the model. If the mountain won’t come to Mohammed…