Tao: AI is solving math problems and breaking math as a discipline

What happened

On September 11, Terence Tao and a group of co-authors published "A Severe Misalignment of AI in Mathematics" on Tao's blog and at mathandai.org. The piece, which shot to the top of Hacker News with 952 points and mirrored on devblogs, is not a hedged think-piece. It is a direct claim from arguably the most respected working mathematician on Earth that the incentives of frontier AI labs are actively corroding the discipline he has spent his career in.

The letter's argument is compact. Over the last few months, LLMs have crossed a threshold where they can solve "major outstanding problems in many fields of mathematics." AI labs have noticed, and are pouring resources into famous open problems as benchmarks — Millennium-style trophies to hang on quarterly slide decks. Tao's claim is that those problems were never the point; they were lighthouses used to measure whether the community had developed new understanding, and a system that lands on the answer without producing understanding breaks the instrument, not the ocean.

The piece frames this as an alignment problem — not in the RLHF sense, but in the older, drier sense of goals-of-parties-diverging. AI companies want tractable, legible, headline-worthy wins. Mathematicians want ideas that propagate: talks, discussions, simplifications, a textbook chapter a graduate student can absorb in a semester, and, decades later, tools the whole population uses. Those two things used to look similar from the outside. They no longer do.

Why it matters

The pushback in the HN thread is worth reading in full, because the strongest counterarguments are the ones that expose what Tao is actually saying. User `gwd` compared the letter to 1990s hand-wringing about computers destroying chess — thirty years on, chess is more popular and players are stronger. `david-gpu` reached for Baudelaire complaining that photography would ruin painting. `tmhn2`, a working mathematician, pointed out that Mochizuki's abc conjecture proof already looked like an unreadable dump on the community, and AI just makes that pattern more common.

These are all fair analogies, and they all miss the specific mechanism Tao is describing. The complaint isn't that AI can do math — it's that a benchmark culture optimized for solving named problems will hollow out the process by which mathematicians decide which problems were worth solving in the first place. Commenter `jeremysalwen` put it more precisely than the letter itself: what AI has destroyed isn't mathematicians' ability to understand and share ideas, it's the yardstick they used to measure how much understanding had actually accumulated. Chess survived Deep Blue because chess is a game with a fixed rule set and a clear win condition. Research mathematics isn't; it's a social process that uses problems as tracer dye.

This matters far beyond math departments. Every field that AI labs decide to "solve" next will face the same structural pattern. Competitive programming benchmarks (Codeforces, IOI-style problems) already look eerily like the mathematics case: the problems were designed as teaching tools, then repurposed as evals, then optimized against directly by frontier models. The moment a benchmark becomes a marketing asset for a $100B lab, it stops being a benchmark and becomes a training objective, and Goodhart's Law does the rest.

For practitioners, the meta-lesson is the one every ML engineer has already learned the hard way on smaller stakes: your eval set decays the moment your team starts looking at it. Tao's letter is that same observation at civilizational scale, from someone with the standing to say it without being dismissed as a Luddite. `pks016` in the HN thread flags the second-order concern — that the ripple effect of the "AI solves X" narrative will reshape what young researchers even choose to work on, in the same way that "AI will replace junior devs" reshaped what CS undergrads chose to major in.

What this means for your stack

If you ship code with LLMs in the loop, treat this piece as a case study in what your own benchmarks are actually measuring. SWE-bench, HumanEval, MMLU, and the rest are already at the stage research math is entering: labs are training against them, scores keep climbing, and the correlation between score and "actually useful engineer" is fraying. If you're using benchmark leaderboards to pick your model provider, you're doing the developer-tooling equivalent of judging a mathematician by how many Millennium Problems they've solved. Build your own evals against your own codebase, keep them private, and rotate them.

The second implication is about what to trust from AI-generated code and proofs. Tao's community-review argument — that a proof matters because a human community can absorb, simplify, and teach it — has a direct analog in code review. A pull request that no human on your team understands well enough to maintain is a liability regardless of whether the tests pass. The industry spent the 2010s learning this about copy-pasted Stack Overflow answers; the 2020s version is the same lesson at 100x throughput. Reviewability isn't a nice-to-have gate on AI-generated PRs, it's the entire point of the gate.

Third: if you build developer tools, the market opportunity Tao's letter implicitly points at is verification and understanding, not generation. Lean, Coq, Rocq, and the formal-methods ecosystem look much more interesting after this piece than they did a year ago. A world where LLMs can propose proofs but can't be trusted without machine-checkable verification is a world where proof assistants become critical infrastructure. The same logic applies to type systems, property-based testing, and anything else that turns "the model said so" into "the compiler said so."

Looking ahead

The interesting question isn't whether Tao is right — on the narrow claim about benchmark culture, he pretty clearly is, and the HN counterarguments end up reinforcing his point more than refuting it. The interesting question is whether the labs will listen, and the honest answer is probably not, because the incentive structure that produced the misalignment hasn't changed. What will change is that fields adjacent to math — theoretical physics, formal verification, algorithm design, and eventually large swaths of software engineering — are about to run the same experiment, with the same actors, and now with a very public warning shot from someone who cannot be waved away. Keep the letter bookmarked; you'll be citing it in a code review within a year.