The editorial argues that the entire AI industry feedback loop — labs, benchmarks, VCs, product teams — is weighted toward the narrow 10% of work tasks that happen to be legible to benchmark authors, who overwhelmingly measure what a Bay Area engineer does. This is framed as a structural sampling bias that misdirects capital and product development away from the 90% of actual US working hours.
Mercor's Labor Index Benchmark Map cross-references BLS O*NET task data against the public AI benchmark corpus and concludes that 90% of US working hours have no meaningful benchmark coverage at all. Their position is that the issue isn't models performing badly on these tasks — it's that nobody is even testing nursing coordination, construction supervision, paralegal review, retail management, or logistics operations.
By submitting the Mercor research under the headline framing '90% of US working hours are spent on tasks not covered well by AI benchmarks,' the submitter amplified the measurement-gap framing to 182 points, signaling community agreement that the absence of benchmarks — not model capability — is the newsworthy finding.
Mercor published a research piece this week called the Labor Index Benchmark Map, and the top-line number is the kind of thing that should make every AI product manager stop scrolling. The research team cross-referenced the Bureau of Labor Statistics' O*NET task database — the canonical breakdown of what Americans actually do at work, hour by hour — against the public corpus of AI evaluation benchmarks. The question: of the tasks that fill the US labor market, how many are even *measured* by the benchmarks the industry uses to decide whether models are getting smarter?
The answer, roughly: 10%. The remaining 90% of US working hours are spent on tasks that have no meaningful public benchmark coverage at all. Not 'models perform badly on them' — nobody is even testing.
Mercor's map goes further than a single headline number. It breaks tasks down by occupation cluster and shows where the coverage actually lands. Benchmarks cluster heavily around coding (SWE-bench, HumanEval, LiveCodeBench), standardized knowledge recall (MMLU, GPQA), math (MATH, AIME), and a narrow slice of agentic computer-use tasks (OSWorld, WebArena). The hours those benchmarks target are concentrated in a small number of occupations — software engineers, some analyst roles, a handful of research jobs. Everything else — nursing coordination, construction supervision, paralegal document review, retail management, operations work in healthcare and logistics — sits on a nearly blank patch of the map.
The interesting thing about this chart is not that AI is overhyped. It's that the industry's feedback loop is pointed at the wrong 10%. Model labs ship, benchmarks score, VCs allocate, product teams pick which workflows to build against — and every link in that chain is weighted toward the tasks that happen to be legible to a benchmark author. The tasks that are legible are, overwhelmingly, the tasks a Bay Area engineer does. That is a sampling bias, not a map of the economy.
The gap isn't that models can't do the other 90% — it's that we have no idea whether they can, because we haven't built the tests. And building those tests is genuinely hard in a way SWE-bench isn't. A pull request has a ground-truth: it merges or it doesn't, CI passes or it doesn't. A nurse coordinating a discharge, a construction PM resolving a schedule conflict, a paralegal deciding which exhibits matter — these are tasks with ambiguous success criteria, long context, physical-world dependencies, and private data. You cannot scrape them off GitHub.
This is why the labs that *are* pushing into the 90% — Anthropic's computer-use work, OpenAI's enterprise rollouts, the long tail of vertical AI startups selling into legal, insurance, and clinical ops — are increasingly building their own private evals and refusing to publish them. The public benchmark corpus has become a historical artifact: it's what was easy to measure in 2020-2023, scaled up. It is not a representative sample of labor in 2026.
The community response on HN was telling. The top-voted comments weren't 'this is wrong' — they were variations of 'yeah, I've been trying to deploy an agent into a logistics workflow and the hard part is nothing any benchmark tests.' One commenter who runs an ops team: *the model can draft the email, the hard part is knowing which of seventeen half-correct documents is the one you're supposed to reply from.* That is the 90%. Context assembly, not token generation, is where real work lives, and no leaderboard scores it.
If you're a practitioner shipping AI into a product, the actionable lesson is to stop treating leaderboard rank as roadmap signal. The frontier model that's three points ahead on SWE-bench may be dead-equal on your actual workflow, because your workflow isn't SWE-bench-shaped. The decision of which model to standardize on should be driven by a private eval set you built from your own task distribution — ideally scored by a rubric you wrote with the humans who do the job today. This is unglamorous work. It's also the only work that correlates with user-visible quality.
Second: if your product is targeting one of Mercor's uncovered occupation clusters, you have an informational advantage that compounds. Every month you run real users through your workflow, you accumulate task data that doesn't exist anywhere else. That's a moat you can't buy from a model vendor. The teams that will own vertical AI in healthcare, logistics, construction, and legal are not the ones with the best prompts — they're the ones with the deepest proprietary eval sets. Start building yours now, even if it feels premature.
Third, be skeptical of benchmark-driven marketing from vendors selling into the 90%. 'Our model scores N on GPQA' tells you nothing about whether it can handle a nursing-handoff workflow. Ask for domain-specific evals. If the vendor can't produce them, that's a signal about where their actual R&D is pointed.
The next frontier isn't a smarter model — it's a better map of what work actually looks like. The lab or startup that builds the equivalent of ImageNet for enterprise workflows will reset the industry's feedback loop, and the next generation of models will be shaped by whatever that map includes and excludes. Mercor's piece is the first serious attempt to show the shape of the gap. Expect every serious AI org to be building something like it internally within a year — and expect the public leaderboards to start looking increasingly quaint.
I think “covered well” is being used here as a synonym for “covered at all”.For example, the “social workers & care -> counsel parents with child rearing problems” category is considered “well covered” because there are 13 prompts.It’s self-evident that the spectrum of work carried out by a s
> Our findings suggest that 90% of American working hours are spent on tasks with little or no coverage from existing benchmarks. This makes it very hard to know how good AI models are at the work Americans actually do.> ...> But even after controlling for AI-exposure, O*NET tasks from occu
One thing I've been thinking about is Baumol's Cost Disease. It's usually applied in at a macro scale across an economy: when a job somewhere becomes much more efficient, it makes all other jobs comparatively more expensive.I've started to notice this on a micro scale as well. I
While I directionally agree with the conclusion, O-Net is not a good source for relevant occupational and task classifications.While it is continuously updated, it lacks many modern white-collar occupations, such as occupations in finance, tech, or consulting. And when it does carry an occupation, i
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
Perhaps a version of the Doorman Fallacy, ie. replacing a doorman with an automatic door seems like an obvious win, but it turns out the doorman was performing a number of other vital functions (customer experience, for example) which weren't considered.