Aphyr: 'The Future of Everything Is Lies' — Trust Is the Real Casualty

5 min read 1 source clear_take
├── "Fabrication is now a baseline property of the medium, not a model defect"
│  └── Kyle Kingsbury (Aphyr) (aphyr.com) → read

Kingsbury argues that calling AI output 'hallucination' misframes the problem as a fixable bug. In reality, every text artifact — emails, commits, CVE reports, kernel patches — now arrives with a non-zero probability of being entirely invented, and this is a structural property of the new medium rather than a defect to be patched out.

├── "The trust crisis is economic: production cost has collapsed and our filters assumed it wouldn't"
│  └── Kyle Kingsbury (Aphyr) (aphyr.com) → read

Kingsbury frames the phase change as fundamentally an economics problem: reputation systems, peer review, journalism, and code review were all built on the assumption that producing plausible content was expensive. With that cost now near zero, every gatekeeping institution we rely on is structurally mismatched to the new reality.

├── "Detection-based defenses (watermarking, classifiers, C2PA) are a losing game"
│  └── Kyle Kingsbury (Aphyr) (aphyr.com) → read

Kingsbury rejects the technical community's reflex to treat this as a detection problem solvable with better classifiers, watermarking, or provenance metadata. The cost asymmetry between generating and verifying content means defenders will always be outpaced, and any solution rooted in detecting fakes is structurally doomed.

└── "Maintainers and reviewers are already drowning — the symptoms are here, not theoretical"
  └── top10.dev editorial (top10.dev) → read below

The synthesis points to concrete present-day evidence: Daniel Stenberg threatening to ban LLM-using submitters from Curl, the Linux kernel tightening patch review, and maintainers/recruiters/teachers all describing the same flood. Aphyr's essay is significant precisely because it names the underlying phase change driving these scattered local symptoms.

What happened

Kyle Kingsbury — better known as Aphyr, the engineer behind Jepsen's distributed-systems torture tests — published *The Future of Everything Is Lies, I Guess: Where Do We Go from Here?* The essay, which hit 244 points on Hacker News in its first day, is not another hand-wringing piece about AI slop. It's a structural argument: the cost of producing plausible-looking text, code, citations, screenshots, and audio has collapsed to roughly zero, and the systems we rely on to filter signal from noise — search engines, code review, peer review, journalism, even our own pattern-matching — were not designed for that economics.

Kingsbury's framing is characteristically blunt. He argues that 'hallucination' is the wrong word because it implies a defect in the model; in reality, fabrication is now a baseline property of the medium itself. Every email, every commit message, every Stack Overflow answer, every Linux kernel patch, every Jepsen-style bug report now arrives with a non-zero probability of being entirely invented. The defenses we built — reputation, institutional gatekeeping, the friction of writing — assumed production cost was the bottleneck. It isn't anymore.

The piece lands at a specific moment. The Hacker News thread is full of practitioners describing the same week: maintainers drowning in AI-generated CVE reports, recruiters drowning in AI resumes, reviewers drowning in AI PRs, teachers drowning in AI essays. The Curl project's Daniel Stenberg has publicly threatened to ban submitters who use LLMs for security reports. The Linux kernel has tightened patch review. Aphyr's essay is the first widely-shared attempt to name the underlying phase change rather than the local symptoms.

Why it matters

The technical community's first instinct has been to treat this as a detection problem — better classifiers, watermarking, provenance signatures, C2PA metadata. Aphyr's central claim is that detection is a losing game because the cost asymmetry is unbounded: a generator costs cents, a verifier costs human attention, and human attention does not scale. He's not the first to point this out (Cory Doctorow has been on the same drum for two years), but coming from someone whose career is built on adversarial verification of distributed systems, the diagnosis carries weight.

Compare the failure modes. A traditional spam filter assumes the attacker has limited budget and the defender has structured features (headers, IPs, reputation). An LLM-generated phishing email has none of those tells, and the attacker's marginal cost is sub-penny. A traditional code review assumes the submitter has invested hours and has a reputation to protect. An LLM-generated PR has neither constraint. The numbers from the field bear this out: the curl project reports that effectively 100% of AI-assisted security submissions in 2024-2025 have been false positives, and the time cost of triage now exceeds the value of legitimate reports from the same channel.

The community reaction in the HN thread splits cleanly. One camp — including several long-time security researchers — agrees with Aphyr that the medium is broken and we need new primitives: signed provenance, web-of-trust, paid verification, smaller and slower channels. The other camp, mostly practitioners building with these tools, argues that the equilibrium will rebalance the way it did for Photoshop, deepfakes, and CGI — uncomfortable for a decade, then absorbed. The disagreement isn't really about the technology; it's about whether trust infrastructure is something you can rebuild on the fly or something that takes generations.

Aphyr is closer to the pessimist camp but stops short of nihilism. His suggestion — and it's deliberately partial — is that we stop trying to verify content and start verifying *people and pipelines*. Cryptographic signing of commits and reports, in-person or video-verified contributor onboarding for sensitive projects, paid bug bounties that require KYC, smaller and more curated information channels. None of this is novel; what's novel is the argument that these are no longer optional hardening, they're table stakes for any project that wants to remain functional.

What this means for your stack

If you maintain anything with a public contribution surface, the prevalence numbers force a decision this quarter, not next year. Assume that anonymous bug reports, security disclosures, and drive-by PRs are now majority-fabricated, and design your intake accordingly. That means rate-limiting unverified submitters, requiring reproducible test cases before human review, auto-rejecting reports that cite non-existent CVEs or functions, and — uncomfortably — being willing to ban repeat offenders even when individual submissions are ambiguous.

If you ship code that depends on LLM-generated artifacts — and most teams now do, whether they admit it or not — the verification burden has shifted onto you. Treat model output the way you'd treat a contractor you've never met: useful, plausible, and to be reviewed line-by-line for anything touching auth, money, or data integrity. Static analysis, fuzzing, and property-based tests are no longer 'nice to have' for the AI-augmented pipeline; they're the only layer that doesn't have a financial incentive to lie to you. The teams that will look smart in two years are the ones investing in verification infrastructure now, not the ones racing to wire more agents into production.

For information consumption, the implication is harder. Stack Overflow answers, blog tutorials, documentation aggregators, and search results are all degrading in real time. The practical move — which Aphyr endorses obliquely — is to anchor on a small set of trusted primary sources (official docs, signed releases, named individuals you've verified exist) and treat everything else as suspect until corroborated. RSS, mailing lists, and IRC are having a quiet renaissance for exactly this reason.

Looking ahead

The essay's title ends with a question, and Aphyr doesn't pretend to answer it. What he does is something the industry has been avoiding: he names the problem as structural rather than incidental. The next eighteen months will tell us whether the response is meaningful — signed provenance becoming default in package registries, verified-contributor requirements in critical OSS, real legal liability for fabricated security reports — or whether we just normalize a noisier, less trustworthy substrate and call it progress. Either outcome is consequential. Pretending the choice isn't being made is the only option that's clearly wrong.

Hacker News 698 pts 729 comments

The Future of Everything Is Lies, I Guess: Where Do We Go from Here?

→ read on Hacker News

// share this

// get daily digest

Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.