How effective altruism quietly captured AI safety — and your roadmap

What happened

The Economist's long read on effective altruism argues that a philosophy once mocked as earnest Oxford undergrads calculating QALYs over coffee is now one of the three or four most influential forces shaping how AI gets built, regulated, and deployed. EA didn't win the public argument; it won the staffing meeting.

The piece traces the arc: GiveWell and 80,000 Hours in the late 2000s, the pivot from global health to "longtermism" around 2015, the FTX implosion in 2022, and — critically for anyone shipping models — the quiet migration of EA-aligned researchers into the policy apparatus that now governs frontier AI. The UK AI Safety Institute, the US AI Safety Institute (now folded into CAISI at NIST), METR, Apollo Research, and the alignment teams at Anthropic, DeepMind, and OpenAI are all heavily staffed by people who came up through EA community-building programs, SERI MATS, or ARENA.

The numbers in the Economist piece are worth sitting with. Open Philanthropy — the primary EA grantmaker, backed by Dustin Moskovitz's Facebook fortune — has moved more than $900m into AI safety and related longtermist work. That's roughly ten times what the US National Science Foundation spends on AI safety research annually. The AI safety field, as a field, is largely an EA construct in terms of who funded it, who staffed it, and whose vocabulary ("deceptive alignment", "scalable oversight", "evals") it uses.

Why it matters

For working engineers this is not a culture-war sidebar. It is upstream of your actual job in three specific ways.

First, evals are becoming compliance. The EU AI Act's codes of practice for general-purpose models, the UK AISI's pre-deployment testing agreements with Anthropic and OpenAI, and the voluntary commitments the White House extracted from frontier labs in 2023 all converge on a shared toolkit: dangerous-capability evals, red-teaming protocols, and "responsible scaling policies" that gate training runs on capability thresholds. The people who wrote those eval frameworks were, overwhelmingly, EA-trained — and the frameworks reflect EA's priors about which risks matter most. Autonomous replication, bioweapon uplift, and cyber-offense evals get first-class status. Labor displacement, surveillance misuse, and concentration of power get second-tier treatment or none at all. If your product touches a frontier model, the audit surface you'll face is shaped by that prioritization.

Second, compute governance is the new export control. The October 2023 and 2024 BIS rules on advanced chips, the proposed KYC requirements for compute providers in the US executive order, and the ongoing debate over training-run reporting thresholds (10^25 FLOPs in the EU, 10^26 in the US) all originated as EA policy proposals circa 2021–2022. GovAI, the Centre for the Governance of AI, and the Center for Security and Emerging Technology at Georgetown — the three think tanks that incubated most of this — are either explicitly EA-adjacent or heavily EA-funded. If you work on infrastructure, this is why your cluster procurement now involves lawyers.

Third, the critique from the other side is also getting institutional traction, and it's worth taking seriously on its own terms. The FAccT community, DAIR (Timnit Gebru's institute), AI Now, and a growing bloc of academic researchers argue EA's framing crowds out present-day harms — biased hiring systems, generative content moderation, the labor conditions of data annotators — in favor of speculative future ones. Émile Torres and Timnit Gebru's "TESCREAL" framing has gone from blog post to peer-reviewed critique to being cited in European Parliament hearings in under three years. The Biden executive order on AI was a compromise document that tried to serve both camps; the Trump administration's January 2025 rescission and the subsequent AI Action Plan tilt firmly toward the "accelerate, deregulate" camp that neither EA nor FAccT fully owns. The result is a three-way policy argument in which EA is the incumbent, FAccT is the challenger, and the accelerationists are the wrecking ball.

The FTX collapse should have discredited EA. In practice it discredited Sam Bankman-Fried personally and left the institutional footprint — Open Philanthropy, 80,000 Hours, the alignment labs — basically intact. If anything, the movement is more operationally mature post-FTX because the people running it learned, in the hardest possible way, that charismatic founders are a single point of failure.

What this means for your stack

Concretely, three things to internalize.

Know which eval regime you're building against. If you ship on top of Anthropic's API, you're inside Anthropic's Responsible Scaling Policy, which uses ASL levels (ASL-2 today, ASL-3 triggers at specific capability thresholds). OpenAI has a Preparedness Framework with analogous tiers. Google DeepMind has a Frontier Safety Framework. These aren't marketing documents — they're the policies that will determine whether a given model is available to you at a given price next quarter. Read your provider's RSP the way you'd read their SLA, because for capability-gated features it is the SLA.

Expect capability-gated deprecation. The RSPs commit labs to pause or restrict deployment when evals cross thresholds. In practice this has already shown up as feature-flagged tool-use restrictions, agentic-mode rollouts limited by region, and the quiet disabling of certain fine-tuning paths for frontier models. If your product depends on a specific capability — long-horizon agents, code execution, biology-adjacent reasoning — assume the deployment surface is subject to policy, not just engineering, review.

Hire or contract for eval literacy. The job title "AI safety engineer" barely existed in 2022 and now commands total comp north of $400k at frontier labs. Even if you're not hiring one, someone on your team needs to be able to read a METR autonomy eval or an Apollo deception-evals paper and tell you what it means for your product. This is the new "someone needs to be able to read a SOC 2 report."

Looking ahead

The interesting question isn't whether EA keeps its grip — it will, for at least the next hyperscaler budget cycle, because the alternative institutional infrastructure doesn't exist yet. The interesting question is which EA ideas survive contact with industrial-scale deployment. Scalable oversight is already looking shaky as agents get more autonomous. Interpretability has delivered real wins (Anthropic's circuit-tracing work, DeepMind's SAEs) but hasn't yet produced a single pre-deployment check anyone would stake a product launch on. The next two years will test whether EA's intellectual bet — that you can engineer safety into frontier systems before you understand them — is a technical research program or a very expensive act of faith. Either way, the people writing the checks and the eval harnesses are the same people, and they'll be there when your next model ships.