Semgrep's internal benchmarks, drawn from the actual production workloads they run through their Assistant product (vulnerability triage, taint-flow reasoning, secure-code review, exploitability classification), show GLM 5.2 winning three of four cyber sub-benchmarks against Claude. As a serious paying customer with real economic stakes, their data challenges the assumption that the Anthropic premium is justified for AppSec code reasoning.
Even in their own benchmark, Semgrep concedes Claude retains a lead on long-context multi-repo reasoning — the one task that genuinely exercises the 200K+ context window. This carves out a defensible niche for Anthropic's premium pricing on workloads where context length is the binding constraint rather than raw cyber-reasoning skill.
Semgrep emphasizes that GLM 5.2 runs at roughly one-tenth the per-token cost of Claude Sonnet on hosted inference, with permissively licensed open weights on top. For a vendor running inference at scale across customer codebases, that order-of-magnitude cost delta combined with comparable or better quality flips the build-vs-buy calculus toward open models.
Semgrep — the static-analysis company behind the eponymous OSS scanner and a paid AppSec platform — published internal benchmark results on Monday claiming that Zhipu AI's GLM 5.2 outperforms Anthropic's Claude family on their in-house cyber-reasoning evaluation suite. The post, cheekily titled *"we have mythos at home: GLM 5.2 beats Claude in our cyber benchmarks,"* hit 715 points on Hacker News within hours of publication. The "mythos at home" framing is the punchline: Anthropic markets Claude as the frontier model for code and security workloads, and Semgrep is one of the more sophisticated consumers of that capability.
The benchmark covers the tasks Semgrep actually pays for at inference time: vulnerability triage, taint-flow reasoning across multi-file changesets, secure-code review against a known-bad corpus, and exploitability classification for raw findings. These aren't toy CTF challenges. They're the production workloads Semgrep runs at scale through its Assistant product — which means the eval has real economic stakes, not just leaderboard bragging rights.
GLM 5.2 is the latest release from Zhipu AI (the Beijing-based lab spun out of Tsinghua University, now operating as Z.ai internationally). It ships with open weights under a permissive license, lands at roughly one-tenth the per-token cost of Claude Sonnet on hosted inference, and — per Semgrep's numbers — wins on three of the four cyber sub-benchmarks. Claude still leads on long-context multi-repo reasoning, which is the one task that genuinely requires the 200K+ window.
For most of 2024 and 2025, the working assumption in AppSec tooling was that you paid the Anthropic premium because Claude was meaningfully better at code reasoning, and the alternative open models couldn't close the gap on the long tail of weird, real-world vulnerabilities. Semgrep's data is the first credible public signal from a serious production user that this assumption no longer holds on a domain-specific axis.
The nuance the HN comment thread keeps missing: this is *not* a claim that GLM 5.2 is a better general-purpose coding model than Claude. It's a claim that on the narrow distribution of inputs Semgrep cares about — short, security-sensitive code spans where the model has to reason about untrusted data flow — the open-weights Chinese model is competitive or better. That's a much narrower claim, but it's also the one that actually matters if you're shipping a security product and watching your inference bill.
There's also a methodological wrinkle worth flagging. Semgrep built the eval, Semgrep ran the eval, and Semgrep stands to benefit commercially from any narrative that loosens Anthropic's grip on AppSec inference. The post is transparent about all of this, includes prompt templates, and offers to share the raw scoring rubric on request — which is more rigor than the typical "X beats Y" vendor benchmark — but the eval set itself is not public. Take the absolute numbers with appropriate skepticism; take the *direction* seriously. When a paying customer publicly benchmarks against their own vendor and announces they're switching workloads to a cheaper competitor, that's the signal, regardless of whether the margin is 3 points or 15.
The other thing the result quietly establishes is that the Chinese open-weights labs — Zhipu, DeepSeek, Qwen, Moonshot — are no longer a quarter behind the frontier. They're shipping models that win specific evals against the named frontier players, on workloads where the frontier players have spent years optimizing.
If you're building anything that runs LLM inference at volume against security-flavored code workloads — SAST augmentation, PR review bots, secret scanners with explanation layers, IaC linting with remediation — the cost-quality frontier just moved, and not in Anthropic's favor. The pragmatic move is not to rip out Claude wholesale, but to run a sub-eval on *your* distribution: pick 200 representative findings, run them through both models with identical prompts, score by hand. If the gap on your data looks like Semgrep's gap on theirs, the routing decision pays for itself in a quarter.
Architecture-wise, the right pattern is a routing layer that picks model per task — GLM 5.2 (or self-hosted Qwen) for the high-volume triage path, Claude reserved for the long-context, cross-repo, multi-hop reasoning where it still wins. Most teams running a single-model strategy in 2026 are leaving money on the table; the inference-cost delta between specialized open-weights models and frontier API pricing is now large enough that even modest routing logic pays back in weeks.
The geopolitical caveat is real and worth saying out loud: GLM weights come from a Chinese lab, and some buyers (defense, regulated finance, anyone with FedRAMP exposure) cannot use them at any quality differential. For those teams, this benchmark is academic. For everyone else — which is most of the industry — it's a procurement signal.
The broader pattern here is the slow death of the single-frontier-vendor strategy. We are entering the phase where "best model" is a function of task, latency budget, deployment constraints, and price — and no single lab is going to dominate every cell of that matrix. Semgrep's post is one data point. There will be more, from companies with less marketing incentive and tighter eval discipline. The teams that have already built model-agnostic routing infrastructure will adapt in an afternoon. The teams that hardcoded `client = Anthropic()` across the codebase will spend a quarter untangling it. Choose accordingly.
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.