Argues DeepSeek is not only pushing technical boundaries but also publishing detailed papers explaining how they achieved their gains — something American labs no longer do. Concludes that Chinese labs are now doing the most interesting work in AI, and notes the HN replies piled on in agreement rather than pushing back.
Notes that speculative decoding has existed since Google's 2022 paper and every serious lab runs some version internally. What's new is that DeepSeek shipped the full training framework, the architectural paper, the eval harness, AND pre-attached speculator weights — meaning anyone can clone the repo tonight and benchmark gains on their own workload by morning.
Submitted the DSpark paper to HN where it hit 751 points in under a day, framing the release as 'inference optimizations with 60-85% faster generation.' The framing emphasizes that this is a real efficiency unlock requiring no extra GPUs, validated by the rapid community response.
Explains the mechanism: a small draft model proposes N tokens, the large model verifies all N in a single parallel forward pass, and accepted prefixes are kept. When the draft is right you get N tokens for the cost of one large-model forward — making this a step change in sequential decoding rather than a marginal optimization.
DeepSeek released DeepSpec, billed as a "full-stack codebase for training and evaluating speculative decoding algorithms," alongside a paper (DSpark) and updated V4 weights with the speculator module baked in. The Hugging Face repos — `deepseek-ai/DeepSeek-V4-Flash-DSpark` and `deepseek-ai/DeepSeek-V4-Pro` — are live and drop-in compatible with the existing V4 inference path. The reported headline number: 60-85% faster generation depending on workload, no extra GPUs required.
Speculative decoding itself isn't new. Google published the original paper in late 2022, and every serious lab has been running some version of it internally for at least two years. The novelty isn't the technique — it's the package. DeepSeek shipped the production training framework, the paper explaining their architectural improvements to the draft model, the eval harness, *and* weights that already have the speculator attached. You can `git clone` tonight, point vLLM at the new HF repo, and measure the gain on your own workload by morning.
The HN thread hit 751 points in under a day. One comment (kamranjon, top of the thread): "DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains — something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now." The replies pile on rather than push back.
The technical mechanism is worth a paragraph because the throughput math explains why this is a step change rather than a marginal improvement. A small "draft" model proposes a sequence of N tokens; the large model verifies all N in a single parallel forward pass; you keep the longest accepted prefix and discard the rest. When the draft is right, you get N tokens for the cost of roughly one large-model forward. When it's wrong, you pay the verification cost and resume from the last correct token. The win is sequential decoding steps eliminated. The loss is draft-model compute spent on tokens you discard. Net: on most prompts the draft is right often enough that wall-clock latency drops 40-60% and throughput on batched inference roughly doubles.
What DSpark contributes is acceptance rate. The paper details a tighter coupling between the draft architecture and the main model's distribution, plus a training recipe that better matches draft outputs to verification statistics. The 60-85% number is workload-dependent — code generation lands at the top end (high token predictability), creative writing at the bottom (high entropy) — but the floor of the reported range is still meaningful for anything coding-heavy.
The openness asymmetry is the part worth naming out loud. OpenAI hasn't published inference research with enough detail to reproduce since the GPT-4 system card in March 2023, and that card was deliberately scrubbed of architecture details. Anthropic's interpretability papers are world-class but tell you nothing about how Claude is served. Google's last genuinely useful inference paper was the speculative decoding original in 2022 — three years ago, before any of the actual production optimizations they've shipped since. DeepSeek, meanwhile, has now published reproducible work on MoE routing (V2), GRPO (R1), FP8 training (V3), and now full-stack speculative decoding (V4). The frontier of openly-documented inference is no longer in San Francisco.
Another thread comment, from @piterrro: "I've been using DeepSeek v4 pro for a month now in Kilo Code and its great. Fast, reliable, large context window and cheap as… Did 1.5B tokens this month and cost me 40usd (majority cached, but still)." 1.5 billion tokens for $40 works out to ~$0.027 per million tokens with caching — an order of magnitude under frontier US API tier pricing for comparable capability. Even uncached, the rack rate sits in the same neighborhood as Haiku, not Sonnet or GPT-4.1.
One more commenter (Havoc) flagged the obvious: "Guessing the timing isn't accidental. Demonstrated openness vs harsh regulation." The H.R. export control debate is heating up in Washington again this quarter, and a 751-point HN thread celebrating Chinese open-source generosity is not the political optics the restrictionist camp wants. Whether DeepSeek is timing releases for legislative cycles or just shipping when ready, the effect is the same: every paper drop is a data point in the "export controls aren't working" column.
If you self-host inference, this is the rare "clone and measure tomorrow" win. DSpark slots into vLLM 0.6+ and SGLang via the standard speculator API. The 60-85% gain compounds with continuous batching — production workloads with consistent prompt prefixes (RAG, agent loops, code completion) will see the high end of the range. Start with the DSpark-Flash variant for latency-sensitive endpoints; reserve V4-Pro for accuracy-critical paths.
If you depend on closed-API providers, the practical near-term move is to re-benchmark. The DeepSeek V4 family at $0.027/M cached tokens makes a different category of product economically viable: agent loops that burn 100M+ tokens per session, long-running background research jobs, eval suites you run on every PR. Workloads you currently can't ship because the API spend doesn't pencil out — sit down and rebuild the spreadsheet with V4 numbers.
If you build evals or fine-tune, DeepSpec ships the *training* code for the draft model, not just inference. You can train domain-specific speculators on your own corpus — a draft model that's been fine-tuned on your codebase's idioms will hit higher acceptance rates than the generic shipped one. This is the kind of last-mile optimization that closed-API providers structurally can't offer.
Speculative decoding is now table stakes for any self-hosted LLM deployment in 2026; if your inference stack doesn't have it by Q3, you're leaving 50%+ throughput on the floor and your CFO will eventually notice. The more interesting question is what DeepSeek ships next — the V4 paper hints at multi-token prediction extensions and a tighter integration with their MoE routing, both of which would compound on top of DSpark. American labs have a window to publish something comparable. If they don't, the next generation of open-source inference infrastructure will be built almost entirely on Chinese research, which is a strategic outcome that nobody in 2022 would have predicted but few should be surprised by in 2026.
The hugging face models are already up and seem to be the original models with the speculative decoding module built in which is very cool:Flash: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSparkPro: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro
DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.
I’ve been using DeepSeek v4 pro for a month now in Kilo Code and its great. Fast, reliable, large context window and cheap as… Did 1,5B tokens this month and cost me 40usd (majority cached, but still).
Nice.Guessing the timing isn't accidental. Demonstrated openness vs harsh regulation
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.