The editorial argues that benchmarks like HumanEval and SWE-bench measure single completions, but daily coding is now a 40-200 round-trip tool-use loop driving Cursor, Cline, Aider, or Claude Code. Open-weights models match on paper but fall out of the tool-use loop on multi-file refactors, which is why nobody shipping non-trivial features claims they replace frontier closed-source agents.
Some respondents in the thread report clean swaps to Qwen 2.5 Coder 32B or DeepSeek-V3 fine-tunes running on an M3 Max or dual-3090 rigs, citing acceptable tok/s numbers. They frame local models as a viable replacement for their daily coding workflow.
The dominant pattern in the 424-comment thread is users who report a local-model setup, then quietly admit they fall back to Claude Sonnet for 'the harder stuff.' This caveat — and the fact that the harder stuff is increasingly all the stuff — is why the 950-point thread produced no confident 'yes' to full replacement.
The editorial notes this is the third time in eighteen months the same Ask HN question has hit the front page, but the meaning changed silently. Two years ago 'replacing Copilot' meant autocompleting a function; today it means driving a multi-file refactor through 60+ tool calls without losing coherence — a fundamentally different problem the open-weights ecosystem hasn't solved.
An Ask HN thread — "Has anyone replaced Claude/GPT with a local model for daily coding?" — climbed to 950 points without producing a single confident yes. The replies follow a now-familiar arc: someone reports a clean swap to Qwen 2.5 Coder 32B or a DeepSeek-V3 fine-tune on an M3 Max or a dual-3090 rig, posts a tok/s number, then quietly adds a caveat about reverting to Sonnet for "the harder stuff." The harder stuff is, increasingly, all of the stuff.
This is the third time in eighteen months the same question has surfaced near the top of HN. The interesting shift is that the goalposts moved without anyone announcing it. Two years ago the question meant "can a local model autocomplete a function as well as Copilot." Today it means "can a local model drive Cursor, Cline, Aider, or Claude Code through a multi-file refactor without falling out of the tool-use loop." Those are different problems with different bottlenecks, and the open-weights ecosystem is winning the first one and losing the second by a margin that isn't closing.
The specific models people cite — Qwen 2.5 Coder 32B-Instruct, DeepSeek-V3-Coder, the various Llama 3.3 70B finetunes, Mistral's Codestral 25.06 — all post HumanEval and SWE-bench Verified scores within striking distance of Claude Sonnet 3.7 and GPT-4.1. On paper, the gap is gone. In practice, nobody who has shipped a non-trivial feature with these models in the last quarter is claiming they replace the frontier closed-source agents.
The disconnect between benchmark parity and lived experience comes from a single architectural fact that the benchmark culture hasn't caught up to: daily coding with an LLM is no longer a sequence of completions. It's a tool-use loop with 40 to 200 round trips per task. Open Claude Code's logs after a medium refactor and count the `read_file`, `edit`, `bash`, and `search` calls. The median is around 60. Aider on a comparable task lands between 30 and 90. The model isn't writing code in one shot; it's running a planning loop, reading context, editing, testing, reading test output, editing again.
Every one of those round trips has a fixed cost: prompt assembly, tokenization, prefill, decode, structured-output validation, tool dispatch. On Anthropic and OpenAI infrastructure, prefix caching makes the second-through-Nth call cheap. Prompts that share a 50K-token system + project context only pay the prefill cost once per ~5 minutes. Local inference stacks — llama.cpp, vLLM, MLX, Ollama — either don't have production-grade prefix caching or have it gated behind specific model architectures and request patterns most users haven't configured. A Qwen 2.5 Coder 32B running at 15 tok/s on an M3 Max with no cache hit is fine for one completion and miserable across 60 sequential tool-use rounds.
The second invisible cost is structured output. Frontier coding agents lean hard on constrained JSON for tool calls. Anthropic ships a tightly-tuned guided-decoding path; open stacks rely on outlines, xgrammar, or hand-rolled regex constrainers. When the model has to emit a 4KB JSON tool call inside a `
The third issue is context. Frontier providers have moved to 200K-1M token effective windows with quality that doesn't collapse past 32K. The strongest open-weight coding models hold quality reasonably well to 64K, but the inference math gets painful: KV cache for a 32B model at 128K tokens runs north of 30GB before you've thought about batching. A single developer on a 24GB consumer GPU is doing aggressive quantization tricks just to fit, and the quality regression from Q4_K_M on long-context tool-use chains is real and measurable.
The practical advice that's emerging — and it shows up in the more honest HN replies — is to stop treating this as a binary swap. Local models have won the autocomplete and the one-shot-completion battle. They have not won the agent battle. Hybrid setups are now the only configuration that survives contact with a workday. A reasonable 2026 stack looks like: Codestral or Qwen 2.5 Coder running locally for IDE inline completion and quick edits where latency matters more than reasoning depth, then Sonnet, GPT-4.1, or whatever you trust this month for the agentic Cursor/Claude Code/Cline sessions where you're chaining 50+ tool calls. The economics work out: you save the per-token cost on the high-frequency low-stakes calls (where you'd otherwise burn $40-80/day on completion APIs) and pay the frontier providers only for the agentic rounds where they're genuinely better.
If you want to push closer to full-local, the configurations that actually work in mid-2026 share three features: dedicated inference hardware (an RTX 6000 Ada, an M3 Ultra with 192GB unified memory, or a dual-3090 rig with NVLink), vLLM or SGLang as the serving layer (not Ollama, which abstracts away the caching primitives you need), and a coding agent that's been configured for the lower tool-use reliability — Aider's `--no-auto-commits` mode, smaller diff windows, explicit per-step plans. People reporting success at this are mostly running Qwen 2.5 Coder 32B FP8 with vLLM's automatic prefix caching enabled and a context window deliberately capped at 32K. They're getting 60-80 tok/s sustained and the tool-use loop holds together. The hardware bill of materials is $6,000-$15,000 depending on configuration, which prices out individual hobbyists but is rounding error for a team that was about to commit to $200/month/seat for Claude.
The second practical answer: write your own thin agent. Most of the failure modes on local stacks come from agent frameworks built against frontier API semantics that local inference doesn't replicate. A 200-line Python loop that issues completions with explicit tool-call parsing, retries on grammar errors, and manages its own context window outperforms a stock Cline configuration pointed at the same local model.
The gap is not unclosable. Two things would close it within the next twelve months: production-grade prefix caching landing in mainstream open inference stacks (vLLM has it; llama.cpp and Ollama are catching up), and speculative decoding for structured tool-use output becoming standard. Both are engineering problems being actively worked, not research problems waiting on a breakthrough. The Ask HN question will keep getting asked, and somewhere between mid-2026 and early 2027, the honest answers will flip. Until then, the operative truth is the one the HN thread keeps circling without quite naming: the model isn't the bottleneck, the loop is.
Has anyone here fully swapped Claude/GPT for a local model as their main coding tool, not just for side experiments? If so, please share your setup and performance (e.g tok/s)
→ read on Hacker NewsTop 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.