Boykis argues that running local models on a MacBook with Apple Silicon has finally become genuinely productive — not just a novelty. She backs this with her actual stack (llama.cpp, MLX, Ollama, LM Studio running Qwen-3 and Llama-3.1) and reports latency and quality on real workloads like summarization, code review, and embeddings that previously required a paid API.
The editorial frames the shift as the closing of the gap between 'doing this for fun' and 'doing this because it actually works' for the median developer. It emphasizes that this is not about matching frontier GPT-4-class capability, but about local inference being good enough for daily practitioner use.
Argues that local inference has been technically possible since Llama 2, so the meaningful change is that llama.cpp, MLX, Ollama, and vLLM have caught up to consumer hardware. The runtime layer — not a new model release — is what turned a fiddly hobbyist setup into something usable by default.
Points out that M-series chips share memory between CPU and GPU on a single die, so a 64GB MacBook effectively has 64GB of VRAM-equivalent for model weights. This makes consumer Macs uniquely well-suited to running large local models compared to the Nvidia consumer GPU ladder where VRAM is the binding constraint.
The top comments are practitioners sharing concrete setups — 30B-parameter models on a 64GB M3 Max, Ollama as a drop-in OpenAI SDK replacement, quantization tradeoffs at 4-bit vs 8-bit — rather than the usual 'well actually' pushback. The unusual absence of skepticism in a 1,296-point thread signals the post is calibrated correctly.
Vicki Boykis — ML engineer, author of *What Are Embeddings?*, and one of the more measured voices in the field — published a post on June 15 with a deliberately flat title: *Running local models is good now*. It hit 1,296 on Hacker News within hours, which for a non-news, non-launch personal blog post is the signal. The argument is not that local models match GPT-4-class frontier capability; it's that the gap between 'I'm doing this for fun' and 'I'm doing this because it actually works' has finally closed for the median developer.
Boykis walks through her actual stack: a MacBook with Apple Silicon, `llama.cpp` for the inference engine, MLX for Apple-native acceleration, and runtimes like Ollama and LM Studio as the user-facing wrapper. She runs Qwen-3, Llama-3.1, and a rotating cast of fine-tunes against real workloads — summarization, code review, embedding generation — and reports latency and quality that two years ago required a paid API key and a network round trip.
The HN thread is unusually free of the 'well actually' energy these posts normally attract. The top comments are practitioners sharing their own setups: a developer running 30B-parameter models on a 64GB M3 Max, someone using Ollama as a drop-in for OpenAI's SDK in a side project, a thread on quantization tradeoffs at 4-bit vs 8-bit. The consensus is that the post is correctly calibrated.
The interesting question is not *whether* local inference works — that has been demonstrably true since at least Llama 2. The question is *what changed*, and the answer is not a model. The change is that the runtime layer — llama.cpp, MLX, Ollama, vLLM — finally caught up with the hardware that consumers actually own.
Apple Silicon is the unappreciated half of the story. The M-series chips share memory between CPU and GPU on a single die, which means a 64GB MacBook Pro has 64GB of addressable VRAM-equivalent for model weights. Compare that to the Nvidia consumer ladder: a 4090 caps at 24GB, a 5090 at 32GB. The 70B-parameter models that matter for serious work fit on the Mac and require multi-GPU rigs on the Nvidia side. For the first time, the laptop on your desk is a more capable inference box than the gaming PC under it — and Nvidia's roadmap doesn't have an obvious answer because the unified-memory architecture is structurally different, not just a matter of stacking more VRAM.
The runtime stack matters because models alone are not products. `llama.cpp` is Georgi Gerganov's solo-started C++ implementation that now has 70K+ stars and contributions from every major model lab; it's the reason GGUF quantization works as well as it does. MLX is Apple's answer to PyTorch for Apple Silicon, shipped quietly in late 2023 and now mature enough that the major model releases ship MLX-format weights on day one. Ollama wraps both into a `docker pull`-style developer UX where `ollama run llama3.1` is the entire setup. Three pieces of infrastructure, all built in the last 30 months, all open source. That's the story.
The HN comment thread surfaces a quieter point worth amplifying: the privacy delta. A model running on your laptop never sees a billing dashboard, a TOS update, a rate limit, or a sudden policy change about what your prompts can contain. For a non-trivial slice of work — anything touching client code under NDA, anything involving health or financial data, anything where you'd rather not have a third party log your reasoning — that's not a nice-to-have. It's the entire reason to bother.
Benchmarks need a caveat. Qwen-3 32B and Llama-3.1 70B are competitive with GPT-3.5 and approach GPT-4 on narrow tasks, but they trail Claude Sonnet 4.5 and GPT-5 on anything requiring deep reasoning or long-horizon tool use. If your work is agentic — multi-step planning, tool use, code generation against unfamiliar codebases — the frontier is still the frontier. Local is good for everything else, and 'everything else' is most of what most developers actually do.
For side projects and prototyping, the calculus has inverted. A year ago, the default was 'wire up the OpenAI SDK because local is a science fair'; now the default should be 'run Ollama, point your SDK at `localhost:11434`, and only reach for a hosted API when you hit a capability ceiling you can name'. The OpenAI Python SDK's `base_url` parameter is all you need to swap — your code doesn't change, your bill goes to zero, and your laptop fan gets a workout.
For production, the answer is less obvious but still shifting. Embedding generation, summarization, classification, and RAG retrieval re-ranking are all tasks where a quantized 7B-13B model running on a modest GPU box beats per-token API economics at any meaningful volume. The ops cost of running your own inference server has dropped — vLLM, TGI, and Ollama-server all give you OpenAI-compatible endpoints with minimal config. The argument for paying Anthropic or OpenAI per token for routine inference is now specifically about frontier capability or operational simplicity, not capability in general.
Hardware choices flow from this. If you're buying a developer laptop in 2026 and you do any AI work, the unified-memory MacBook is the answer that the market hasn't fully priced in yet — a 64GB M4 Max is roughly the same money as a 32GB Windows laptop with a discrete GPU, and the Mac will run models the Windows machine can't load. The Framework Desktop with AMD's Strix Halo (also unified memory, also 128GB max) is the first non-Apple machine that competes on this axis, and it's worth watching.
The next 12 months are about the runtime layer eating more of the stack: structured output, tool use, multi-model routing, and inference-time scaling all need to work locally for the 'good now' verdict to hold against frontier improvements. Watch llama.cpp's tool-calling support, MLX's distributed inference work, and Ollama's enterprise push — those three projects will determine whether 'local-first' becomes the default architecture for developer AI or stays a power-user choice. Boykis's post is best read as a marker: the threshold moved this year, and most developers haven't updated their mental model yet.
After having been a happy user of Qwen3.6-27B for a few weeks, due to being away from the hardware, I'm currently forced to use Claude Sonnet 4.6It is such a downgrade. I don't understand how that's even possible. The thing has so many strongly-held opinions I did not ever ask it for,
This is the kind of thing that Anthropic et al should be worried about. As it becomes easier and easier to run local models, the ceiling of what they'll be able to charge will get lower and lower. Not that nobody will be willing to pay $$$$$ per month, but a lot of people are going to multiply
[meta] I wonder why people have such wildly different bar for what is "good" agentic coding?In a way, it's absolutely amazing that we've went from "Playing 'Set a Timer' on Apple Music" intelligence to something that may pass the Turing Test, but in practical
Show us the resulting code of using them! :) I want to use local models, I have the hardware for it, but while trying them out as replacements for GPT 5.5 xhigh or Opus or other SOTA models, they aren't quite ready to be replaced yet, sadly. The quality and bumps they encounter just slows down
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
I don't know about good, I use a lot of local models and they're still pretty painful to run locallyYou have dense models (qwen 27b, gemma 31b) who are pretty smart, but pretty slowYou have MoE models (gemma 26b, qwen 35b, north mini code 30b) who are pretty fast, but make a lot of mistake