A 7MB embedding model that runs in your browser — no server, no API key

4 min read 1 source clear_take
├── "Browser-native embeddings unlock a large class of apps that don't need OpenAI-grade quality"
│  ├── top10.dev editorial (top10.dev) → read below

Argues that Ternlight's real value isn't beating OpenAI on MTEB but that fuzzy in-page search, autocomplete ranking, note clustering, and offline retrieval don't need frontier-quality embeddings. At 7MB and zero per-query cost, the model is 'good enough' for a huge slice of embedding workloads that currently pay for network round-trips to hosted APIs.

│  └── soycaporal (Hacker News, 140 pts) → read

The submitter framed Ternlight as a live, working demo — a real embedding model small enough to ship to a browser, fast enough to be interactive, with literally zero per-query cost. The pitch treats client-side inference as a first-class deployment target rather than a novelty.

├── "Ternary quantization is the key breakthrough that makes 7MB viable"
│  └── top10.dev editorial (top10.dev) → read below

Credits aggressive ternary weight quantization (values constrained to roughly {-1, 0, +1}) as the trick that compresses the model to a fraction of a float32 checkpoint. Combined with a WASM runtime that avoids needing WebGPU, this is what makes browser-side embedding practical at this size rather than merely possible.

├── "Browser-side embedding isn't new — the story is the size, not the capability"
│  └── @HN commenters (skeptical camp) (Hacker News) → view

Multiple commenters pointed out that running embeddings in the browser has been technically possible for a while via transformers.js and ONNX Runtime Web. Their position is that Ternlight is an incremental win on model size, not a new category — and they want to see MTEB scores before declaring it useful.

└── "The privacy and cost implications matter more than the raw benchmark numbers"
  └── top10.dev editorial (top10.dev) → read below

Emphasizes that every embedding API call is both a network hop and a data-sharing decision, and that moderately busy products can spend hundreds of dollars a month just turning strings into vectors. Client-side inference eliminates both concerns simultaneously, which is a structural win independent of quality benchmarks.

What happened

A project called Ternlight hit Hacker News with a live demo showing a 7MB embedding model running entirely in the browser via WebAssembly. No server round-trip, no API key, no fetch to OpenAI or Cohere — you type text, WASM computes the embedding locally, and the vector comes back in milliseconds. The demo page loads the model once, caches it, and then handles every subsequent query on the client.

The pitch is unusually blunt: a real embedding model, small enough to ship to a browser, fast enough to be interactive, cheap enough that per-query cost is literally zero. The trick is aggressive quantization — ternary weights (values constrained to roughly {-1, 0, +1}) compress the model to a fraction of the size of a full-precision float32 checkpoint, and the WASM runtime keeps inference in the browser sandbox rather than requiring WebGPU or a native binary.

The HN thread (score 140 at the time of writing) split into the predictable camps: people impressed by the size-to-quality ratio, people asking about MTEB scores, and people pointing out that browser-side embedding has been technically possible for a while via transformers.js and ONNX Runtime Web — just not at 7MB.

Why it matters

Embeddings have quietly become one of the most-called API endpoints in modern apps. Every RAG pipeline, every semantic-search feature, every "related content" widget hits an embeddings API on the way in. At OpenAI's `text-embedding-3-small` price of $0.02 per million tokens, a moderately busy product can rack up hundreds of dollars a month just turning strings into vectors. More importantly, every one of those calls is a network hop and a data-sharing decision.

Ternlight's real argument is not that its model beats OpenAI's on MTEB — it almost certainly doesn't — but that for a large class of applications, it doesn't need to. Fuzzy in-page search, autocomplete ranking, note clustering, offline document retrieval, browser-extension features — these workloads don't need frontier-quality embeddings. They need embeddings that are good enough, fast, and free. A 7MB model that ships with the page and runs client-side hits all three.

The technical tradeoff is worth naming plainly. Ternary quantization is lossy. Model quality typically drops noticeably versus the full-precision teacher, and cosine similarities become coarser. For nearest-neighbor lookups over a few thousand items, that's usually fine. For distinguishing subtle semantic nuance across a million-document corpus, it isn't. There's also the WASM ceiling: WebAssembly SIMD is fast, but it's not GPU-fast, so throughput on very long documents will lag a server-side inference stack.

What's genuinely new is the *deployment model*. Transformers.js has been able to run MiniLM-class models in the browser for a couple of years, but the checkpoints are 20-90MB and the cold-start hurts. 7MB is comfortably smaller than most JavaScript bundles shipped by a mid-sized SPA. It caches in the service worker, streams over a CDN, and stops being a design constraint. Once the model is smaller than your marketing site's hero image, "call an embeddings API" stops being the default choice.

Community reactions on the HN thread landed on a useful frame: this isn't a replacement for hosted embeddings in high-quality retrieval systems, it's a wedge into a category that mostly didn't have a good option before. Client-side semantic search on a documentation site. Extension-side deduplication of bookmarks. Local vector search over a Notion export. The kind of features that would never justify a per-query API bill but would be genuinely nice to have.

What this means for your stack

If you're building anything with semantic search over user-owned data — notes apps, personal-knowledge tools, privacy-forward search, offline-first PWAs — a browser-side embedding model changes your architecture diagram. You can push the vector index into IndexedDB, run queries against it locally, and never build a backend for it at all. That's not just a cost story; it's a compliance story. Health data, legal drafts, financial notes, anything covered by GDPR data-minimization guidance — if the embedding never leaves the device, the threat model shrinks dramatically.

For RAG prototypes, Ternlight (or something like it) is a serious way to skip the first infrastructure decision. Instead of picking a vector DB, provisioning an embeddings endpoint, and figuring out API-key rotation before you've validated the idea, you can ship a working prototype as a single static page. If the retrieval quality holds up, you graduate to a real backend. If it doesn't, you spent an afternoon instead of a sprint.

The honest caveat: measure before you migrate. Run your actual queries through the model and compare recall against your current baseline. Ternary-quantized embeddings will underperform frontier models on hard retrieval — code search across a monorepo, cross-lingual queries, subtle intent disambiguation. If your workload sits in that band, keep the server-side embeddings and use client-side models for the cheap tier only. Hybrid is fine.

Looking ahead

The direction of travel is clear: model sizes for utility-grade inference are collapsing faster than most infra teams have internalized, and the browser is quietly turning into a legitimate inference target. Ternlight is one data point, but the pattern — ternary or 4-bit quantization, WASM SIMD, service-worker caching — is going to keep pushing more of the AI stack onto the client. The interesting question isn't whether browser-side embeddings will get good enough for mainstream use. It's which product categories will look silly for still paying a per-query API bill in eighteen months.

Hacker News 305 pts 60 comments

Ternlight – 7 MB embedding model that runs in browser (WASM)

→ read on Hacker News

// share this

// get daily digest

Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.