Google argues that its Shielded Email prototype demonstrates FHE is no longer just theoretical — encrypted lookups now complete in tens of milliseconds per query, close to plaintext speed. They credit the maturation of their open-source FHE transpiler and HEIR MLIR project for making the compilation from high-level ops to lattice-crypto primitives efficient enough to ship.
By submitting the Google post to HN where it reached 323 points, the submitter signals this is a genuine engineering milestone worth the developer community's attention. The high engagement (192 comments) reflects broad interest in FHE finally clearing the latency bar.
The editorial emphasizes that Google's deployment is explicitly scoped to private information retrieval and set-membership checks — not full transformer inference on ciphertext. The 1,000×–10,000× ciphertext blow-up and multiplicative-depth constraints mean this works only because the computation graph is small; 'Gemini running on ciphertext' remains out of reach.
Google's security blog dropped a detailed writeup on how it's using fully homomorphic encryption (FHE) in production — the long-promised technique that lets a server compute on data it can't read. The vehicle is a feature inside Google's Shielded Email prototype (part of the private-communications work shown at I/O), where an on-device client encrypts a query, the server does the matching against a private index, and the plaintext never leaves the phone.
The headline number: Google claims the encrypted lookup completes in tens of milliseconds per query, close enough to plaintext that users won't feel it. That is the first time an FHE deployment at Google's scale has cleared the latency bar that keeps this class of crypto out of shipping products. The stack is built on Google's open-source FHE transpiler and the HEIR (Homomorphic Encryption Intermediate Representation) MLIR project, which compiles high-level operations down to lattice-crypto primitives (CKKS/BGV/TFHE, depending on the workload).
The post is careful about scope. This is not "Gemini running on ciphertext." It's a narrow, structured operation — private information retrieval and set-membership checks — where the computation graph is small enough that the ciphertext blow-up (typically 1,000× to 10,000× over plaintext) stays inside a budget the network and CPU can absorb.
FHE has been the theoretical answer to "how do you do cloud AI without trusting the cloud" since Craig Gentry's 2009 thesis. The reason nobody ships it is arithmetic. Every multiplication on ciphertext adds noise; you either bootstrap (an expensive noise-reset that costs seconds per operation in naive implementations) or you design your circuit to fit inside a fixed multiplicative depth. Neither has historically played nicely with the trillion-multiply workloads inside a transformer.
What Google is quietly signaling here is that the FHE toolchain has matured enough that you can compile a real, useful primitive without a cryptographer in the room — that's the actual news. The HEIR compiler exposes FHE the way LLVM exposes SIMD: a backend target, not a research project. That changes who can build with it. A backend engineer with an ONNX graph and a threat model can now, in principle, produce an encrypted-inference endpoint without hand-tuning polynomial moduli.
Compare this to the other two paths the industry has taken toward "private AI." Confidential computing (Nvidia H100 Confidential Compute, AWS Nitro Enclaves, Apple's Private Cloud Compute) leans on hardware attestation — fast, general-purpose, but you're trusting Intel/AMD/Nvidia's SGX-descendant TEEs, and the sidechannel literature is not kind to that trust. Federated learning + differential privacy keeps data on-device but only works when the aggregate is what you want. FHE is the only one that gives you a mathematical, hardware-independent guarantee that the server saw ciphertext and nothing else. The tradeoff has always been cost, and cost is the thing that just moved.
HN commenters flagged the obvious limits fairly: this is a lookup, not a language model, and PIR-style workloads were always the low-hanging fruit for FHE. Fair. But the more interesting subthread pointed out that Google shipping a production HEIR-compiled binary is a forcing function for the rest of the ecosystem — Microsoft SEAL, OpenFHE, and Zama's Concrete now have a de-facto reference for what "production FHE" looks like at a hyperscaler, and the compiler passes will get copied.
If you're building anything that touches regulated data — health, finance, legal, government — FHE just went from "file this under crypto research" to "evaluate for the next threat-model review." The practical question isn't "can I run my LLM under FHE" (you can't, and won't for years); it's "which specific server-side operation on user data can I move behind an FHE boundary this quarter." Think: private set intersection for contact discovery, encrypted feature lookups, blocklist/allowlist checks, ML feature enrichment on tokenized inputs. All of those are within reach today.
Second-order effect: the compliance story changes. "The server mathematically cannot read the query" is a much cleaner story to tell an auditor or a DPO than "the enclave is attested and we rotate keys." Expect the FHE-based architectures to show up in RFPs from EU public sector and healthcare procurement inside 12 months, because they close the residual-risk column in a way TEEs never quite did after the Foreshadow/Downfall era.
Third, and this is the trap: don't confuse "private inference" with "private training." Training under FHE is still orders of magnitude away from being economic; anyone selling you a fully-encrypted fine-tune today is either running a research demo or misusing the acronym. The realistic near-term architecture is plaintext training on curated data + FHE-protected inference on user queries — which, notably, is exactly what Google shipped.
The interesting benchmark to watch isn't Google's latency number in isolation — it's whether HEIR and the open-source FHE stacks converge on a common IR the way ONNX unified model formats. If that happens, the next 18 months produce a small but real category of "encrypted-inference-as-a-service" offerings, and confidential-computing-only vendors have to add an FHE story or cede the highest-assurance tier. If it doesn't, this is a great Google demo that stays a Google demo. Either way, the era where "just trust the enclave" was the only shippable answer for private AI is ending.
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.