fork()+exec() is the 50-year-old hack Linux finally wants to retire

5 min read 1 source explainer
├── "fork()+exec() is structurally broken for modern large-memory workloads and the kernel needs first-class alternatives"
│  └── jwilk (submitter) / LWN article (Hacker News, 277 pts) → read

The LWN piece argues that fork()'s page-table cloning cost — roughly 200MB of structured data for a 100GB process — makes it absurd accounting when the child immediately calls execve(). It frames renewed interest in posix_spawn(), pidfd_spawn(), clone3() extensions, and io_uring spawn opcodes as overdue kernel-level fixes rather than glibc workarounds.

└── "The original Unix elegance argument no longer holds — fork()'s design assumptions were valid for kilobyte processes, not modern heaps"
  └── top10.dev editorial (top10.dev) → read below

The editorial contends that fork()'s 'perfect snapshot' elegance was reasonable when processes were measured in kilobytes but stopped being defensible once the JVM and large in-memory databases like Redis became mainstream. It points to Redis BGSAVE being a tail-latency landmine for a decade and 200-400ms fork latencies as concrete evidence the abstraction has outlived its original justification.

What happened

An LWN article making the rounds on Hacker News (277 points) revisits a discussion that has been simmering on the Linux kernel mailing list for years: fork() followed by exec() — the canonical way to launch a child process on Unix since 1972 — is structurally wrong for modern workloads, and the kernel community is finally treating that as a problem worth solving.

The specific trigger is renewed interest in `posix_spawn()` as a first-class kernel primitive rather than a glibc wrapper, plus work around `pidfd_spawn()`, `clone3()` extensions, and io_uring's `IORING_OP_WAITID` and process-spawn opcodes. The thread quotes maintainers acknowledging what application authors have known for a decade: when your parent process has a 50GB heap, asking the kernel to clone its page tables just so the child can immediately drop them in `execve()` is absurd accounting.

The original Unix justification — that fork() is elegant because the child inherits a perfect snapshot and can decide what to keep — was reasonable when processes were measured in kilobytes. It stopped being reasonable somewhere between the rise of the JVM and the moment Redis started recommending you disable transparent huge pages.

Why it matters

The cost of fork() is not where most developers think it is. The data pages are copy-on-write, so duplicating a 100GB resident set does not copy 100GB. What it does copy is the page table itself — and on x86-64 with 4KB pages, the page table for a 100GB process is roughly 200MB of structured data that the kernel has to walk and clone under a write lock. Benchmarks circulated on the LKML thread put fork() latency for large Redis instances in the 200-400ms range. For a database that bills itself as sub-millisecond, that is a catastrophe — and it is precisely why Redis's BGSAVE has been a tail-latency landmine for a decade.

The JVM has the same problem, dressed differently. Any Java process that calls `Runtime.exec()` to shell out to `git` or `convert` triggers a full fork of the JVM's address space. The OpenJDK community has shipped no fewer than three workarounds — the `jspawnhelper` binary, posix_spawn variants, and most recently `Process.Builder` paths that try to use vfork() or posix_spawn under the hood. None of them are clean, because glibc's posix_spawn() implementation, until very recently, just called clone() with `CLONE_VM | CLONE_VFORK` and then exec(), which inherits all the historical sharp edges of vfork() (you can't touch the stack, you can't allocate, signal handling is a minefield) without exposing them as such.

The kernel-side proposal is that posix_spawn() should be a real syscall that allocates a fresh address space from the start, runs the requested file actions (dup, close, chdir) in a controlled minimal context, and execs — never inheriting the parent's page tables at all. This is what Solaris and FreeBSD have moved toward. Windows, for all its faults, has had `CreateProcess()` as a single atomic call since NT 3.1, and the gap shows up in benchmarks: launching `cmd.exe` 1000 times on Windows is dramatically cheaper than launching `bash` 1000 times on Linux when the parent process is large.

The io_uring angle is where this gets genuinely interesting. Jens Axboe and others have been quietly adding process lifecycle operations to io_uring — async exec, async waitid, pidfd integration — which means a sufficiently advanced runtime could pipeline process spawning the same way it pipelines I/O. A build system that needs to launch 10,000 compiler invocations could submit them all to a ring and let the kernel handle the spawn-and-reap loop without a single fork() round trip. This is not hypothetical; it is in mainline as of 6.5+.

Community reaction on HN split predictably. The traditionalists argue fork() is fine, that you should just not have a giant parent process, that posix_spawn() already works. The pragmatists point at the actual production numbers: Postgres connection establishment, Python multiprocessing, Nix build sandboxes, every CI system that shells out a hundred times per job. The traditionalist position is easier to defend in a textbook than in a flame graph.

What this means for your stack

If you run a service with a large resident set that occasionally shells out, audit those code paths now. Java services that use `ProcessBuilder` on modern OpenJDK (17+) on modern glibc (2.34+) are mostly fine — the JVM picks posix_spawn() when it can. Python is the laggard: `subprocess.Popen` still uses fork()+exec() by default; you need to pass `close_fds=True` and ideally use `subprocess.run` with a small wrapper, or wait for the long-discussed `posix_spawn` default. Ruby and Node.js have similar histories worth checking against your runtime version.

Redis users: this is why `io-threads` and the on-disk persistence story keep getting reworked. If you are still using BGSAVE-based persistence on a large instance, the fork() cost is a known and unfixable tax until the kernel changes. AOF with `appendfsync everysec` has different tradeoffs but does not trigger the fork cliff.

For anyone writing process supervisors, init systems, or container runtimes — systemd, runc, containerd, your homegrown sidecar — the move to pidfd_spawn() is the modern path. pidfd gives you a race-free handle to the child that survives PID reuse, integrates with epoll, and can be combined with posix_spawn() to give you the atomic create-and-exec semantics that fork() never had. If your code still uses `kill(pid, 0)` to check liveness, you are writing 1990s code on a 2026 kernel.

Build systems are the sleeper case. Bazel, Buck2, and the modern wave of remote-execution-aware tools spend a non-trivial fraction of their runtime in process spawn overhead. Ninja's reputation for speed is partially explained by how aggressively it minimizes fork() calls. A future where these tools target io_uring's process opcodes could see meaningful real-world speedups on monorepo builds — single-digit percentages, but compounding.

Looking ahead

fork() is not going away. It is too deeply wired into Unix semantics, too many programs depend on the inheritance contract, and POSIX is forever. But the trajectory is clear: new code should use posix_spawn() or pidfd_spawn(), runtimes should default to them, and the days when launching a subprocess from a 50GB parent was an unavoidable 200ms hit are numbered. The fact that the kernel community is willing to entertain a first-class spawn syscall — something that would have been rejected as un-Unix a decade ago — is the real story. The 1970s called and they want their abstraction back.

Hacker News 338 pts 324 comments

Moving beyond fork() + exec()

→ read on Hacker News

// share this

// get daily digest

Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.