Restartable Sequences: The Linux Trick That Kills Your Mutex

What happened

Justine Tunney published a deep technical writeup on restartable sequences (rseq) — a Linux kernel feature that has been quietly sitting in glibc since 2.35 (2022) and which almost no application developer uses. Her post, currently sitting at 192 points on Hacker News, lays out what rseq actually is, why it's faster than the lock-free alternatives most engineers reach for, and how to hand-roll a critical section in assembly that the kernel will atomically restart if the thread gets preempted mid-flight.

The mechanism is deceptively simple. A thread registers a small struct with the kernel via the `rseq(2)` syscall. Inside that struct lives the current CPU ID — updated by the kernel on every context switch — and a pointer to a `rseq_cs` descriptor that declares the start, end, and abort address of a critical section. When the scheduler preempts a thread inside that range, instead of resuming at the instruction pointer where it left off, the kernel redirects execution to the abort handler. The critical section either runs to completion on one CPU, or it never observably ran at all.

Tunney walks through a per-CPU counter implementation: read `__rseq_abi.cpu_id`, index into a per-CPU array, increment, store. No `lock` prefix. No cache-line ping-pong between cores. No memory barriers. If you get preempted, the kernel restarts you and you retry on whatever CPU you woke up on. The win shows up immediately in microbenchmarks — she reports per-CPU rseq counters running 3-5x faster than `std::atomic` increments under contention, and scaling linearly with core count instead of collapsing.

Why it matters

The lock-free community has spent two decades teaching developers to reach for `compare_exchange_strong`, hazard pointers, and RCU when they need concurrent data structures. Restartable sequences invert the premise: instead of designing algorithms that tolerate concurrent access, you guarantee there is no concurrent access in the first place. That's a categorical shift, not an incremental one. The expensive part of `lock cmpxchg` isn't the instruction — it's the cache coherence protocol the instruction triggers across every core that has touched the line. rseq sidesteps it entirely by partitioning state per-CPU.

This isn't theoretical. Google has been shipping rseq inside TCMalloc since 2020, and the per-CPU caches that replaced the old per-thread caches there were the largest single allocator-throughput improvement in a decade. Facebook's folly library uses it. The Linux kernel scheduler itself uses related primitives. What's notable about Tunney's post is that she's pulling the technique down from "things hyperscalers do" to "things you can put in your own runtime if you're willing to write 40 lines of assembly per architecture."

The HN thread surfaced the predictable objections. The portability story is grim: rseq is Linux-only, requires glibc 2.35+, and the critical section must be hand-written assembly because the compiler will happily reorder, spill, or inline calls that break the abort guarantee. One commenter put it bluntly: "rseq is what you reach for when you've already proven atomics are the bottleneck and you're willing to take on a maintenance burden in exchange for 2-5x." That's the honest framing. It is not a replacement for `std::atomic`. It is a replacement for the specific case where contention on a single cache line is your profile's hottest sample.

The other quiet implication is what rseq says about where systems performance work is actually moving. The lock-free era assumed cores were the scarce resource; the rseq era assumes cache coherence bandwidth is. On a 128-core EPYC or a 192-core Graviton4, a single contended atomic counter can throttle the whole machine. Per-CPU sharding stops being a nice-to-have and starts being the only design that scales. Rust's `crossbeam` already exposes per-CPU shards behind a portable API; Go's runtime does it internally for the scheduler. rseq is the underlying primitive that makes those shards actually free.

What this means for your stack

If you're writing application-level Go, Java, or Node, this is interesting trivia. Your runtime either already uses rseq under the hood (Go's allocator, increasingly), or your workload is nowhere near the regime where it would matter. Skip to the next story.

If you're writing a memory allocator, a per-CPU cache, a metrics aggregator, a scheduler, a lock-free queue, or anything else where you've stared at `perf top` and seen a single atomic operation eating 30% of cycles — this is the technique you should evaluate before adding another sharding layer. The migration path is incremental: identify the one hot counter, replace it with a per-CPU rseq sequence behind a feature flag, fall back to atomics on non-Linux or older glibc. The work is in the abort handler and the cross-arch assembly; the win is the next two years of capacity planning you don't have to do.

For library authors the calculus is different. Exposing rseq directly is a footgun — one developer copying the assembly wrong gets silent data corruption that survives every test on a quiet machine and only manifests under production load. The right abstraction is what Google did with TCMalloc: bury it under a `PerCpu` template that hands users a normal-looking API and handles the assembly internally. If you maintain a runtime, an allocator, or a database, the cost of building that abstraction once is amortized across every user forever.

Looking ahead

The interesting question isn't whether rseq is faster — Tunney's numbers and Google's production deployment have settled that. The question is whether the next generation of systems languages will hide it well enough that application developers benefit without having to learn assembly. Rust is the closest, with `std::thread::current()` already exposing affinity hints and several crates wrapping rseq experimentally. Zig's incremental compiler and the new wave of high-performance runtimes will face the same choice. Expect rseq-backed primitives to quietly appear in standard libraries over the next 24 months, the same way SIMD intrinsics did — invisible to most, transformative for the people writing the layer underneath.