GCC nested functions may finally lose their executable stack

5 min read 1 source explainer
├── "GCC's trampoline-based nested functions are a security liability that should be replaced"
│  └── Martin Uecker (uecker.codeberg.page) → read

Uecker frames GCC's runtime-generated stack trampolines as an anachronism incompatible with modern W^X hardening, hardware CFI (Intel CET, ARM BTI), and OS-level protections like SELinux and systemd's MemoryDenyWriteExecute. He presents an alternative indirect-calling technique specifically to eliminate the need for an executable stack, treating the current default as a security wart worth removing from the toolchain.

└── "Nested functions remain a useful GNU C extension worth preserving through better implementation"
  └── Martin Uecker (uecker.codeberg.page) → read

Rather than deprecating nested functions outright, Uecker invests effort in showing how they can be called indirectly without the executable-stack requirement. His technique implicitly argues that the extension has enough legitimate users — glibc, qsort_r callbacks, numerical and DSL code — that the right move is to fix the implementation, not abandon the feature.

What happened

On August 29, GCC contributor Martin Uecker — better known in C-standards circles for his work on `_BitInt` and provenance — posted a walk-through of a technique for calling GCC's nested functions indirectly without needing an executable stack. The write-up sits at `uecker.codeberg.page/2026-08-29.html` and drew immediate attention on Hacker News (currently 71 points), because it takes aim at one of the oldest security warts in the toolchain.

Nested functions are a GNU C extension: a function defined inside another function, closing over the enclosing frame's locals. They are not part of ISO C. The Linux kernel uses them sparingly, glibc uses them, `qsort_r` callbacks are a classic case, and a surprising amount of numerical and DSL-adjacent code leans on them. The problem is how GCC has always implemented them when the address is taken and passed to something like `qsort`: at runtime, GCC writes a tiny snippet of machine code — a *trampoline* — onto the current stack frame. That snippet loads the static chain pointer into a scratch register and jumps to the real function body. The address of the trampoline is what gets handed out as the function pointer.

For that to work, the page holding the stack must be executable. GCC signals this to the linker by emitting `.note.GNU-stack` with the executable bit set, and the loader honors it by marking `[stack]` `rwxp` instead of `rw-p`. On any modern hardened system, that is a red flag.

Why it matters

An executable stack is the substrate for the entire shellcode-injection playbook the industry has spent 25 years trying to stamp out. W^X ("write xor execute") — no page ever both writable and executable — is table stakes on OpenBSD, standard on hardened Linux configurations, enforced by SELinux and by systemd's `MemoryDenyWriteExecute=`, and increasingly backed by hardware: Intel CET's shadow stack and IBT, ARM BTI, Apple's hardened runtime. All of these assume nothing generates code on the stack at runtime. GCC nested functions have been the notable, embarrassing exception in the GNU stack for as long as most of us have been shipping C.

The fallout is not theoretical. Debian's `dpkg` linter has warned about `execstack` binaries for years; Fedora rejects them in package review; the kernel prints `process 'foo' started with executable stack` to dmesg when it loads one; and glibc 2.41 hardened the loader so that a shared library requesting an executable stack can now be *refused* rather than silently upgrading the mapping. That last change quietly broke a chunk of downstream code that took nested-function pointers, and the standard workaround — `-Wl,-z,noexecstack` plus praying nobody actually calls those pointers indirectly — is not a fix, it's a bet.

Uecker's approach sidesteps trampolines entirely by using function descriptors: a small read-only record containing the code address plus the static chain, with the caller cooperating to unpack both. This is not a new idea — IA-64 and 32-bit PowerPC have used descriptor-style function pointers forever, and GCC has an `-fno-trampolines` mode for exactly those targets — but pulling it off portably on x86-64 and AArch64 means the *caller* side has to know how to invoke a descriptor rather than jump to raw code. The write-up shows how to bridge that gap for the indirect-call case that hurts most in practice: passing a nested-function pointer to a library routine like `qsort` that was compiled without any of this awareness. The trick, in essence, is to arrange for the pointer handed to the library to still be a plain code address, but one that lives in a normal `r-x` page and derives its static chain from a side channel rather than from bytes patched into the stack.

The community reaction on HN split, predictably, along two lines. Systems programmers who have been burned by `execstack` warnings on their own libraries welcomed it. The other camp — mostly people who consider nested functions a mistake GCC should never have shipped — asked, not unreasonably, why we're rehabilitating an extension that Clang has always refused to implement and that the C committee is never going to standardize. Both sides are right; the answer is that nested functions exist in shipping code today, and "rewrite it in Clang-compatible C" is not a migration plan, it's a bug report with extra steps.

What this means for your stack

If you maintain a C codebase that uses nested functions, three things follow.

First, audit now. `readelf -lW your.so | grep GNU_STACK` will tell you whether your artifact requests `RWE`. If it does, and you don't know why, the answer is almost always "somewhere in this tree, someone took the address of a nested function." `grep -rn 'auto ' | grep '('` is a crude first pass; `-Wtrampolines` is a better one and will name the culprit at compile time. Turn on `-Wtrampolines` in CI today, whether or not Uecker's technique ever lands — you want to know before glibc's next hardening round breaks your load.

Second, if you're a library author whose public API takes function pointers (callbacks, comparators, event handlers), assume your callers will hand you nested-function pointers and design accordingly. The descriptor-based approach only works if the *call site* is aware; a plain `void (*)(void)` parameter in a stable ABI cannot be quietly upgraded. This is why the fix is a language-and-runtime problem, not something GCC can ship unilaterally without ABI implications.

Third, watch what happens to `-ftrampoline-impl=heap`, which GCC 14 introduced as a stopgap: it allocates the trampoline on a heap page rather than the stack, which lets you keep `noexecstack` at the cost of needing an `rwx` heap allocator (typically via `libgcc`'s `__gcc_nested_func_ptr_created`). It's a lateral move — you've traded an executable stack for an executable heap — but it is available *today*, works with existing binaries, and is probably the right interim answer for teams that need to ship before descriptor-based calling is upstream.

Looking ahead

The honest read is that this is a proof of concept, not a merged patch, and getting descriptor-style indirect calls into mainline GCC in a way that stays ABI-compatible with the entire GNU ecosystem is going to be a multi-year conversation. But the direction of travel is unambiguous: hardware and OS vendors are done tolerating writable-executable memory, glibc is done papering over it, and the last major toolchain feature that requires it is now being actively dismantled. If you're still shipping nested-function callbacks, the window for pretending this isn't your problem is closing — and the good news is that, for the first time, there's a plausible path where you don't have to give them up to close it.

Hacker News 74 pts 51 comments

Indirect Calling of Nested Functions on GCC Without Executable Stack

→ read on Hacker News

// share this

// get daily digest

Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.