The reporter measured and quantified the amplification — 49KB on ext4, 110KB+ on btrfs for a single trivial log line — and framed it as a real-world problem for embedded Linux, cheap VPS instances, and flash-backed edge nodes. The implication is that journald's per-message fsync pattern and hash table/offset updates are the root cause and belong on the systemd project's fix list, not a downstream workaround.
Kernel developers in the thread argue btrfs's copy-on-write behavior is doing exactly what it's supposed to do — every fsync forces the CoW chain of b-tree nodes to be flushed. From this view, the amplification isn't a filesystem bug but the natural cost of asking a journaling/CoW filesystem to durably persist tiny, frequent, metadata-heavy writes.
Some systemd maintainers in the thread push back that persistent journaling with durability guarantees inherently costs disk writes, and users concerned about SSD wear or embedded flash have an existing escape hatch via Storage=volatile. Under this framing, the reported amplification is a configuration choice, not a defect requiring upstream code changes.
The editorial reframes the story by noting that write-amplification discussions have historically focused on the SSD's flash translation layer, ignoring a second amplification stack that happens in software before the FTL ever sees the data. When journald's 40-byte log line becomes a 49KB filesystem write and then gets multiplied again by the SSD's own 2-3x factor, the true NAND cost is orders of magnitude beyond the semantic payload.
A GitHub issue filed against systemd (#40262) has resurfaced a problem that sysadmins running embedded Linux, cheap VPS instances, and flash-backed edge nodes have quietly suffered for years: systemd-journald's write amplification is absurd. The reporter measured a single log line — the kind of trivial `logger hello` invocation you'd use to test a syslog pipeline — and clocked the resulting disk traffic at roughly 49KB on ext4 and 110KB or more on btrfs.
That's a four- to five-order-of-magnitude blow-up between the semantic payload (a few dozen bytes) and the bytes actually hitting the block device. The issue traces the amplification to journald's write path: it opens the journal file, appends the record, updates internal hash tables and offset arrays, then calls `fsync()` — and on many systems, does this per message or in very small batches. Each `fsync` forces the filesystem to flush not just the journal file's dirty blocks but the metadata, the filesystem journal, and — on btrfs — the copy-on-write chain of every b-tree node touched along the way.
The thread pulls in the usual crowd: kernel developers pointing out that btrfs's CoW semantics are working exactly as designed, filesystem folks noting that journald's access pattern is pathological for any journaling FS, and systemd maintainers weighing whether the fix belongs upstream in journald, downstream in distro defaults, or nowhere at all because "just use `Storage=volatile`."
Write amplification is not a new topic — SSD vendors have talked about it for a decade — but the framing has always been about the flash translation layer turning 4KB logical writes into larger physical ones. What this bug report highlights is a second, entirely software-side amplification stack that happens *before* the FTL ever sees the data. By the time journald's 40-byte log line becomes a 49KB write on ext4, the SSD's own amplification factor of ~2-3x pushes the real NAND cost well past 100KB per line. On btrfs, you're talking about a quarter megabyte of flash wear per `logger` call.
For a workstation, this is invisible. Modern NVMe drives are rated for hundreds of TBW and a chatty desktop generates maybe a few MB of logs per day. For a Raspberry Pi running a homelab on a cheap SD card, a fleet of edge devices logging telemetry to eMMC, or a low-cost VPS on shared cloud storage with IOPS quotas, this is the difference between hardware lasting five years and hardware lasting five months. Community reactions in the HN thread lean toward exasperation: multiple commenters report having discovered this the hard way by watching their SD cards die on OpenWrt-style deployments, and several point out that the standard workaround — `Storage=volatile` in `journald.conf`, which keeps logs in `/run` (tmpfs) — has been folk wisdom for years but is nowhere in the systemd getting-started docs.
The btrfs numbers deserve a moment of their own. A 2-3x amplification versus ext4 for the same workload isn't a btrfs bug — it's the CoW tax made visible. Every metadata update writes a new b-tree node rather than mutating in place, and journald's fsync-per-record pattern is essentially a worst-case generator for that design. If you've been on btrfs because you like snapshots and don't do much writing, fine. If you're on btrfs because your distro picked it and you happen to run a service that logs aggressively, you're paying for snapshots you may not use.
The deeper lesson is that observability infrastructure has a physical cost that grows non-linearly with your logging verbosity, and most teams have no visibility into it. `iostat` will show you the bytes hitting the device but won't attribute them back to journald versus your application's own writes. `iotop` gets closer but doesn't decompose the amplification chain. Unless you're specifically instrumenting for it, journald's overhead is invisible in every dashboard you have.
First, the quick wins. If you run journald on flash and don't need persistent logs across reboots, set `Storage=volatile` in `/etc/systemd/journald.conf` and be done with it — logs go to tmpfs, survive nothing, and cost nothing on disk. If you *do* need persistence, `Storage=persistent` with `SyncIntervalSec=` bumped from the default 5 minutes to something larger (say 30s-60s for a fleet you actively monitor, longer for edge devices) batches fsyncs and cuts amplification substantially. Setting `RateLimitBurst=` and `RateLimitIntervalSec=` clamps runaway services from taking your disk with them.
Second, know your filesystem. If you're running btrfs on flash and journald is persistent, the numbers above should make you reconsider — either move `/var/log/journal` to an ext4 partition, mount it with `nodatacow` (which disables CoW for that subvolume and undoes most of the amplification, at the cost of snapshot fidelity for logs), or move to volatile storage. XFS sits somewhere between ext4 and btrfs; ZFS is closer to btrfs in amplification character.
Third, if you're building or operating edge/embedded devices, treat journald's default configuration as actively hostile to your hardware budget and template a hardened `journald.conf` into your provisioning. The SD-card-death stories in the HN thread are not edge cases — they're the median outcome for anyone shipping stock systemd to flash. And for anyone running centralized logging (Loki, Vector, Fluent Bit), consider whether journald needs to persist at all on the node when you're already shipping structured logs elsewhere; the local journal becomes a redundant, expensive buffer.
The fix, if one lands, will probably not be dramatic — journald's sync semantics exist for a reason (crashes shouldn't lose the log line that would tell you *why* they happened), and any batching optimization has to defend that guarantee. More likely you'll see distros ship more aggressive defaults for `SyncIntervalSec` and better documentation of `Storage=volatile` as the recommended posture for flash-backed systems. The interesting outcome would be journald gaining awareness of the underlying filesystem — detecting btrfs and adjusting fsync cadence, or preferring `sync_file_range` over full `fsync` where the semantics allow. Until then, the workaround is the fix, and the fix is one config line most people don't know to write.
Something must have happened along the way, because this was not the original design intent of the database (emphasis mine):"""The native journal file format is inspired by classic log files as well as git repositories. It is designed in a way that log data is only attached at the end
journald is awful for many reasons, but what makes it worse is that everything running on your machine thinks it has any rights to dump all the logs it wants unprompted. Open a file picker and kio will decide it's a good idea to spam tens or hundreds of thousands of entries into it a day, listi
My hunch having looked at the journald code as an amateur is that this write amplification is coming from scattering, with a few possible sources:1. Writes try to compress away duplicate metadata at the application layer, which causes them to issue scattered writes when new metadata shows up.2. Inde
The cherry on the cake is that you practically cannot filter journald. The only option is limiting by severity (e.g. errors and higher) or switch to non persistent journald storage and forward to rsyslog and filter there.Am a bit vague on the details but sometimes a driver goes bezerk and starts log
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
I completely agree with one of the comments from there:> But the fundamental conclusion is: the design was wrong. It should not have used mmapped writes. pwrite would have been far better.It really does not make any sense to use memory-mapped files when writing logs.Not even pwrite makes sense, b