Sync policy

crates/vike-core/src/journal/sync.rs opens with the sentence the rest of the journal is built around: every blocking msync this journal performs happens in that file, on a thread the appender never joins. The reason is where the appender runs. CommandJournal is appended to from the execution core's fold thread, the one the latency gate holds to a p99 < 10µs core hop, and an msync costs time proportional to dirty bytes — measured at about 8.6 ms for 1 MB, 100 ms for 16 MB, and 292 ms (max 432 ms) for a dirty 64 MB segment.

Three stalls that used to be inline

The parent module's header lists them, each found by measurement rather than by reading. The per-chunk sync in write_framed put a single stall of about 141 ms into the measured hop, at hop 58 518 of 100 000. roll's opening whole-segment flush was the 292 ms one. And the checkpoint flush ending every cadence Snap — CoreThread::write_snap calling CommandJournal::flush — survived the first fix, because the latency harness pinned snapshot_every to never fire while the production default is 1024. Once looked for: the journal-snap variant's p99.9 was 153.9 µs against 38.6 µs, its maximum 14.1 ms against 77.7 µs, with 102 hops over 100 µs — one per snapshot, the worst around 1 400 times the budget. All three now go through CommandJournal::queue_sync, and that variant stays a gate so the call cannot come back unnoticed.

A watermark, not a queue

The appender's hand-off is Syncer::request: store the target byte offset into SyncShared::target, swap a wake_pending flag, and send one Wake if no wake is already queued. That is the entirety of what a chunk boundary costs the fold — one atomic store, one atomic swap, at most one channel send; never an msync, never a wait. The design holds because the work is a watermark: target is a monotonic offset, the syncer always reads the newest value when it gets round to it, so a wake that coalesces with an earlier one drops no work. The backlog therefore cannot grow no matter how far behind the disk falls, which is why the appender never has to block to apply back-pressure. The channel is unbounded and carries at most one un-consumed wake plus one message per segment roll.

How much is owed at once

sync_chunk_bytes bounds the un-synced debt at one sixteenth of the segment (SYNC_CHUNK_DIVISOR), floored at 4 MiB (SYNC_CHUNK_MIN) — 4 MiB on the production 64 MiB segment. It is a fraction for a measured reason: a fixed 4 MiB made every journal pay, including ones that never roll and so have no seal to bound, and while the sync was inline it put about three forced syncs of roughly 30 ms each into the gate, at a hop index that was exactly the chunk boundary. A fraction keeps a constant sixteen syncs per segment whatever the size, and a segment smaller than the floor never forces a sync at all. Deferring until a segment is nearly full does not remove the work, it concentrates it into a burst the syncer can fall behind on. The checkpoint reuses the same watermark — "sync up to the snap" and "sync up to this boundary" are one statement about a different offset — and at the production defaults it fires about fourteen times more often than the chunk (1024 records at roughly 287 B is about 287 KB): smaller, more frequent msyncs, and a syncer that falls behind coalesces the extra wakes.

The thread's three jobs

run_syncer is panic-free by construction; an msync failure is logged and retried on the next wake, never unwrapped, and the region stays owed. On a Wake it warms the page-table window ahead of the appender's cursor, flushes the owed range, then warms again aimed at wherever the appender is now — the before-flush warm is the half that carries the fold thread through a 36–141 ms msync, and warm_from takes the maximum of appender and watermark so a clamped target can never aim the window backwards. Segments carries that measurement. On a Roll it seals the segment it holds with one whole-segment flush, adopts the new mapping and warms it; Syncer::hand_over rebases target to the header offset before sending, so a stale old-segment watermark can never mark fresh bytes as synced. When the channel closes it drains, performs one final whole-segment flush and exits. One thread serves a directory for its whole life; roll moves the handle rather than respawning, and a vanished syncer is warned about once — durability degrades to kernel writeback.

What a graceful stop guarantees, and what durability was never about

Syncer::shutdown_and_join publishes the final watermark, drops the sender and joins. The join is the point: recv returns an error only once the queue is drained and every sender is gone, so the final flush covers the tail, and a shutdown that is about to prune the directory or hand it to a reader finds the bytes on disk. CoreThread::run takes itself by value, so the teardown Snap is appended and the whole journal drops inside the core thread — the final msync completes before CoreHandle::shutdown_and_join returns.

A snap is not more durable than any other record, and never was. The mapped pages belong to the page cache: a process crash, an OOM kill and a clean reboot lose nothing with zero msync; only a hard power cut loses the un-synced tail, a torn tail already stops reading at the last valid record, and the lost tail is repaired by venue reconcile. The blocking checkpoint flush bought ordering against nothing — an msync of one range implies nothing about later pages — and readability against nothing either, since a reader sees the page cache, not the disk. The sync is latency shaping.