vike
ConceptsThe command journal

The command journal

An opt-in write-ahead journal of every exec-lane message the execution core folds, appended before the fold on pre-allocated memory-mapped segments — so a session can be replayed through the real core, restored after a crash, and proven bit-identical.

The execution core can keep a write-ahead command journal. Off by default, it costs nothing: crates/vike-core/src/runtime/mod.rs's JournalConfig is None in the default CoreConfig, no journal is opened, and the fold path is byte-identical. Turned on, it opens a CommandJournal at construction and journals every exec-lane message before that message folds. The authority for this tree is crates/vike-core/src/journal/mod.rs, whose header describes the format as pre-allocated mmap segments with one frame per record: [len: u32 LE][check: u32 FNV-1a32(payload)][payload: serde_json(JournalRecord)]. A length of zero means the end of written data (segments are zero-filled at creation), and a record that fails its check is a torn tail — reading stops at the last valid record and never panics.

What is journaled, and when

The append sits in CoreThread::dispatch, after the wall-clock stamp and before the match that folds the message, so a crash mid-fold still finds the message on disk. Only exec-lane verbs are journaled: venue events and commands. Market, bar and tick lanes are not command state and never touch the journal; a runtime mount or unmount is session topology, recorded in the mount sidecar instead; Command::Shutdown never reaches the append — it gets a Snap, not a Cmd. The wall-clock waker is journaled only when a state-mutating sweep can ride it (the stuck-order watchdog or either dead-man's switch), because that record is what makes a session refuse to replay. The cadence snapshot decision is deferred to after the fold, so a Snap reflects the message that triggered it.

It is a write-ahead journal, not a log of what happened

Because the record is appended before ExecutionEngine::on_event folds it, the journal necessarily contains events the live state machine went on to refuse — an amend-ack racing the fill that already terminalized the order, for one. crates/vike-ops/src/journal_mat.rs's header spells out the consequence: any reader that re-derives order state must fold through the real ManagedOrder::apply, guards included, or it records transitions that never happened. Nothing in the journal is an observation of the screen; it is the input to the fold, plus checkpoints.

Turning it on

crates/vike-core/src/run_profile.rs's journal_config_from_env resolves the sink for a production binary. VIKE_RUN_PROFILE=<run.toml> loads a run profile and its [sinks.journal] section decides — a profile is authoritative even when that means "no journal", and a set-but-broken profile disables journaling with a warning rather than falling through. Absent a profile, VIKE_JOURNAL_DIR=<dir> is the single-knob opt-in, with VIKE_JOURNAL_SNAPSHOT_EVERY=<n> optionally overriding the checkpoint cadence. Neither set means off. The defaults both paths share are JournalConfig::at's: 64 MiB segments, an async flush every 256 records, and a full-state Snap every 1024 records (plus always one on shutdown). The config.journal_dir key exists in config.toml but is declared unconsumed in crates/vike-config/src/consumed.rs — the directory stays an environment variable until the three-variable family moves together — so set the variable, not the key.

The compatibility contract

Each segment header stamps a format VERSION. This build writes VERSION and reads and resumes anything in MIN_READABLE_VERSION..=VERSION (check_version): an older supported journal is opened and appended to in place, restamped, so upgrading the binary never costs an operator their live journal. Two things remain hard errors — a version below the floor, and any version above VERSION, written by a later build that may carry variants this one does not know. The rule that makes the range safe: a version step may only add a JournalRecord variant, or add a #[serde(default)]-guarded field to an existing one; anything that renames, removes or reshapes must raise the floor in the same commit. The range today is 4 through 15, and the module's readable_version_range_is_4_to_15 test is the tripwire that makes moving either end a reviewed edit. Determinism is unaffected by reading an older journal: the added variants are ones the older build never wrote.

Durability, stated honestly

Appending is a bounded memcpy into a mapped page. The msync that forces those pages to disk costs time proportional to dirty bytes — measured at roughly 8.6 ms for 1 MB, 100 ms for 16 MB and 292 ms (max 432 ms) for a dirty 64 MB segment — and the journal is appended from the fold thread the latency gate protects. So every blocking msync runs on a dedicated syncer thread the appender never joins; Sync policy is that contract. What makes a late sync safe is the durability contract the header cites: the mapped pages belong to the OS page cache, so a process crash, an OOM kill, memory-pressure eviction and a clean reboot all lose nothing with zero msync. Only a hard power cut loses the un-synced tail, and that tail is repaired by the post-replay venue reconcile. The sync is latency shaping — it bounds how much dirty data one msync ever has to move — not the durability mechanism. A graceful stop still syncs the tail: CommandJournal's Drop publishes the final watermark, closes the channel and joins, which is what carries the exit Snap to disk.

One writer

CommandJournal::open takes an exclusive advisory lock on the directory (<dir>/LOCK) and holds it for the journal's lifetime, so a double-launched instance fails fast instead of interleaving frames into a live journal. The read-only entry points take no lock at all, so offline inspection and replay against a directory a live core owns keep working. Locking carries the argument.

The pages in this tree

The module is a directory with one job per file, and this tree follows it:

pagewhat it covers
RecordsJournalRecord, the externally-tagged enum every frame carries, and the contract for adding a variant
Framesthe [len][check][payload] codec, the one frame walk, and the three torn-tail causes
Segmentsnaming, the .prep staging name, pre-allocation, the roll, and the prune
Sync policythe fold-path latency contract and why the syncer is a thread
Readingread_since, the materializer checkpoint, and every reader that takes no lock
Replaydeterministic replay through the real core, and the fences
Lockingthe duplicate-instance interlock

The payoff is the last of those: a recorded session re-derived through the same single-writer runtime and fenced on a state hash, which both gates the format — any nondeterminism in the fold surfaces as a mismatch — and is the substrate the crash restart restores from.

On this page