The command journal
An opt-in write-ahead journal of every exec-lane message the execution core folds, appended before the fold on pre-allocated memory-mapped segments — so a session can be replayed through the real core, restored after a crash, and proven bit-identical.
The execution core can keep a write-ahead command journal. Off
by default, it costs nothing: crates/vike-core/src/runtime/mod.rs's JournalConfig is None
in the default CoreConfig, no journal is opened, and the fold path is byte-identical. Turned on,
it opens a CommandJournal at construction and journals every exec-lane message before that
message folds. The authority for this tree is crates/vike-core/src/journal/mod.rs, whose
header describes the format as pre-allocated mmap segments with one frame per
record: [len: u32 LE][check: u32 FNV-1a32(payload)][payload: serde_json(JournalRecord)]. A
length of zero means the end of written data (segments are zero-filled at creation), and a record
that fails its check is a torn tail — reading stops at the last valid record and never panics.
What is journaled, and when
The append sits in CoreThread::dispatch, after the wall-clock stamp and before the match that
folds the message, so a crash mid-fold still finds the message on disk. Only exec-lane verbs are
journaled: venue events and commands. Market, bar and tick lanes are not command state and never
touch the journal; a runtime mount or unmount is session topology, recorded in the mount sidecar
instead; Command::Shutdown never reaches the append — it gets a Snap, not a Cmd. The
wall-clock waker is journaled only when a state-mutating sweep can ride it (the stuck-order
watchdog or either dead-man's switch),
because that record is what makes a session refuse to replay. The cadence snapshot decision is
deferred to after the fold, so a Snap reflects the message that triggered it.
It is a write-ahead journal, not a log of what happened
Because the record is appended before ExecutionEngine::on_event folds it, the journal
necessarily contains events the live state machine went on to refuse — an amend-ack racing the
fill that already terminalized the order, for one. crates/vike-ops/src/journal_mat.rs's header
spells out the consequence: any reader that re-derives order state must fold through the real
ManagedOrder::apply, guards included, or it records transitions that never happened. Nothing in
the journal is an observation of the screen; it is the input to the fold, plus checkpoints.
Turning it on
crates/vike-core/src/run_profile.rs's journal_config_from_env resolves the sink for a
production binary. VIKE_RUN_PROFILE=<run.toml> loads a run profile
and its [sinks.journal] section decides — a profile is authoritative even when that means "no
journal", and a set-but-broken profile disables journaling with a warning rather than falling
through. Absent a profile, VIKE_JOURNAL_DIR=<dir> is the single-knob opt-in, with
VIKE_JOURNAL_SNAPSHOT_EVERY=<n> optionally overriding the checkpoint cadence. Neither set means
off. The defaults both paths share are JournalConfig::at's: 64 MiB segments, an async flush
every 256 records, and a full-state Snap every 1024 records (plus always one on shutdown). The
config.journal_dir key exists in config.toml but is declared unconsumed in
crates/vike-config/src/consumed.rs — the directory stays an environment variable until the
three-variable family moves together — so set the variable, not the key.
The compatibility contract
Each segment header stamps a format VERSION. This build writes VERSION and reads and resumes
anything in MIN_READABLE_VERSION..=VERSION (check_version): an older supported journal is
opened and appended to in place, restamped, so upgrading the binary never costs an operator their
live journal. Two things remain hard errors — a version below the floor, and any version above
VERSION, written by a later build that may carry variants this one does not know. The rule that
makes the range safe: a version step may only add a JournalRecord variant, or add a
#[serde(default)]-guarded field to an existing one; anything that renames, removes or reshapes
must raise the floor in the same commit. The range today is 4 through 15, and the module's
readable_version_range_is_4_to_15 test is the tripwire that makes moving either end a reviewed
edit. Determinism is unaffected by reading an older journal: the added variants are ones the older
build never wrote.
Durability, stated honestly
Appending is a bounded memcpy into a mapped page. The msync that forces those pages to disk
costs time proportional to dirty bytes — measured at roughly 8.6 ms for 1 MB, 100 ms for 16 MB and
292 ms (max 432 ms) for a dirty 64 MB segment — and the journal is appended from the fold thread
the latency gate protects. So every blocking msync runs on a
dedicated syncer thread the appender never joins; Sync policy
is that contract. What makes a late sync safe is the durability contract the header cites: the
mapped pages belong to the OS page cache, so a process crash, an OOM kill, memory-pressure eviction
and a clean reboot all lose nothing with zero msync. Only a hard power cut loses the un-synced
tail, and that tail is repaired by the post-replay venue reconcile.
The sync is latency shaping — it bounds how much dirty data one msync ever has to move — not the
durability mechanism. A graceful stop still syncs the tail: CommandJournal's Drop publishes
the final watermark, closes the channel and joins, which is what carries the exit Snap to disk.
One writer
CommandJournal::open takes an exclusive advisory lock on the directory (<dir>/LOCK) and holds
it for the journal's lifetime, so a double-launched instance fails fast instead of interleaving
frames into a live journal. The read-only entry points take no lock at all, so offline inspection
and replay against a directory a live core owns keep working. Locking
carries the argument.
The pages in this tree
The module is a directory with one job per file, and this tree follows it:
| page | what it covers |
|---|---|
| Records | JournalRecord, the externally-tagged enum every frame carries, and the contract for adding a variant |
| Frames | the [len][check][payload] codec, the one frame walk, and the three torn-tail causes |
| Segments | naming, the .prep staging name, pre-allocation, the roll, and the prune |
| Sync policy | the fold-path latency contract and why the syncer is a thread |
| Reading | read_since, the materializer checkpoint, and every reader that takes no lock |
| Replay | deterministic replay through the real core, and the fences |
| Locking | the duplicate-instance interlock |
The payoff is the last of those: a recorded session re-derived through the same single-writer runtime and fenced on a state hash, which both gates the format — any nondeterminism in the fold surfaces as a mismatch — and is the substrate the crash restart restores from.
Alerting
A persisted rule set, a pure evaluator and a delivery seam that run strictly off the hot fold — with a default build that links no trading crate at all, because the watchdog it exists for must be able to page without the core.
Records
The journal's serde schema — the externally-tagged JournalRecord enum every frame carries, its eleven variants and what replay does with each, and the compile-time contract that stops a new variant from being silently dropped at recovery.