Records
The journal's serde schema — the externally-tagged JournalRecord enum every frame carries, its eleven variants and what replay does with each, and the compile-time contract that stops a new variant from being silently dropped at recovery.
Every frame's payload is one JournalRecord, the externally-tagged serde_json enum defined in
crates/vike-core/src/journal/record.rs. Every variant carries a seq — the globally monotonic
record number record_seq reads back — and a now_ms, the wall clock the core stamped for the
dispatch. The writer never serializes an owned JournalRecord: it serializes JournalRecordRef,
a borrowed twin with identical variant and field names, so a hot-path Ingest and a snapshot's
engine slice go to bytes by reference with no clone on the fold. The two must match exactly or a
frame would not deserialize back, which is the first of the hand-checked items below.
The eleven variants
Four of these are commands the replay re-applies; the rest are checkpoints, observations or decision markers that move no fenced state.
| variant | what it is | on replay |
|---|---|---|
Cmd | one exec-lane ingest message — a venue event or a command | re-applied |
Snap | the full-state checkpoint: every engine's snapshot, the client-order-id session and counter, the arm-id counter, the resting conditional books, the OTO/OCO contingency book, the per-mount attribution ledgers, and the state_hash | the restore base, never re-applied |
StrategySubmit | a mounted strategy's already-resolved order intent, journaled at the drain_broker boundary with its mount_id as provenance | re-applied through the same apply_intent site a Cmd order takes |
MintedSubmit | a server-minted submit's resolved request, written right after the coid is minted — the durable tie between a minted coid and its terms | neutral: the mint is re-derived by re-applying the empty-coid submit |
PortfolioSnap | a compact cross-venue portfolio observation on the opt-in CoreConfig::portfolio_snapshot_interval cadence | neutral: carries no engine state and is not a replay base |
ConditionalArmed | an emulated conditional's resolved terms after its arm_id is minted, including a trailing arm's mark-seeded extreme | neutral: the arm replays from its own write-ahead intent |
ConditionalFire | the trigger decision, written ahead of the release it causes, with the released request's coid still empty | re-applied as a submit; the trigger itself is never re-evaluated |
ConditionalDisarmed | an actual disarm, written ahead of the book mutation | neutral: the disarm intent carries the whole arm_id |
MarginCallLiquidate | a margin-call sweep's released reduce-only market order; mount_id is None for the account-wide sweep and Some for a per-mount budget latch's flatten | re-applied as a submit; the breach is never re-evaluated |
GtdExpire | a managed GTD/Day expiry decision, written ahead of the cancel it issues | neutral: cancel_order publishes nothing locally; the venue's OrderCanceled replays as its own Cmd |
ScheduleFire | a wall-clock schedule fire decision, written ahead of the on_schedule it drives | neutral: the orders it produces ride their own StrategySubmit records |
The emulated-order fire and the
margin-call liquidation exist for one reason: each is a runtime
reaction to market data, which is deliberately never journaled, so before ConditionalFire and
MarginCallLiquidate existed a session that fired a stop or a liquidation could not be replayed.
Recording the decision — never the tick or the bar that caused it — lets replay re-mint the
identical order without re-deciding anything. The other two emulated-order records serve readers
rather than replay: ConditionalArmed is the durable tie between the minted arm_id and the
terms the book actually holds, which the write-ahead intent cannot carry for a trailing arm whose
extreme is seeded from a mark; ConditionalDisarmed completes that arm's record lifecycle, so a
reader folding the conditional records can reconstruct the resting book at any point without
re-evaluating a trigger. GtdExpire and ScheduleFire go one step further: their only local effect is journaled
elsewhere, so they are audit markers that let a reader tell a venue-side cancel from a
clock-driven one, and a bar-driven submit from a scheduled one.
What rides the Snap, and why most of it is not hashed
Snap.hash is state_hash over the engine snapshots alone. The books and ledgers that also ride
the record are deliberately outside it, each for a stated reason on its field: the conditional
books can move fenced state only through a fire, and every fire journals its own record; the
contingency book moves it only through a fill that releases a held child or cancels a sibling, and
every such fill is a journaled Cmd; the attribution ledgers are a read-only view folded from
fills that are themselves journaled. Each of those fields is #[serde(default)], which is the
compatibility seam — a Snap written before the field existed reads back empty, and the restore
behaves exactly as it did before the feature, never as an error.
Absent and zero are different things
Snap.arm_seq is an Option: a pre-v7 frame has no such field and reads back None, meaning
absent, never Some(0). The restore path then falls back to a prune-safe bound (the maximum
global record seq plus one). The pre-v7 formula — counting currently readable records — was
prune-unsafe, because prune_before_latest_snap deletes whole early segments, so a restart after
a prune undercounted the ids already spent and re-minted duplicates. Since the arm id is the
disarm key, a duplicate lets a stale disarm remove a fresh stop.
Adding a variant: the no-silent-drop contract
Three independent places interpret this enum — the writer here, the replay fold in the same crate,
and vike_ops::journal_mat's materializer in a different one — and both fold sites used to end in
a _ => catch-all. A variant added to only the writer therefore compiled everywhere, was written
to disk, and was dropped at recovery, so restored state silently diverged from pre-crash state.
Not hypothetical: MarginCallLiquidate carried no mount_id for weeks, so replay restored the
order unattributed and its fill vanished into the residual row. The catch-alls are gone from every
site that makes a per-variant decision, so a new variant is a compile error until each of these
names it and records its answer:
| site | the decision it forces |
|---|---|
record_seq | which field carries the global seq |
vike_ops::journal_mat::record_seq | the same, in the materializer's crate |
replay_from's tail extraction | re-apply on replay, or replay-neutral? |
fold_coid_mounts | does it open, close or leave alone coid-to-mount ownership? |
fold_conditionals | does it change conditional-book membership? |
OrderTracker::fold_record | does it produce an order-log row? |
"Nothing" is a perfectly good answer, written as an explicit arm with a one-line reason; the point
is that it is recorded rather than defaulted. Four things the compiler will not catch are listed on
the enum: the JournalRecordRef mirror, the VERSION bump, the single-variant lookups left as
catch-alls because they ask "is this the Snap?" rather than dispatching, and the materializer's
fill extraction, an if let on Cmd that no exhaustiveness check can reach. The version rule is
on the index: a purely additive variant leaves the readable floor
where it is.
The wire, pinned as bytes
One test in this file compares a frame's bytes against a literal rather than against the types: a
default-account balance snapshot must serialize with no route_key key at all, so a default-account
box writes exactly the frame it always wrote, and the version bump that added the field exists so
a downgraded build refuses such a segment rather than parsing it wrong. The codec around each
payload is on Frames.
The command journal
An opt-in write-ahead journal of every exec-lane message the execution core folds, appended before the fold on pre-allocated memory-mapped segments — so a session can be replayed through the real core, restored after a crash, and proven bit-identical.
Frames
The [len][check][payload] frame codec, the one frame walk every reader layers over, and the three torn-tail causes it tells apart — because a benign post-crash tear and a segment written by a different build deserve opposite readings.