The fit cache

The cross-run cache over the learner seam — a validation score content-addressed by everything the fit consumed, why a wrong hit is worse than no cache, and what the key deliberately leaves to the caller.

crates/vike-ml/src/fit_cache.rs is the cross-run fit cache: a validation score that a previous run already computed for these training bytes, this hyperparameter point, this seed, this validation half and this objective is not refitted. It sits over the learner seam described in Models, and crates/vike-ml/src/lib.rs's header explains why it lives in the library rather than in a study — a cache over a learner is library work, and a second consumer of the seam would otherwise have to rewrite it.

Why it exists, measured

The module header carries the numbers. Fitting is the whole cost of a hyperparameter search: one 20-trial run of the study that first needed this cache was about 1,591 LightGBM fits at a p50 of 6.9 s each, spawn and I/O were 2.3% of a fit, so wall clock is fits divided by parallelism and the only lever is fewer fits. Cross-run duplication is real for any search: a seed-deterministic TPE startup deck means a 20-trial and a 50-trial run at the same search seed share every startup point per fold, seed sweeps overlap on grid points, and a re-run of an identical config duplicates everything. Those are searches over model fits — the header places them in the cohort study that first needed the cache — not the strategy-parameter sweeps over backtests that Search describes.

The constraint that outranks everything

The header states it in one line: a wrong cache hit silently corrupts results and is worse than no cache. So the key content-addresses the actual bytes the fit consumes, never descriptions of them — not a path, not a fold id, not a config name. score_key is a SHA-256 over three parts, in an order the doc says is frozen because it is the address of every entry in every existing on-disk cache:

The fit identity, produced by the learner through Learner::fit_identity (crates/vike-ml/src/learner.rs), because only the learner knows what its trainer eats. For the real learner, crates/vike-ml/src/gbdt_learner.rs's GbdtLearner, the header lists what it covers: the training CSV bytes exactly as TrainData::write_csv streams them to the binning, the rendered train config and the rendered save-binary config with their path lines normalised (every searched axis, the fit seed, the categorical declaration and every non-searched parameter among them), the model text-format version, and the trainer binary's PROVENANCE file bytes. The trait doc adds the contract: two calls may share an identity only when their fits are bit-identical by construction, and a shape fit would refuse gets no identity, so a refusal is never served from a cache.
The validation half's own bytes — n_rows, n_cols, every feature's bit pattern and every label: the exact in-process input to predict_proba and to whatever scores its output.
The objective identity — an opaque digest the caller supplies, for the same reason the learner supplies the first part.

Two hashing tools make that possible. Transcript is a domain-separated transcript in which every part enters as its own fixed-width digest, so two part sequences with equal concatenations can never collide — the framing a plain running hash lacks. DigestWriter is an io::Write that hashes instead of storing, which is what lets an ~8 MB training CSV enter the key as its exact bytes without ever materialising; its f64_bits hashes the bit pattern, not the value, because -0.0 and 0.0 are different bytes on every wire a trainer reads and NaN must hash stably.

What the crate deliberately does not own

The objective is a digest this crate cannot compute, and the header says that is structural: the workspace's scorers live in vike-analytics, which declares the same layer as vike-ml, and the layer gate forbids the edge. The caller hashes its own objective and every parameter it scores with, using Transcript and DigestWriter, and hands the result in.

A learner that cannot prove its inputs' identity answers None from fit_identity — the default — and is simply never cached: a refit, never a wrong answer. crates/vike-ml/src/test_support.rs's header records that the crate's own test double, ScriptedLearner, answers None deliberately, because the only way a scripted double could key on its own behaviour is a caller-supplied salt, and two doubles scoring differently at one salt would share every key — exactly the wrong hit. crates/vike-ml/tests/learner_double.rs pins both directions: a cache over the double never hits, and the same cache does hit for a learner that answers an identity.

Content-addressing is exactly as reproducible as its inputs

The header records that a data source which does not answer one request the same way twice — parallel-reduction noise in a server-side aggregate — gives every re-run a fresh key for every fit, stores thousands of entries and grows a directory that is never read. That happened for four weeks to the first consumer. The cure was to canonicalise at the parse boundary (crates/vike-backfill/src/vikedata/parse.rs's canonical_notional), and it deliberately does not live in the key: snapping inside the key would declare two genuinely different datasets equivalent, the wrong hit this design refuses.

The frozen tag

KEY_SCHEMA is the key's domain tag, and its value is the string vike-research fit-cache v1. Its doc says the spelling is frozen on purpose: renaming it is a bump — it re-keys every entry in every existing on-disk cache — and a move that changes no keyed byte has no business charging a user thousands of refits for a tidier constant. The crate it names has been dissolved and deleted, and that changes nothing; a domain tag is allowed to name something that no longer exists. the_key_bytes_are_frozen pins the whole key over a fixed vector against a digest computed outside this implementation. The one legitimate reason to bump it is the declared residual: the in-process inference and the scorer bodies are code, not data, so a change to them moves scores without moving any keyed byte. gbdt_learner.rs's fit-identity tag, vike-research fit-identity v1, is governed by the identical argument.

The value, the directory, the failure modes

The value is the validation score — one f64 — never the model file: only the score enters an argmin, and a caller's final refit, whose model is consumed, is deliberately not cached. Entries are one file per key, named by the key's hex, holding a magic line and the score's 64 bits as 16 hex digits (entry_bytes, read back by a strict whole-file parse_entry).

FitCache::dir_beside owns the one location rule that is a property of the cache: the directory is a sibling of the caller's scratch root, never a child, so a scratch wipe does not empty it and every run of every config shares one pool. The directory's name is the caller's, because it is part of an application's on-disk layout and renaming it orphans a user's accumulated entries as surely as a KEY_SCHEMA bump invalidates them.

A corrupt or uninterpretable entry is a miss — deleted on the way to None and refitted — never an error, because a cache must not be able to fail a run. A non-finite score is never persisted: a caller that folds a deterministically rejected point and a transient spawn failure into the same INFINITY would otherwise make the transient permanent for that key. Two threads racing on one key both fit and both store; the values are equal by construction, so last-write-wins through store's atomic rename is benign, with a remove-and-retry fallback for Windows, which refuses a rename over an existing file. note_hit and note_miss are public because the code that decides a hit is a hit lives in whichever crate owns the search loop; FitCacheStats is what a run summary line prints so reuse is visible.

The cost, per lib.rs: the crate now owns a hash, and its dependency list grew from vike-model alone to vike-model plus sha2 and hex — both already linked into every binary in the workspace by vike-bridge-core's signer, so nothing new entered the supply-chain audit surface.