Training

How vike-ml trains — the upstream LightGBM binary driven as a child process over a rendered config and a CSV, parameters spelled in LightGBM's own names, the traps the renderer closes, and the measurements behind bin-once-fit-many.

crates/vike-ml/src/lib.rs splits training from inference and says why. train and cli drive the upstream LightGBM CLI binary as a child process: they render LightGBM's own config text, write a data file, spawn the binary and read output_model back. That binary is built once, from a pinned upstream tag, on the box that trains. Inference is compiled into every binary that runs a strategy — including a live daemon signing real orders — and shipping a C++ library there to predict from a model already trained is a cost with no benefit, so Inference is pure Rust over the model text.

Why a process and not a binding

crates/vike-ml/src/cli.rs's header records the bakeoff: every Rust binding drops categorical_feature at dataset construction, silently, and then records it in the saved model's parameter block anyway — an artifact that lies to an auditor. Driving the binary is the only route that gets the feature right, and it costs no crate, no FFI, no unsafe, no cmake or bindgen, and nothing in the cargo deny --all-features surface. crates/vike-ml/src/grid.rs adds the second reason: a binding's Booster::train consumes its dataset, so it re-bins on every fit — a measured 95 ms/fit floor that cannot be amortised — while a process can bin once.

The binary is a parameter. LightGbmCli::new takes a path, reads no environment variable and knows no default location; a binary resolves the path and hands it down — vike-study --lightgbm in the study runner.

The binary cannot name itself, so a file beside it does

LightGbmCli::new refuses a binary whose PROVENANCE file — written by the build recipe beside it — records a tag= other than PINNED_LIGHTGBM_TAG. It reads a file because upstream has no version flag: --version yields an unknown-parameter warning, "No training/prediction data, application quit", and a non-zero exit. The header states the honest weakness: the file is self-reported and only its tag= line is read, so the check catches a missing file and a different recorded tag and nothing else — not a different build of the pinned tag, since nothing in the crate hashes the binary. The equality gate, not this check, is the real proof — Model pins.

run_config returns LightGBM's captured stdout, and the doc says that is not a convenience: Log::Info, Log::Warning and Log::Debug all print to stdout and only Log::Fatal to stderr, so the unknown-parameter warnings that are the only detector for a renamed parameter after a version bump arrive there and nowhere else. A non-zero exit carries the last ten lines of stderr as MlError::Backend; crates/vike-ml/src/error.rs keeps that distinct from Trainer, which says the thing I was told to drive is not what I expect — an operator fault with an operator fix.

The config is the seam

crates/vike-ml/src/train.rs is pure functions — text in, text out — so its traps are testable on a machine with no binary. TrainConfig::render emits the text LightGBM reads; LightGBM emits a model text; parse_model_text reads that. Nothing else crosses between the halves.

TrainData is a row-major &[f64] matrix with f32 labels, LightGBM's own shape. TrainData::new validates once — a whole number of rows, one label per row — rather than letting a child process report a shape error forty lines into a file. It declares no categorical column; with_categorical is what declares one, and it re-runs the same TrainData::validate to refuse an index outside a row. That rule is a refusal, not a clamp, because LightGBM ignores an out-of-range index in silence, bins the column as a number, and the saved model's parameter echo still claims it was categorical. write_csv writes label first, no header, matching LightGBM's defaults; a missing value is spelled nan (MISSING_TOKEN), proven by a smoke that asserts the walker and LightGBM route that row the same way. Values are the shortest decimal that parses back to the same bits, and the renderer writes precise_float_parser=true on every task because LightGBM's default parser clamps the exponent at 308 and is not correctly rounded — the wire is lossless because it is asked for.

render refuses to leave three things to a default: output_model must be named (LightGBM's default writes LightGBM_model.txt into the working directory, which collides the moment two fits share a scratch root); num_threads is always written, always 1; and predict_raw_score is explicit on a predict task, because a margin and a probability are both plausible numbers. A predict file carries a dummy label column, since LightGBM reads column 0 as the label and a label-less file makes an N-feature model see N-1 — and predict_disable_shape_check=true is not the fix; it silences the error while every feature shifts a column.

Parameters in LightGBM's own names

crates/vike-ml/src/params.rs's GbdtParams spells every field as LightGBM does, so a missing field is looked up in LightGBM's docs rather than ours. Defaults are LightGBM's own — binary, 100 iterations, learning_rate=0.1, num_leaves=31, min_data_in_leaf=20 — with departures for determinism: deterministic=true, force_row_wise=true (with both force_* false LightGBM benchmarks the two histogram strategies at runtime and picks by timing, changing the float reduction order), num_threads=1, a pinned seed=42, and verbosity=-1. apply sets one parameter by name and refuses an unknown one — a search axis matching no parameter would sweep a knob that was never connected and report a winner — and refuses an integer that would wrap. One pairing bites: bagging_freq=0 disables bagging however small bagging_fraction is.

A list-valued parameter is a comma-separated string, never brackets

LightGBM's config parser strips quotation marks but not brackets: categorical_feature=0,3 names two columns, while [0,3] — the shape anyone reaches for who has used the Python API — names nothing, the column is binned as numeric, nothing errors, and the model's parameter echo still claims otherwise. The defence is a type: categorical_feature is an Option<String>, and categorical_feature_csv is how indices become one. Indices are 0-based over feature columns and do not count the label.

Bin once, fit many

grid.rs gives the shape this backend was chosen for. A grid varies parameters over a fixed set of datasets, so each is binned once with task=save_binary — about 20 s for 92 of them — and every later fit reads a ~2.2 MB binary instead of re-parsing an 11.6 MB CSV, measured at ~212 ms of write plus ~300 ms of parse per fit; over 22,400 fits that is roughly 260 GB of text I/O that does not happen. The binning config must carry categorical_feature, because binning is where LightGBM decides categorical-versus-numeric and a binary built without it is all-numeric forever.

The second construction-time decision is min_data_in_leaf: under the default feature_pre_filter, binning drops every feature that cannot satisfy it, and a dropped feature leaves the feature_fraction sampling universe. BinPreFilter's doc carries the measurement: a bin built at min_data_in_leaf=10 and trained at 50 differs from the plain CSV fit at 50, and so does a feature_pre_filter=false bin, while a bin built at the fit's own value reproduces the CSV fit byte-for-byte. MinDataAgnostic serves a sweep over that axis; MatchFit is what the real learner's per-fold cache uses, keyed on the value (Models). The mismatch is silent — LightGBM's CheckDataset does not validate min_data_in_leaf.

check_grid_params refuses any num_threads other than 1, with the measurement in the message: 143,293 ms against 2,105 ms at one thread on this data shape, a 68× cliff in the wrong direction. The pinned binary is built without OpenMP so it cannot honour more anyway; the refusal means a binary rebuilt with it does not turn a 30-minute grid into weeks silently.

Logging, and where the binary comes from

grid.rs logs nothing: a tracing event per fit writes 22,400 records into the trace-level file layer, which the console filter does not turn down. A caller reports per fold and sets VIKE_LOG_FILE_LEVEL=warn for a grid run — Log volume has the incident. just lightgbm-build builds the pinned binary into bin/lightgbm/lightgbm on the Linux build box, where a dedicated CI job runs the binary-driven suites — refusing a run in which every gated test self-skipped, because a skipped test passes and 0 passed is the shape of a green job that proved nothing. Off Linux there is no trainer at all, which is why a study that must fit answers NoLearner rather than degrading into a number. Train a model runs the whole path.