backtest

backtest is the standalone run driver: it reads a harness profile, runs the simulator against the history store and prints a report. It is behind required-features = ["datafusion-store"] (crates/vike-backtest/Cargo.toml) because it opens a concrete DataFusionHist — a default build of the crate compiles none of it.

usage: backtest --profile PATH [--store DIR] [--json]
       backtest --profile SWEEP.toml [--rank-by sharpe|return|max_dd|equity|multi]
                [--search grid|euler] [--euler-depth N]
                [--optimizer tpe [--trials N] [--seed S]]
       backtest --list
       backtest --seed-demo [--store DIR]
       backtest --fetch VENUE:SYMBOL:INTERVAL (--days N | --from LABEL --to LABEL) [--store DIR]
       backtest --fetch-starter [--store DIR]
       backtest --export VENUE:SYMBOL:INTERVAL --out FILE [--from LABEL --to LABEL] [--store DIR]

--profile names the harness profile TOML — the data slice, the engine costs and the strategy — and is required unless --list. --store picks the hist-store root, falling back to $VIKE_HIST_STORE, then the <project>/market_data/hist of a project named outright by $VIKE_SETTINGS_DIR, then <repo>/market_data/hist if the checkout that built the binary is still on the box, then the project the working directory sits in, and finally the per-user data directory (…/vike-data). --json prints the report as pretty JSON instead of the human table. --list prints every registered strategy name and exits: no store, no profile needed. --help and --version are answered on stdout with exit 0, before vike_log::init, because answering either must not create a log directory.

The bin (crates/vike-backtest/src/bin/backtest.rs) is a thin shim over vike_backtest::backtest_cli::run. It is the composition root and holds the two things the library may not: the one std::env::vars() sweep, and the ambient clock — crates/vike-ops/tests/clock_pin.rs's CLOCK_PIN keeps SystemTime::now() out of the library tree, so run takes the timestamp as a parameter.

Getting data into the store first

Three of the verbs exist so a fresh install has something to run.

--seed-demo writes the synthetic demo tape into the store and exits — venue demo, a closed-form curve, explicitly not market data. It is the slice the shipped user_data/profiles/backtest.toml names, so a fresh install can run that profile immediately, and it is safe to re-run: a second seed writes nothing.

--fetch VENUE:SYMBOL:INTERVAL pulls real public bars into the store with no credentials (binance is the only fetchable venue; any other spec is refused by name), bounded by --days N counting back from now or by --from/--to taking epoch milliseconds or YYYY-MM-DDTHH. Re-fetching the same window writes nothing. --fetch-starter downloads the published starter dataset over plain HTTPS — no venue, no credentials, verified against its published SHA256SUMS — for a box a venue cannot be reached from, a geoblock or a locked-down network. Both need the venue-fetch feature; the release binary and the container image have it.

--export VENUE:SYMBOL:INTERVAL --out FILE writes one series from the store to a standalone Parquet file, optionally bounded by --from/--to, for any venue the store holds — demo included. It sits behind the same venue-fetch feature as the two fetch verbs.

Sweeps and search

A profile carrying a [sweep] table (profile.is_sweep()) runs harness::run_sweep instead of a single run_backtest: the strategy runs once per point in the cartesian parameter grid, and the result is a ranked SweepReport table rather than one BacktestReport.

• --rank-by picks the ranking metric, default sharpe, and is ignored — not an error — on a non-sweep profile. --rank-by multi ranks by the composite multi-metric objective instead, and rows gain a score column; the four classic metric names keep the original path and produce byte-identical output.
• --search euler replaces the exhaustive grid with a bounded successive-halving refinement (harness::run_sweep_euler): the coarse grid runs once, then the per-axis step is halved up to --euler-depth times (default 3, capped at 16) around the running best point. Each completed halving doubles the effective resolution for a fraction of the backtests an equally fine grid would cost — at the cost of being a local refinement (it can only descend into the basin the coarse grid already found) and of requiring every axis to be numeric and type-homogeneous. The stderr budget line compares against the grid matching the depth actually reached, so a search that ran out of candidates early reports the smaller, honest saving. --search grid is the default and is completely untouched.
• --optimizer tpe is the Bayesian tree-structured-Parzen-estimator ask/tell search (harness::run_tpe) — the alternative to enumerating the grid, converging in --trials backtests (default 64) by modelling which regions score well. --seed (default 0) makes it fully reproducible. It takes precedence over --search and reuses --rank-by as its objective.

See sweeps, search and objectives for what each of those does.

What a finished run leaves behind

A finished single run also persists: a <run_id> directory under <project>/user_data/runs/ gets manifest.json — the common, kind-agnostic manifest documented on vike_backtest::runs::RunManifest — beside report.json, the same BacktestReport that --json prints. Until this existed the bin saved nothing, so there was no history and nothing a UI could list.

Persistence is additive: the report reaches stdout first, and a persist failure is reported on stderr without changing the exit code, so saving a run can never become a way to lose one. --json stdout is byte-identical to before.

Related: running a backtest, run profiles, runs on disk, the harness registry, the history store.