Sweep, then walk forward

A sweep answers "which parameters scored best on this slice". A walk-forward answers a different question — "was that profitable across the whole sample, or in one lucky stretch" — and it answers it without optimising anything. Running the second after the first is the point of this tutorial; running only the first is how a curve-fit ships.

Both profiles are shipped by vike-cli init (profiles/sweep.toml, profiles/walkforward.toml), and both carry their script INLINE in [strategy.params].src, because neither the sweep nor the walkforward verb takes --script. vike-cli init writes that inlined copy from the same source as strategies/rhai/sma_cross/sma_cross.rhai, so a fresh scaffold agrees — edit one afterwards and you must edit the other.

1. The sweep

[sweep]
fast = [5.0, 10.0, 20.0]
slow = [30.0, 50.0, 100.0]

harness::sweep's expand_sweep turns a profile carrying that table into the cartesian product of single-run profiles — each with strategy.params overridden for its point and its own sweep field cleared, so every expanded point is itself a plain runnable single-backtest profile. A profile with no [sweep] table expands to exactly one point with no overrides, which is why the non-sweep path is unchanged.

vike-cli sweep --profile user_data/profiles/sweep.toml --rank-by sharpe

run_sweep runs each point through run_backtest, keeps a BacktestReport or the error string per point, and ranks the successful rows; failed rows always sort last. --rank-by takes sharpe (default), return, max_dd or equity. The name is validated locally and applied server-side by the canonical RankMetric comparator, so the rows arrive already ordered best-first and the printed statistics are the server's own report numbers.

Three properties worth carrying:

• Determinism does not rest on scheduling. Points run on a rayon pool by default, but par_iter().map().collect() is order-preserving and the ranking sort is a stable slice::sort_by over that same input order, so the bytes are identical to the one-at-a-time loop (parallel_and_sequential_sweeps_are_byte_identical).
• Parallelism multiplies MEMORY, not just speed. The shared Arc<dyn HistStore> shares the handle, not the loaded data: every point re-scans and fully materialises its own Vec<Bar> / Vec<Tick>, so peak RSS is N_threads × slice. The pool is therefore bounded rather than global — sweep_threads defaults to min(4, available_parallelism()) and VIKE_SWEEP_THREADS raises it. Size that against the slice, not the core count. VIKE_SWEEP_SEQUENTIAL=1 pins the process back to the old loop.
• Cost multiplies fast. The shipped grid is 3 × 3 = 9 runs; a fourth axis of five values is 45. Write the values as floats — the shipped profile's own warning — because a non-numeric value is skipped in silence.

2. What the sweep actually reported

The runs below use a smaller grid than the shipped profile, so the whole table fits on the page: 3 × 2 = 6 points over the scaffolded rsi_meanrev script's len and lo axes. The data is real Binance BTCUSDT 1h bars for the first half of 2025, run at cash = 10000, fee_rate = 0.001 and slippage = 0.

[sweep]
len = [7.0, 14.0, 21.0]
lo = [20.0, 30.0]

Through the standalone backtest binary:

backtest --profile tut_rsi_sweep.toml --rank-by sharpe

sweep ranked by sharpe (6 points)
*#1  len=14.0 lo=30.0               ret=0.2129 sharpe=1.7800 max_dd=0.0835 trades=33
#2   len=21.0 lo=30.0               ret=0.1542 sharpe=1.1885 max_dd=0.1341 trades=15
#3   len=14.0 lo=20.0               ret=0.0755 sharpe=0.6565 max_dd=0.1402 trades=12
#4   len=7.0 lo=30.0                ret=0.0246 sharpe=0.2807 max_dd=0.0873 trades=103
#5   len=7.0 lo=20.0                ret=-0.0242 sharpe=-0.1026 max_dd=0.1137 trades=50
#6   len=21.0 lo=20.0               ret=-0.0992 sharpe=-0.6544 max_dd=0.1738 trades=2

Two of the six points lost money, and the table keeps them. Ranking is not filtering: rows 5 and 6 print with their negative returns and negative Sharpes, in last place, because a grid point that failed is evidence about the grid.

The winner returned 21.29% at a Sharpe of 1.7800 over 33 trades. Notice where that ordering comes from — every lo = 30.0 point outranks every lo = 20.0 point. This is a one-axis result. The grid found an entry threshold that the first half of 2025 rewarded, and len only shuffles rows within it.

Read the trade counts before the returns. Row 4 traded 103 times for +2.46%; row 6 traded twice for -9.92%. A Sharpe of -0.6544 computed over two trades is a number, not a measurement — and the same caution applies to the winner's 33.

What the table does not say is anything out-of-sample. All six points were scored on the same bars, and the top row was then chosen by looking at those scores, so 1.7800 is the maximum of six draws rather than an estimate of what len = 14, lo = 30 will do next. That is the gap the walk-forward exists to measure.

The same grid through the service

vike-cli sweep ships the same profile text to a vike-datahub server, which parses it with the one BacktestProfile::from_toml_str and ranks the rows itself:

vike-cli sweep --profile tut_rsi_sweep.toml --rank-by sharpe

parameter sweep — 6 point(s), ranked server-side by sharpe
rank  overrides                             final_equity     return    sharpe    max_dd   trades
   1  len=14.0, lo=30.0                         12128.52    +21.29%    1.7800    0.0835       33
   2  len=21.0, lo=30.0                         11542.23    +15.42%    1.1885    0.1341       15
   3  len=14.0, lo=20.0                         10755.26     +7.55%    0.6565    0.1402       12
   4  len=7.0, lo=30.0                          10246.32     +2.46%    0.2807    0.0873      103
   5  len=7.0, lo=20.0                           9758.30     -2.42%   -0.1026    0.1137       50
   6  len=21.0, lo=20.0                          9008.27     -9.92%   -0.6544    0.1738        2

Same six points, same order, same numbers. Only the presentation differs: the standalone table prints ret as a fraction where the service table prints a percentage, and the service table carries final_equity as well.

Neither client computes a statistic. print_sweep in crates/vike-backtest/src/harness/sweep.rs reads each field straight out of the server's own row, and the standalone table is SweepReport's Display in crates/vike-backtest/src/harness/sweep.rs. The star on the standalone table's first row marks rank 1.

3. The walk-forward

Take the winning row's parameters, put them in [strategy.params], and run:

vike-cli walkforward --profile user_data/profiles/walkforward.toml

harness::walkforward's run_walkforward is assembly, not new engine code: it resolves the strategy through the same strategy_by_name registry, builds the same bar-lane EngineParams and loads the same bars as a plain run_backtest of that profile, then hands the series to walkforward::walk_forward_strategy. So a walk-forward can never disagree with a single run about what the profile's [engine] means — including the fee SCHEDULE and [risk], which the old DTO-shaped path could not carry.

It is BAR mode over ONE series, deliberately: the splitter divides a single &[Bar] by index, so a tick or multi-symbol profile is a clean validation error rather than a silent first-symbol fallback.

[walkforward] n_splits = 4 is the whole configuration. walk_forward_splits cuts the series into n_splits + 1 chunks; window s tests [s·chunk, (s+1)·chunk) with the last window running to the end, and under Anchored mode every window's training range starts at index 0. Each window runs fresh at [engine] cash and the OOS curves are rebased onto one running equity, so the stitched curve compounds.

WalkForwardReport returns the per-window oos_returns, the stitched oos_equity_curve, its oos_return and oos_sharpe, and wf_consistency — the fraction of windows that were profitable out of sample. That last number is the one to look at first: a strong stitched return with a consistency near 0.25 is one window carrying the strategy.

Point both profiles' [data] at a slice your store actually holds before the first run. The shipped sweep names binance over the demo tape's dates, and the shipped walk-forward spans 2024 to 2026; an empty slice is a run with no trades, which reads like a bad strategy rather than an empty query.

4. What the walk-forward actually reported

The profile below carries the sweep's winning row — len = 14.0 and lo = 30.0 — over the identical [data] and [engine] block, plus [walkforward] n_splits = 4. The --addr flag defaults to 127.0.0.1:7878, so a datahub on the same box needs no flag.

vike-cli walkforward --profile tut_rsi_wf.toml

walk-forward: 4 out-of-sample window(s)
window  test_range             oos_return
     1  [869, 1738)               +11.27%
     2  [1738, 2607)               +5.01%
     3  [2607, 3476)               -3.44%
     4  [3476, 4345)               +6.05%

oos_return = +19.64%   oos_sharpe = 1.983   wf_consistency = 75.0%

Window 3 lost 3.44%. That is the result, not a run to repeat until it behaves: one of the four out-of-sample stretches of the first half of 2025 was a losing stretch for these parameters.

wf_consistency = 75.0% is that same fact counted — three of four windows finished above the cash they started with. walk_forward_strategy in crates/vike-backtest/src/walkforward.rs counts a window profitable on a strictly positive oos_return, so a perfectly flat window would count against it too.

The index ranges are the anchored splitter showing its work. The series is 4345 bars; walk_forward_splits in crates/vike-analytics/src/validation.rs sets chunk = n / (n_splits + 1) = 869, window s tests [s·869, (s+1)·869), and the last window runs to the end.

So the first 869 bars are never tested. They are training-only in every window — that is what anchored mode buys, and it is also what the stitched number is missing.

Which makes +19.64% not comparable with the +21.29% of the sweep's top row. The sweep scored the whole slice; the walk-forward scored the last four fifths of it. The same goes for oos_sharpe = 1.983, computed on the stitched curve and so covering the same four windows. The two returns landing near each other is mildly reassuring and nothing more.

What the run does not say is that the strategy is stable. Four windows is four observations, and 75% and 100% are one window apart. At n_splits = 4 over six months of hourly bars each window is about five weeks — short enough for a single trend leg to fill one. Raising n_splits buys more windows, not more data.

Nor is this a forward test. These parameters were chosen by looking at these same bars in the sweep above, so the selection has already seen every window it is now scored on. The honest reading is narrow: with len = 14 and lo = 30 held fixed, this slice produced three up windows and one down one.

Where next

• Sweeps, Search and Walk-forward.
• Bayesian parameter search — when the grid is too big to enumerate.
• Overfitting and Monte Carlo and overfitting.
• Sweep on the service.