The signal backtest

The vectorized backtest for sweeping signal variants — a position series and a log-return series in, per-bar PnL, equity and trade statistics out — plus the frictioned scorer a model search ranks by.

crates/vike-analytics/src/signal_backtest.rs is about a hundred lines of arithmetic. The event-driven simulator in backtesting — orders, a broker, real fee, slippage and adverse-fill semantics — produces the more honest number, and the header says it is the wrong tool for sweeping hundreds of signal variants: reaching a hundred lines should not cost 32,000 lines of simulator.

The one invariant

pnl[t] = position[t − 1] · log_ret[t]. The position must be the one held before the return was known. The header states it as the line everything else rests on: every lookahead bug in a signal backtest is a violation of it.

The kernels

apply_min_hold(positions, n_bars) locks a position in for n_bars after every entry or flip. When the position becomes non-zero at bar i — from flat, or by changing sign — bars i ..= i + n_bars − 1 are forced to that value and any signal change inside the window is ignored, including a flip to the opposite sign; a minimum hold that any opposite signal could break would not be a minimum hold. The comparison against the previous bar reads the already-overwritten output, not the input — deliberate, matching the oracle: a bar forced by a lock is, for detecting the next entry, the position that was actually held.

bar_pnl(positions, log_ret, slippage_bps) applies the invariant and subtracts slippage_bps on every position change. The series starts flat, so entering at bar 0 is charged; so is an exit to flat. A NaN return is 0.0 PnL for that bar, and a change on that bar is still charged. A flip — long straight to short — costs exactly one slippage unit, not two; the doc says that is faithful to the oracle's 0/1 change indicator and is not a bug to "fix". Mismatched lengths truncate to the shorter.

equity_from_pnl(pnl) is exp(cumsum(pnl)) from 1.0 at the first bar. A NaN contributes 0.0, a declared divergence from the pandas oracle at that one bar — pandas emits a hole and resumes; this emits a finite, stale value — reachable only from a hand-built series, since bar_pnl never emits NaN. The exponentiation is libm::exp, and the crate header records why: the platform's own exp differs between the Windows desktop and the Linux boxes by up to one ulp, and this function — the equity curve — was measurably among the casualties, so a backtest's headline number depended on which box ran it.

trade_stats(positions, pnl) returns a SignalTradeStats: n_trades counts entries and flips (an exit to flat is not a new trade), avg_holding_bars is the mean run length of consecutive identical non-zero positions, and win_rate and avg_pnl are over non-flat bars only. Pass same-length slices: the first two walk the full position series while the last two walk a zip that truncates to the shorter, so a mismatch computes the two halves over different horizons with no error.

Two numbers that look like ones you already have

sharpe_from_bar_pnl is not metrics::sharpe, and max_drawdown_signed is not metrics::max_drawdown

sharpe_from_bar_pnl(pnl, periods_per_year) is mean / sd · √periods_per_year over the PnL directly, with the sample standard deviation (ddof = 1), and it never exponentiates. metrics::sharpe takes an equity curve and derives simple per-bar returns from it. Both are correct in their own vocabulary and they disagree numerically, so the input is in the name — and a test exists solely so that substituting one for the other fails loudly instead of silently moving the headline. ddof = 1 is load-bearing: it reproduces the oracle's Series.std() default, and the survival gate compares the result against a fixed threshold. Fewer than two non-NaN observations, or a zero or non-finite deviation, return 0.0 — zero fails every threshold. max_drawdown_signed(equity) is the same story: it returns the oracle's dd.min(), negative, and is what the parity gate compares against 271 stored runs, while metrics::max_drawdown returns the positive magnitude the tearsheet renders. Do not unify them: the sign of the second is user-visible.

The scorer a model search ranks by

SharpeObjective bundles bar_returns, bands, min_hold_bars, slippage_bps and periods_per_year; neg_sharpe_mini_backtest(proba, &objective) runs the frictioned pipeline verbatim — hysteresis → apply_min_hold → bar_pnl → sharpe_from_bar_pnl — and negates it, so a model search can argmin. It ranks a candidate model by the Sharpe of a backtest of its own out-of-sample probabilities rather than by calibration against labels, in the frictioned shape so a churny candidate pays for its churn at selection time exactly as it would out of sample. Pass the same periods_per_year the reported backtest uses, or the two Sharpes cannot be compared.

No evidence scores worst, explicitly

A candidate is ranked by its Sharpe only when the mini-backtest produced evidence: it must trade (at least one non-flat bar after the min-hold), and its PnL must have a defined sample Sharpe (at least two bars, positive finite variance). Anything else scores INFINITY, the worst an argmin can receive. This cannot be left to sharpe_from_bar_pnl, which answers 0.0 for exactly these cases — right for threshold callers, and exactly wrong for an argmin, where 0.0 ranks a candidate that never opened a position above every negative-Sharpe candidate that traded. Not an edge case: with the default 0.75/0.25 entry bands over a hundred-odd-row slice, an all-flat candidate is common, and a search without this rule keeps answering "the model that never trades". A single-trade slice is not degenerate — one entry then holding gives n − 1 market bars and a defined variance.

Two artifacts a caller accepts by ranking with it, identical for every candidate and so fair across them: the fold starts flat at the slice's first row, and the slice is scored as one series even where it crosses a seam such as an asset boundary. The caller's own per-span folding is the reported backtest; this is a ranking device. The Rust study contract honours the rule from the other side: vike-user-research's StudyOutcome::metric accepts a non-finite value precisely because this scorer deliberately scores a degenerate slice INFINITY, and rounding it to zero would turn a gap into an observation.

Why the scorer lives here and not with the learner

neg_sharpe_mini_backtest ranks a model, so vike-ml looks like its home. It cannot go there, and the header says that is a merge gate rather than a judgement call: the body names only symbols from this crate and both crates declare layer = 20, while crates/vike-ops/tests/layer_gate.rs demands a strictly lower layer for every normal vike-* dependency (see architecture). Which is the right answer anyway — the scorer is arithmetic over a position series and a return series, which is exactly what this module is. The learner side keeps only the choice of which score ranks its candidates.

The survival statistics take this module's per-bar PnL as their input.